Deploying Logging

The YOUnite ecosystem uses the Open Source Elastic Stack (https://www.elastic.co/products) to log (Logstash), search (Elasticsearch), analyze, and visualize (Kibana) all API requests and message bus traffic.

For development, a combined Elastic stack Docker image that contains Elastic, Logstash, and Kibana is used. However, for production deployments, it is suggested that an organization run a three-node cluster; this requires purchasing three node licenses from Elastic. 

Configuring the Elastic stack for l]Logging and Dashboards.

The default docker-compose.yml file contains most of the required configuration, with the exception of some configuration required for the Elastic and Kibana portions. 

Elastic-specific configuration is in the elasticsearch.yml file. In the dev environment, this is done via  /open-mdm/docker/elk/elasticsearch.yml.

Currently the changes to the elasticsearch.yml file from the defaults are:

• action.auto_create_index: -mdm-* This is added to disallow the creation of indices on the fly, forcing MDM's logging system to create the index definition with the appropriate mappings. This may change if templates are used for the mappings, depending on how we choose to handle changes when future versions roll out.

Kibana-specific configuration is in the kibana.yml file:

• elasticsearch.url: http://elk:9200 This is added to force Kibana to go back through the OAuth proxy for hits to Elastic, instead of the default of localhost. The name of the OAuth proxy is from of the "elk-backend" (the actual, combined Elastic stack image) is "elk."