The YOUnite ecosystem uses the Open Source Elastic Stack (https://www.elastic.co/products) to log (Logstash) , search (Elasticsearch), analyze and visualize (Kibana) all API requests and message bus traffic.
For development, a combined Elastic stack docker image that contains Elastic, Logstash, and Kibana is used. However, for production deployments, it is suggested that an organization run a three node cluster; this requires purchasing three node licenses from Elastic.
This section covers configuring the Elastic stack for logging and dashboards.
The default docker-compose.yml file contains most of the required configuration, with the exception of some configuration required for the Elastic and Kibana portions.
Elastic-specific configuration is in the elasticsearch.yml file. In the dev environment, this is done via /open-mdm/docker/elk/elasticsearch.yml.
Currently the changes to the file from the defaults are:
- action.auto_create_index: -mdm-* This is added to disallow the creation of indices on the fly, forcing mdm's logging system to create the index definition with the appropriate mappings. This may change if templates are used for the mappings, depending on how we choose to handle changes when future versions roll out.
Kibana specific configuration is in the kibana.yml file.
- elasticsearch.url: http://elk:9200 This is added to force Kibana to go back through the OAuth proxy for hits to Elastic, instead of the default of localhost. The name of the OAuth proxy is from of the "elk-backend" (the actual combined Elastic stack image) is "elk."
Background on How the YOUnite UI Makes the Dashboard Connection to Kibana
When a user clicks a link to display logging information via Kibana, the URL is modified as it is passes through authorization code in the NGINX proxy. The modification uses the information in the OAuth check_token response to pass along the Elastic alias that Kibana will use to populate the dashboard's data. Access to Kibana is allowed if a user has the correct permissions for Kibana access. By default all users have this permission. Logging information that is displayed/available is controlled through the usage of either the alias for the user's UUID, which contains all the indices they have access to, or via Kibana searches hitting specific indices to which the user has direct permissions (also encoded in the OAuth token).