Link

DOCKER CONTAINER LOG COLLECTOR

Zebrium’s docker container log collector collects container logs and sends logs to Zebrium for automated Incident detection. Our github repository is located here.

Note: A non-containerized Docker log collector based on Fluentd is also avalaible here

ze-docker-log-collector

Getting Started

Docker

Use the following command to create a docker log collector container:

sudo docker run -d --name="zdocker-log-collector" --restart=always \
                -v=/var/run/docker.sock:/var/run/docker.sock \
                -e ZE_LOG_COLLECTOR_URL="<ZE_LOG_COLLECTOR_URL>" \
                -e ZE_LOG_COLLECTOR_TOKEN="<ZE_LOG_COLLECTOR_TOKEN>" \
                -e ZE_HOSTNAME="<HOSTNAME>" \
                -e ZE_DEPLOYMENT_NAME="YOUR_DEPLOYMENT_NAME_HERE" \
                zebrium/docker-log-collector:latest

NOTE:

The ZE_DEPLOYMENT_NAME label essentially defines a failure domain boundary for anomaly correlation. This allows you to collect logs from multiple applications and isolate the logs of one application from another application so as not to mix these in a Root Cause Report. This is referred to as Service Groups in the Zebrium UI.

Docker Compose

Use the following configuration file to deploy via docker-compose command:

version: '3.5'

services:
  zdocker-log-collector:
    image: zebrium/docker-log-collector:latest
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      ZE_LOG_COLLECTOR_URL: "<ZE_LOG_COLLECTOR_URL>"
      ZE_LOG_COLLECTOR_TOKEN: "<ZE_LOG_COLLECTOR_TOKEN>"
      ZE_DEPLOYMENT_NAME: "<YOUR_DEPLOYMENT_NAME_HERE>"
      ZE_HOSTNAME: "<HOSTNAME>"

AWS Elastic Container Service (ECS)

Add the following service to ECS on EC2 cluster configuration.

services:
  zdocker-log-collector:
    image: zebrium/docker-log-collector:latest
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      ZE_LOG_COLLECTOR_URL: "<ZE_LOG_COLLECTOR_URL>"
      ZE_LOG_COLLECTOR_TOKEN: "<ZE_LOG_COLLECTOR_TOKEN>"
      ZE_DEPLOYMENT_NAME: "<YOUR_DEPLOYMENT_NAME_HERE>"

To collect container logs from all nodes in an ECS cluster, zdocker-log-collector service must be configured to run as an ECS daemon task. Please follow the steps below to configure the daemon task:

  1. Log in to the AWS console and navigate to the ECS Clusters section. Click into your cluster you run the Agent on.
  2. Choose Service tab, click on the Create button.
  3. For launch type, select EC2, for service type, select DAEMON, type a service name, and click on Next step.
  4. For Load balance type option, select None, and click on Next step. On next page, select Next step without configuring Auto Scaling.
  5. Review and click on Create Service.

Please note ECS tasks must be configured to use ‘json-file’ Log Driver for Zebrium log collector to receive container logs. If there is special log configuration on ECS instances, for example, using UserData section on instance to set log configuration, those configurations may need to be modified or deleted.

Environment Variables

The following environment variables are supported by the collector:

Environment Variables Description Default value Note
ZE_LOG_COLLECTOR_URL Zebrium log host server URL None. Must be set by user Provided by Zebrium once your account has been created.
ZE_LOG_COLLECTOR_TOKEN Authentication token None. Must be set by user Provided by Zebrium once your account has been created.
ZE_HOSTNAME Hostname of docker host Empty. Optional If ZE_HOSTNAME is not set, container hostname is used as source host for logs.
ZE_MAX_INGEST_SIZE Maximum size of post request for Zebrium log server 1048576 bytes. Optional Unit is in bytes
ZE_FLUSH_TIMEOUT Interval between sending batches of log data to Zebrium log server. 30 seconds. Optional Unit is in seconds. Please note Zebrium output plugin sends data immediately to log server when accumulated data reaches ZE_MAX_INGEST_SIZE bytes.
ZE_FILTER_NAME Collect logs for containers whose names match filter name pattern. These can include wildcards, for example, my_container1* Empty. Optional
ZE_FILTER_LABELS Collect logs for containers whose labels match the labels as defined in ZE_FILTER_LABELS. The format is: label1:label1_value,label2:label2_value These can include wildcards, for example, my_label:xyz* Empty. Optional

Testing your installation

Once the docker log collector software has been deployed in your environment, your container logs and incident detection will be available in the Zebrium UI.

Contributors

  • Brady Zuo (Zebrium)