Docker Logging Through Syslog

You can send syslog from your applications to Loggly by linking them with the Loggly Docker container. It uses ryslog to listen for syslog events and then forwards them to Loggly. Docker will automatically manage the port mapping. It’s made available by SendGrid Labs on GitHub or Docker Hub. These instructions were tested with Docker client version 1.3.0 and nginx 1.7.7. For alternatives, see the Advanced Options below.

Docker Syslog and Logging Container Diagram

Docker Syslog Setup

  1. Start the Loggly Docker Container
  2. Run the following command to download and run the Loggly docker container. The rsyslog daemon is running inside this container and will send syslog to Loggly. This will also open a high numbered port on the host machine, which maps to port 514 inside the container where rsyslog will receive it and send it to Loggly.

    sudo docker run -d -p 514/udp --name loggly-docker -e TOKEN=TOKEN -e TAG=Docker sendgridlabs/loggly-docker
    

    Replace:

    Note: You may get some standard output like shown below. You can ignore this and rsyslog will continue running as usual.

    rsyslogd: imklog: cannot open kernel log(/proc/kmsg): Operation not permitted.
    rsyslogd: activation of module imklog failed [try http://www.rsyslog.com/e/2145 ]
    

  3. Send Test Logs From The Host
  4. You can verify the container is running and see the port that Docker has opened up for syslog by running

    sudo docker ps -a
    

    Here you can see it’s redirecting port 49154 on the host to port 514 in the container

    CONTAINER ID        IMAGE                               COMMAND             CREATED             STATUS                       PORTS                             NAMES
    1a9d7496ae42        sendgridlabs/loggly-docker:latest   "/tmp/run.sh"       18 minutes ago      Up 5 seconds                 514/tcp, 0.0.0.0:49154->514/udp   loggly-docker 
    

    If you send test messages to the host’s port, they will be sent to Loggly

    echo netcat:"Host test log" | nc -u -w 1 127.0.0.1 UDP_PORT
    

    Replace:

    • UDP_PORT: the high numbered port as shown above that maps to port 514 inside the loggly container

  5. Link To Other Containers
  6. You can link other containers to Loggly’s container so that all your syslog gets sent to Loggly. Docker will automatically inject environment variables telling you the IP and Port to send syslog to.

    In this example, we will configure an Nginx container to send syslog to Loggly. Run the Nginx Docker container in interactive terminal mode and link it with the running loggly-docker container

    sudo docker run -i -t --name nginx --link loggly-docker:loggly nginx /bin/bash
    

    Now run the following command inside the container to check the linked environment variables.

    env | grep LOGGLY_PORT_514_UDP
    

    Here is an example output with the values for my variables. Yours will be different.

    LOGGLY_PORT_514_UDP_PORT=514
    LOGGLY_PORT_514_UDP_ADDR=172.17.0.5
    

  7. Send Test Logs From Nginx Container
  8. Try sending a test event from inside the Nginx container. You can use netcat to confirm the link is working and that logs are reaching to Loggly.

    apt-get install netcat
    echo netcat:"Nginx test log" | nc -u -w 1 $LOGGLY_PORT_514_UDP_ADDR $LOGGLY_PORT_514_UDP_PORT
    

  9. Configure Nginx for Syslog
  10. We can change nginx’s configuration to log over syslog to our Loggly container instead. Here’s how you can edit the Dockerfile of the existing Nginx container.

    Insert the line below in the Nginx Docker file before the command which starts the Nginx server inside the container. This will configure the Nginx container to send logs to the “loggly” link we just set up.

    RUN sed -i "s/server {/server {\n \n  error_log syslog:server=loggly;\n    access_log syslog:server=loggly;\n/" /etc/nginx/conf.d/default.conf
    

    Build the image and run your container normally. Then, visit a webpage from the host machine to generate a log. The access log event should show up inside Loggly.

  11. Verify Events
  12. Search Loggly for events with the Docker tag over the past 20 minutes. It may take a few minutes to index the events. If it doesn’t work, see the troubleshooting section below.

    tag:Docker
    

    nignx docker

Advanced Docker Syslog Options

Docker Logging Troubleshooting

If you don’t see any data show up in the verification step, then check for these common problems.

    Check Docker Container:

  • Wait a few minutes in case indexing needs to catch up
  • Verify the container is running and that it has mapped port 514 by running sudo docker ps -a
  • Send test events from inside each of the containers and from the host to see which point in the chain is dropping logs
  • Make sure your app is sending syslog to the injected environment variable address instead of the usual syslog address
  • See our Rsyslog Troubleshooting Guide if the files are not being sent to Loggly
  • Still Not Working?

  • Search or post your own Docker logs, Docker daemon, or other Docker question in the community forum.

Docker Logging

Thanks for the feedback! We'll use it to improve our support documentation.


Top