Docker logging through Logspout
You can send Docker container logs to Loggly with the help of Logspout. This setup doesn’t support multiline logs, each line will be treated as a separate log event. This setup allows sending logs from multiple containers to a centralized guest container from which Logspout can then send to Loggly. All the logs from the guest container will be forwarded to Logspout automatically until the guest container is restarted or exited, in which case, you will have to repeat the procedure again for centralizing logging. This setup has been tested with Docker version 1.13.1 and Logspout version 3.2.4.
Docker Logspout Setup
Run the Logspout container
Run the command below to start the Logspout container and configure it to send logs to Loggly.
docker run --name logspout -d --volume=/var/run/docker.sock:/var/run/docker.sock -e SYSLOG_STRUCTURED_DATA="TOKEN@41058 tag=\"Logspout\"" gliderlabs/logspout syslog+tcp://logs-01.loggly.com:514
- TOKEN: enter your customer token from the Source Setup page in Loggly
Go to the Search tab in Loggly and search for events with the Logspout tag over the last 20 minutes. It may take a few minutes to index the events. If it doesn’t work, see the troubleshooting section below.
Note: The container name and ID of the container will be parsed as syslog.appname and syslog.host respectively.
Centralization of logs from multiple containers
If you want to send multiple container logs to Loggly, you don’t need to set up each container to send the logs. The already running Logspout container will take care of other container’s logs and will forward them with its own container logs to Loggly.
“The logspout container has the ability to gather logs from other containers that are started without the -t option and are configured with a logging driver that works with docker logs (journald and json-file).”
Example for centralizing nginx server logs with Logspout
Step 1. You should already have started the Logspout container and you can see the running container status by running the command below:
root@ip-172-31-57-234:/home/ubuntu# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 4280a44b3a0a gliderlabs/logspout "/bin/logspout sys..." 3 minutes ago Up 3 minutes 80/tcp logspout root@ip-172-31-57-234:/home/ubuntu#
Step 2. Simply Run the nginx container using the command below.
docker run --name nginx -d -p 8080:80 nginx
You can now see that both Logspout and nginx containers are running.
root@ip-172-31-57-234:/home/ubuntu# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9f6e5e63fb4c nginx "nginx -g 'daemon ..." 1 second ago Up 1 second 0.0.0.0:8080->80/tcp nginx 4280a44b3a0a gliderlabs/logspout "/bin/logspout sys..." 4 minutes ago Up 4 minutes 80/tcp logspout root@ip-172-31-57-234:/home/ubuntu#
Step 3. Your nginx container has started and is open to listen at port 8080. Now you are all set to generate nginx server logs by hitting the localhost URL with the 8080 port and logspout will ship all the logs to Loggly.
Search Loggly for events with the Logspout tag over the last 20 minutes. It may take a few minutes to index the events. If it doesn’t work, see the troubleshooting section below.
You should see tag: Logspout which shows that nginx server logs are coming from the Logspout container. The logtype is nginx because the logs are nginx. You will also notice the container name and ID in syslog.appName and syslog.host, respectively. You can easily differentiate between different containers and their logs by looking at the container IDs.
- Docker Logging Strategies – See the top ways to log on Docker and pick the best one for you
- Linking Docker Containers – More details on how container linking works in Docker
- Loggly Libraries Catalog – You can bypass Docker and log from applications directly using our HTTP/S libraries
- Search or post your own Docker syslog and Docker logging question in the community forum.
- If you want a buffer or a queue that can store your logs during outages then use the logspout container
Docker Logging Troubleshooting
If you don’t see any data show up in the verification step, check for these common problems.
Check Docker Container:
- Wait a few minutes in case indexing needs to catch up
- Verify the container is running by running sudo docker ps -a
- Make sure your machine has Rsyslog installed on it
- Send test events from inside each of the containers and from the host to see which point in the chain is dropping logs
- You can only relay the other container’s logs to logspout which supports “json-file” or “journald” driver because only “json-file” and “journald” driver supports the “docker logs” command. Read more on limitations-of-logging-drivers
Still Not Working?
- Search or post your own Docker logs, Docker daemon, or other Docker question in the community forum.