LoggingThe Ultimate Guide

your open-source resource for understanding, analyzing, and troubleshooting system logs

curated byloggly

0

Centralizing Node Logs

When you’re logging in applications, a good practice to follow is centralizing logs. What this means is to store them in a central location. This includes things such as shared file systems; servers such as MySQL/PostgreSQL, Riak, and Cassandra; as well as remote logging services, such as Loggly.

Benefits of Centralizing Logs

Keeping all of the log data in a central location makes it easy to extract and query later, as the needs arise. Depending on the logging library you choose, there will be an assortment of built-in transports, as well as third-party extension libraries that will allow your application code to log to a variety of locations.

Winston Transports

Winston, for example, comes with transports for a range of services, which you can see in the screenshot above.

Transports

Logs can be sent to a number of locations, including files in the local Filesystem, Syslog daemons, HTTP/S endpoints, and more. Depending on your logging back end, such as Winston and Bunyan, you may have a much wider array of options to use, such as CouchDB, MongoDB, Redis, and Loggly.

Syslog

Now let’s see how to connect to a remote Syslog server. To do this, I’m using a third-party transport for Winston, suitably called winston-syslog. The code below imports both dependencies and adds a Syslog transport, which will attempt to send messages to the Syslog server at 192.168.0.5 on port 514. As in the earlier examples, two info message calls are made, passing in simple text strings.

 

HTTP/S

Another option for handling or receiving logs is a remote HTTP/S endpoint. This could be a Syslog server, or any number of third-party integrations which are becoming increasingly popular, including HipChat, Slack, Loggly, Airbrake, Google+ Hangouts, and Intercom. In the example below, you can see how to configure a simple HTTP endpoint, using Winston’s http transport.

Here it’s connecting to a remote server, located at IP 192.168.0.5. It passes authentication credentials and makes a request to the path /log-receiver. This is a simplistic example, but it does show the basics of how to get started.

Log Rotation

If you are centralizing logs to another destination, you should make sure logs aren’t filling up your local server. If you’re familiar with Unix/Linux systems, then you’ll be familiar with the concept of log rotation. If not, log rotation cycles through log files, creating an archive of them over time. It starts with one, and when certain conditions are met, be that time, file size, etc., the file is archived and a new file is then logged to.

Both Bunyan and Winston offer a log rotation transport. To do so in Winston, add the following to the previous code example.

This will log to somefile.log, and when the file grows larger than 5 bytes, back up the file, appending the current date in dd-MM-yyyy format, and create a new log file. If you wanted to implement log rotation in Bunyan, the following example shows how.

It starts by importing the Bunyan library and then creates a new logger object which uses a single stream. The stream is of the type rotating-file. At the end of every day, the log file is archived, with a maximum of three archived files being kept.

Written & Contributed by

Matthew

David

Lukas

This guide will help software developers and system administrators become experts at using logs to better run their systems. This is a vendor-neutral, community effort featuring examples from a variety of solutions

Meet Our Contributors Become a contributor