With a microservices architecture running in Docker containers with Kubernetes orchestration, the development team at Molecule knew that it needed a centralized log management solution. “Pairing AWS EC2 with Kubernetes means that we don’t have to do anything to scale up and down,” says Adam Sunderland, Chief Architect at Molecule. “That easy automation extends to centralizing our log data.”
Molecule initially deployed a log management solution based on Elasticsearch-Logstash-Kibana (ELK) running on Amazon EC2 (mostly m4.2x instances) but found that this solution cost more to maintain than the company thought it would. “It got expensive for us to run because of the amount of data we were logging, which peaked at about 8GB on some days,” Sunderland reports. “That meant we had a lot to maintain.”
After searching for dedicated log management services and evaluating several, the Molecule team chose Loggly for its rich feature set, easy implementation, and support for fluentd.
Because Molecule was already aggregating its logs using fluentd, implementing Loggly was a quick transition. “We were surprised at how easy it was to switch over,” Sunderlands notes. “All we had to do was add the fluentd plugin for Loggly.”
Kubernetes provides an important log management benefit: structured log data. With Docker containers operated in Kubernetes, Molecule gets logs formatted as JSON and tagged with metadata indicating the application and instance from which they came.
Loggly is a really good value. It’s fast, and it gives me the information that matters. With Loggly, we have a lot more capability than what we got from Kibana.
— Adam Sunderland, Chief Architect
Molecule’s primary use case is to investigate operational issues. Because all of Molecule’s log data is formatted as JSON, the team can quickly drill down into specific components using Loggly Dynamic Field Explorer™ or look at multiple components side by side. “Having the ability to slice our log data by field gets us to answers much faster,” Sunderland says.
The team also uses Loggly Live Tail in its daily work. “The regex and filtering that you get in Live Tail is pretty neat,” Sunderland adds. “I also like being able to use it in my command line client.”
“If your logs are in JSON, Loggly makes it super easy to filter down to what matters and get more information from your logs. And JSON comes for free from Kubernetes.”
Loggly alerts help the development team at Molecule to detect potential issues before they affect customers. “We are continuing to look at new alerts to add,” Sunderland says. Molecule is also starting to build dashboards to track critical data and to take advantage of Loggly Anomaly Detection for one specific process in its application. In this process, Molecule parses a file. It expects to see a certain number of lines fail to parse in a day and logs all of those cases. So, if for some reason that number goes up or down, the team knows that action may be required.
“Before, when something went wrong, we sometimes struggled to find the logs that mattered,” Sunderland concludes. “Now that we have Loggly, we get our log data broken down by namespace, task, time, and other parameters. When we start investigating a problem, we’re already really close to an answer.”