As a SaaS service providing critical identity infrastructure for thousands of enterprise production deployments, large-scale web application, and high-growth tech startups, Stormpath needed an effective way to troubleshoot operational issues and solve customer problems.
Stormpath runs its service on Amazon Web Services (AWS) and utilizes a large number of AWS Compute, Networking, and Storage Services, including Amazon EC2, Amazon Route 53, Amazon ElastiCache, Amazon DynamoDB, Amazon S3, and Amazon VPC. “What we can accomplish with AWS is huge,” says Les Hazlewood, co-founder and chief technology officer at Stormpath. “We get everything we need to run a full product stack with a smaller skilled operations team.”
Stormpath recognized early on that it needed insights from log data to operate effectively – and that building its own log management system was not a good use of its resources. “Setting up a log management solution with Elasticsearch-Logstash-Kibana (ELK) was an option.” Hazlewood comments. “In fact, we have ELK experts on our team who have done it many times. But the labor we would have to expend to do it again is better spent on amazing identity management.”
After an experience with an early version of Loggly as well as a Loggly competitor, Stormpath re-evaluated Loggly in mid-2015. The team chose to standardize on Loggly after determining that:
Every process that runs within the Stormpath infrastructure generates JSON logs and emits them to
stdout. This includes scraped/secured HTTP request logs and any developer-issued logs within the code base. From there, syslog and logspout log aggregators consume the logs from
stdout and funnels them to Loggly. “We were all happy that Loggly supports JSON,” Hazlewood comments. “It makes field-based queries much simpler.”
Stormpath is a cutting-edge identity management provider. I don’t want our team spending resources to build out a log management infrastructure. Offloading log management to Loggly is an easy trade-off.
— Les Hazlewood, Co-Founder / Chief Technology Officer
Stormpath has deployed SignalFx for system monitoring, system metrics, and alerting. Because the core of its service constitutes HTTPS requests, most operational issues manifest themselves with status codes that can be tracked with tools like SignalFx. Stormpath also uses Loggly alerting to detect some unexpected errors and anomalies that can be detected from log data. When operations team members receive an alert, they use Loggly to correlate log data with the alert condition and to understand what to do next.
“The Loggly UI is intuitive and straightforward. Even though it’s geared to a broader set of use cases than other log management solutions, our DevOps team prefers the way Loggly works.”
If a customer contacts Stormpath support to report a failed request, the customer support engineers can use Loggly to identify the source of the problem. Using log data, they can direct customers on how to fix problems in their systems or escalate to Stormpath development and DevOps if necessary.
Hazlewood summarizes the unique value of Loggly as follows: “At Stormpath, non-operations people and even non-engineers need to interact with logs. Loggly makes it easier for these people to formulate queries in the way they naturally think about them than it is with log management systems that are focused on the workflows of sysadmins and operations people.”