I’m pleased as punch to announce the Loggly Add-on for Heroku is now in private beta! The fine folks over at Heroku just emailed me the good news a few hours ago:
“Heroku is very excited to announce the availability of the Loggly add-on to our thousands of developers. Loggly is intuitive, easy-to-use and makes logging fun again by providing a rich set of features enabling users to search and analyze their logs.”
It’s been over a year since we first visited Heroku’s offices to discuss providing a logging add-on for their platform users. The result of both company’s efforts over the last year is the first third-party Heroku logging add-on which leverages the power of the our highly scalable logging search engine and the sophistication of Heroku’s new Logplex infrastructure.
Simply put, it’s awesomesauce in the cloud.
Solving a Big Problem
The challenge Heroku faced with customer logs centered around getting access to all of the logs out of a dyno’s stack. Heroku’s stack can generate log events from the load balancer, cache, and database server, as well as logs from other add-ons, and more. While Loggly customers have been able to send logs from the app layer for a while now using Ed Muller’s super duper Logglier library , getting the remainder of the events from the stack required a rework of the way those events were routed around on the Heroku platform.
Earlier in the year Heroku released the first set of features based on this work, dubbing the project LogPlex and Open Sourced it over on Github. This solution allowed Heroku and Loggly users to use the Heroku authored Logging Add-on to forward logs to a syslog port over on Loggly. This worked well for getting access to events out of the remainder of the Heroku stack, and set the stage for Loggly to build a proper add-on for users to add to their Heroku account.
Without all the hard work from the fine folks at Heroku, we’d never been able to pull off writing our Add-on. You guys rock!
Scaling is Hard
Scaling a sophisticated platform as a service offering like Heroku is a massive challenge. There are brilliant people over at Heroku who have spent insane amounts of time working on scaling their platform to 100s of thousands of applications, all the while adding non-trival features like Logplex to their infrastructure. It’s a bit like changing tires on a fighter jet flying at mach 2.
Loggly has been spending time changing tires on jets too. When we launched in February of this year we supported a paltry 2GB a day volumes on accounts. In April we raised that to 4GB a day per account and in June we doubled that to 8GB a day, as well as releasing new pricing plans supporting custom volume and retention times. Today Loggly has over 2,500 customers, and we just upped our volumes to 12GB a day per account in anticipation of our Heroku Add-on launch. We’ll continue to up our volumes over the next few months, and continue to add features which provide custom logging solutions for web applicatoin developers.
Be sure to keep an eye out for more Loggly features like realtime feeds and alerting real soon.
Keep on logging!