My relationship with Trinity goes back over a year and a half ago – well before Raffy and I started thinking about doing a cloud based log management offering. Like many other startups, Trinity was started by two entrepreneurs. For years, Trinity’s motto has been focusing on early stage companies in specific technology categories, such as cloud computing and systems management.

Loggly is extremely fortunate to be working with Trinity and their bright team, and we greatly value the market experience they bring to the relationship.

History Lessons

I met Trinity through a fairly short introduction path. My good friend and former colleague, Dakota Sullivan, introduced me to a gentleman named Matt Strand in January of 2009. Matt and I had coffee at Crossroads Cafe in South Beach where I told him I was looking to join or start a cloud computing based company. Matt figured he should hook me up with a VC buddy of his, Dan Scholnick at Trinity.

Here’s the email introducing the two of us:

Matt was right about it being a valuable connection. Over the next year Dan and I would spend time together drinking coffee, chatting on the phone, and emailing each other about ideas in and around the enterprise and cloud computing space.

It was because of my conversations with Dan that Raffy and I were able to come to the idea of a cloud based logging service. Even when it came time to start pitching Loggly to others, Dan and Noel assisted us in honing our pitch, which eventually led to us being funded by True in a seed round.

Start Small, Go Big

When you are starting out, even the smallest conversation or the shortest email could potentially be the most important one you’ve had in years. Having an idea, growing it, and turning it into a business is a complicated process. That process takes time, and doesn’t happen over a matter of days, or even weeks, but instead over months and even years.

Our relationship with Trinity has been a long time in the works. While it may have appeared to happen rather quickly, Loggly’s efforts with Trinity started at the very beginning of its life.

In as much as your idea should evolve over time, your ability to convey the idea and the opportunity it represents should grow as well. I’ve lost count of how many times Dan has told me to ‘crisp up’ my presentation, discussed with me partnership negotiation strategies, or told me how to approach feedback with our current private beta testers, but I’m sure the hell glad he did.

Without investors like Trinity and True, it’s unlikely I’d be here telling you this story. You would be well to seek out these types of investors when you are looking for direction and guidance for your idea.

Now you’ll excuse me while we get back to coding. We have beta testers who have logs in need of indexing!

“Despite the economies of scale offered by public cloud providers, private cloud services will prevail for the foreseeable future while public cloud offerings mature, according to Gartner, Inc. Through 2012, IT organizations will spend more money on private cloud computing investments than on offerings from public cloud providers.”

This statement is a bit like NASA doing a press release announcing the moon is continuing to orbit the earth. Wow! The moon, still here next year? That’s awesome. Of course IT is going to spend more money on virutalization for the next few years. The success of the private cloud can be attributed to the fact virtualization has been around for a good while now, and is finally being pressed into mainstream use behind the firewall. Shoot, I think I was running Wine on some of my Linux boxes back in the mid-90s, which means virtualization has been commercialized for at least 15 years at the least. The idea of virtualizing an OS goes back well into the 60s. Come to think of it, so do I.

The public cloud, specifically IaaS and SaaS, is a grouping of emerging technologies. We’re just now starting to figure out how to wield it correctly for new business models. Poking holes in it at this point is simply rabble rousing by companies who’s business models are threatened by it and people who don’t understand it or have a use for it.

It’s a Complicance

Guy Churchward tries to make some good points in his talk with Dave, but at the end of the day, LogLogic is mainly an appliance vendor, and not only do they have big-time COGS to worry about, they also have to figure out how exactly a cloud customer is going to deploy their box on Amazon’s EC2 service. (Hint: They aren’t.) While you might be able to send logs back out of the cloud to an appliance behind the firewall, it’s unlikely to make economical sense to do so in the long term.

While there is a valid point in calling out cloud concerns, security itself is ALWAYS a concern, regardless of whether you run in the cloud or in your own datacenter. Frankly, with Loggly I’m likely better at storing and securing your logs than you are by yourself in your own data center, mostly due to the fact I’m under pressure by multiple people like you to provide a service which is expected at the outset to be secure. It’s no different than the pressure that Google has on them for securing your email, SalesForce for securing your leads, or Amazon securing your credit card info. We’re all culpable here for the security of your data.

Additionally, not all that cloudy data is created equal. A lot of the companies running in the cloud today are web based app companies, and the data they generate is often times very public in nature and not at all affected by compliance concerns. Do you think some user on Flickr cares if I stole all their comments? What about getting access to all those juicy tweets of mine? Oh wait, those are already in the Library of Congress. Nevermind, false alarm!

When IT Rains IT Pours

Log file data is already one of the largest sets of data on the planet. Logging alone in the public cloud is going to be absolutely staggering over the next few years. These trends are being driven by people switching to SaaS based applications, in turn who’s infrastructure either requires the elastic capabilities only the public cloud can provide, or who’s price point can’t be matched by private cloud offerings.

The elastic nature of these infrastructures means the logs which they generate need to be collected and stored in centralized location before the box that generated them disappears. There are many types of logs which are valuable to a company for understanding their business, and not so valuable for those data-thieving ruffians everyone keeps talking about.

While the security access data or net-flow information from public cloud vendors might alleviate the concerns of some consumers, I think there are much higher value adds to these offerings by being able to power availability and analytics services around a company’s application via a log file storage platform.

While the private cloud may continue to orbit peacefully for the next few years, the use of it for web based services will decay eventually, and it’ll be regulated to the more mundane stuff like storing my dental records and tracking my orders over on RadiatorBarn.com.

BTW, I’m still waiting on my radiator, Burton.

Perhaps it was fate that we spoke with Puneet Agarwal at True Ventures first. True has a massive amount of experience investing in and managing early stage companies. Their record of past successes speaks for itself, and their team has experience with over 100 early stage investments that have generated significant investment returns. Frankly, Raffy, Jon and I are extremely fortunate to be working with the True Ventures team.

That first meeting with Puneet was actually quite easy; we had no other expectations other than sharing what we were thinking with someone who knew the space well. That was the calm before the storm though – over the coming weeks we struggled with writing our investor deck, meeting schedules, market size expectations, investor lack of familiarity with our market, and became consumed with correctly casting the “going big” portion of our pitch.

Going Big

The best way to describe what “going big” means is to just be blunt about it. It means, “How are you going to make your idea – your startup – compete effectively in a multi-billion dollar market?” You know, how are you going to get big like Apple, or SalesForce, for example. “Say what?”

A bunch of you capital guys and seasoned entrepreneurs will nod your heads vigorously at this statement. “Yes, yes. You need to show how this gets really big!” And, for all practical purposes, you guys are absolutely right. For a largish VC, say with a fund of a billion or so dollars to invest, they HAVE to go with early startup guys who are going to go really, REALLY big. It’s not a matter of money, it’s actually a matter of time. If you have a BILLION dollars, and you invest a million in each company you fund, that’s a THOUSAND companies you need to talk to, investigate, vet, poke at, wrangle with, grow to love, etc. Yeah, no. That’s not going to work unless you focus on a given market.

You’d have to filter fast. Kick out the guys that MIGHT do a run rate of low 10s of millions a year (crazy, right?) because they aren’t big enough. Shoot for the guys who tell a good story about how they are going to turn into Twitter, or Facebook, and exit for billions. Find those guys! This results in an investor funding a bare handful of early stage startup each year, even if they say otherwise on their website.

It also serves another purpose, all those meetings with those small startups. It allows an investor to form early relationships with companies who are successful at getting through the valley of death. If an investor finds out a startup they talked to early on is doing well, has revenue coming in, is growing, and expanding to the “going big” event, then maybe they might need some more capital. Maybe it’s time to invest.

Entrepreneur Up

If you are doing an early stage startup and are going to raise capital, you need to toughen up a bit before you go out. Remember, it only takes one firm to believe in your idea, but you are going to get an inverse number of rejections before that event transpires. If you get a bad review from someone, take their advice in stride and figure out how it applies to you. Tweak as needed, and move on to the next firm to vet what you’ve discovered.

Above all, be honest with yourself and your assumptions and don’t give the investor who gave you a bad review a hard time. If you think you’ve been asked to prove something unreasonable, like how you are going to become a billion dollar company when it’s just the 2 of you and a 1,000 lines of code, then say as much to the investor. Don’t be afraid to say you don’t know, or restate what you do. Don’t be afraid to talk through how you get big with the investors.

After being asked how we got to a billion dollar valuation by one VC, I turned it around on him and asked how he knew he was going really big on his company (which exited for >$1B). His answer? “We didn’t know until we got there!”

Loggly is going to go big, of that I assure you. But first, we have a product to build, customer and partner relationships to forge, and problems to solve for storing a ridiculous amount of log files. Once we have these tasks behind us, we’ll have a great handle on how we’re going to go really, really big.

One of the things you’ll notice when you use Unfuddle is the presence of a subdomain in the URLs you use on the site.  Our subdomain is ‘loggly’ on Unfuddle, and we  log into our project area by going to http://loggly.unfuddle.com/ (no, you can’t check our code out).  This type of customer segmentation allows for multiple unique usernames per customer, but doesn’t require a unique username site-wide.  For non-SEO sections of the site, this is a perfect solution.

We are taking a similar approach with Loggly, where a user will sign up for an account and define a unique customer identifier (we’re kicking around calling this a “mill”), which will then be mapped to a subdomain on the system.  So, for example, if Foobar, Inc. were to sign up for a Loggly account, they would access the site via http://foobar.loggly.com/, and then could create any number of user/pass combinations they wanted to access their company’s log resources.

The only problem with this approach is that we use Django, and their built in auth system (which is fantastic, BTW) doesn’t really have facilities for this type of functionality.  While we could certainly hack the Django auth system by writing our own multi-tenant auth module, it would take away from more pressing issues – like launching the beta!

Enter the Middleware Solution

One way to solve this is by munging the subdomain and username together, which provides a unique system-wide username. If, for example, you were to log in as steve under foobar.loggly.com, then we’d stick them together to be something like “foobar_steve”. Obviously we can’t have everyone remembering this long monstrosity for their username, so we’ll need to munge the subdomain off the URL and the username the user types in to get the correct combination to send off to the auth system.

Thankfully Django provides a super-easy way to add middleware to a project. By injecting a small piece of code into the request from the user’s browser, we are able to do our on-the-fly transformation before the auth system takes over. Nobody is the wiser because we can modify the display name code in the profile model to show the “normal” username to the user. Here’s what the result looks like:

settings.py:
...
MIDDLEWARE_CLASSES = (
'loggly.profile.MungeMiddle.MungeForMillMiddleware',
...
)
...

MungeMiddle.py:
class MungeForMillMiddleware:
    def process_request(self, request):
        if request.POST.has_key('username'):
            data = request.POST.copy()
            user = "%s_%s" % (request.META['HTTP_HOST'].split('.')[0], data['username'])
            data['username'] = user
            request.POST =  data

When a request comes in, we pull out the POST data and make a copy of it with .copy(). We then munge up the username with the subdomain out of HTTP_HOST, and then set the POST data to forward on to the rest of the stack. We don’t do this for all requests, just ones with the username set, so it’s lightweight enough for production use. We end up sticking the shorteded version of the username into the profile table, and use it for display.

So there you have it. A 5 minute fix for a 5 hour problem. I’m sure there are more elegant solutions to doing subdomain segmentation with Django’s out-of-the-box auth system, but frankly we don’t have time to stop and code them up. We’re bent on getting our beta out as soon as possible, and if it requires hacks like these to do it, then so be it! Release early, release often.

We’ve heard everything from “you need to show me how this is a billion dollar company” (lol wut?) to “move the team slide just below the problem statement”. Bad advice aside, I remain focused on our market size and, in particular, the user conversion assumptions that go along with it.

I’ll be transparent here. I expect Loggly’s model to yield a 10-20% conversion rate to paid customer from a signed up freemium account. While I have good justifications for those numbers, posts like this, by Derek Haynes of ScoutApp, would appear to contradict my assumptions, especially with comments like “The 1% rule is the Pi of freemium web apps!”. In all fairness he has an excellent point about focusing on retention optimization, but should we so blindly ignore our conversion pipe just because of the 1% rule says we’ll get the users?

Theories != Rules

As it turns out, the “1% rule” is based on observations on other websites and expected conversion from a site visitor to a paid user. Coincidentally enough, this assumption echos the controversial content creation theory and smacks of Chinese Math.

The main problem with this approach is it completely and utterly ignores the remainder of a given site’s user pipeline. Pipelines, or funnels, are the bread and butter of any sales driven organization. While sites like YouTube may have simple funnels, sites like Loggly are a bit more complicated, and a missed assumption somewhere along the way can turn your assumed 1% conversion into a .1% conversion fairly quickly. To illustrate, here’s a short list of what we’ll be tracking at Loggly (per month):

• number of ad views on ad network
• number of visits
• uniques
• % visits and uniques from ads
• bounce rate
• content consumed (price sheets/videos)
• participation in the demo (low friction getting started)
• freemium signups
• use of a freemium account
• use of certain features in the freemium account
• conversion rate to paid accounts
• use of paid account
• logging rates
• upgrades from one paid tier to another

It’s within this pipeline that we focus our attentions to help maximize conversions from visit to paid user.  Not surprisingly, the devil is in the details.

Care and Feeding of Your Pipeline

Portions of a pipeline need to have estimated percentages assigned to them, and then you need to actively monitor those conversions from step to step when the site goes live. I’ll start with the expectation of 1 conversion from 100 unique visits to a paid account and then project assumptions about the other portions of the pipeline:

• 10,000 ad views yields 100 unique visits (1%)
• 100 unique visits yields 20 content consumers (20%)
• 20 content consumers yields 10 demo participators (50%)
• 10 demo participators yields 5 freemium signups (50%)
• 5 freemium signups yield 1 paid account (20%)

Notice this gives us our 1% conversion from unique visit to paid user, but still shows a 20% conversion from freemium to paid account. Now we do a breakdown for the intermediate steps, and find places we can optimize. Your mileage may vary, so make sure you understand your steps, and measure the effects of changes on your conversion numbers when you are able to do so!

Content is King

Based on prior experience, we know the subject material of Loggly should yield decent conversions from a visitor to content consumer, where the parties are interested in the offering. At previous engagements we were able to get 10K unique video views a month off 100K visits per month. Add in another 10% viewing pricing, examples, etc., and 20% starts to seem reasonable. By driving viewing content, we increase stickiness, educate the users about the service, and increase conversions to freemium signup.

Once the user is viewing content, we want to convert them to using the demo, and then on to creating an account later. Loggly’s demo will actually be a live demonstration of the service using the user’s own content, but it won’t require the user to sign up in the traditional sense. By flipping the signup process on it’s head, we get the user using the features first, then ask them for account information after they’ve decided they like the service. All in, and based on the frictionless signup, we expect 25% of users who viewed advanced content to convert to a freemium account.

Get Out of the Way!

Remember, your signup form is a HUGE barrier to users. If you require them to provide 12 fields of data to fill out, it’s that much less likely you’ll get a signup out of the deal. Just look at this signup form – I gave up after 5 minutes of trying to fill it out properly (picky password, clearing fields on submission, etc.), even though I’m highly interested in IBM’s offer. Most people won’t spend more than a minute or so filling out your signup form. The lesson here is to get out of the way so a user can start using your service quickly.

Finding Cash in the Pipe

Remember, freemium accounts are not the same as free trials. A free trial expires, but may contain all the features of a paid account. A freemium account doesn’t expire, and usually contains a subset of features of the paid account. By keeping the freemium account active for a longer period of time than the trial (and by keeping the cost of providing services to it low) you extend the amount of time a freemium user can convert to a paid user. While this may not show up in the first few months, over time it will have a compound effect on your conversions as users desire more features from the service.

At this point in the pipeline we are at a 5% conversion rate from visitor to freemium account, which so far seems to be a reasonable assumption. If we take the 10-20% estimated conversion rate to paid assumptions, then we arrive at a .5% to 1% conversion rate from visitor to paid account, which is within striking distance of our 1% goal.

In the next segment of this post, I’ll discuss analyzing how users use their accounts, and steps you can take to maximize conversion from freemium to paid.

Edit (11/12/09): Fixed my bullets in the second list.

As for the rest of the team, Raffy is now officially our COO and Chief of Product (BTW, Swiss PMs rule), and I’ll be serving as our Geek CEO and Chief Evangelist. All three of us will be coding on Loggly over the next few months, and with a dash of a soon-to-be-announced designer, we’ll be assembling the framework needed for getting Loggly up and running in private beta over the coming months.

I’m really excited to be working with such smart and capable individuals. I can’t wait to start working with equally smart and capable partners and customers.

See you in your logs!

Last year I did a coolcam mashup based around Poly9’s Flash Globe and events from my web server’s log files. I never did get around to publishing the code, but the URL was accessed 100s of times by the sales guys I worked with. If it ever was down, or broken, I’d get an email from then in minutes. As it turns out, a lot of them used the globe to start conversations with their customers.

Loggly Globe
Flash forward to present day. I’ve completely rewritten the code and put it up on Loggly’s site to share with everyone. The way the globe works is pretty simple. Using the web.py framework, it starts a web server which does two things. First, it serves up the HTML to your browser, which includes the Poly9 globe object and the jQuery library. Second, it serves up a JSON object to the page which is parsed and sent to the globe object. The code that serves up the JSON object does a few magical things for you:

• tails yor web access log file for visits
• parses out the ip address, timestamp, etc. from the log event
• takes the ip address and does a geoip lookup on it
• removes duplicate visits from a single ip address
• wraps the whole thing up in a JSON object

While Loggly Globe is hard coded to parse our logs, it should be fairly easy to use it yourself. To get started, download the tarball for Globe 1.0, and then extract it somewhere on your server:

kord@loggly> tar xvfz globe_1.0.tar.gz

You may need a couple of Python libraries installed. Assuming you have easy_install installed you can run:

kord@loggly> easy_install web.py
…
kord@loggly> easy_install httplib2

You’ll want to edit the globe.py file and modify the location of the Apache log file to point to your local log file. You’ll also want to edit the regular expression extractions to match your log file format. Here’s a line out of our logs for reference, and the corresponding extractions, most of which were pulled from Random Encounter. Make any changes you need to match the regex up with your logs.

75.101.142.96 - - [11/Sep/2009:09:19:17 -0700] "GET / HTTP/1.1" 200 6196 "-" "collectd/4.4.2" 195546

parts = [
   r'(?P\S+)', # host %h
   r'\S+', # indent %l (unused)
   r'(?P\S+)', # user %u
   r'\[(?P.+)\]‘, # time %t
   r’”(?P.+)”‘, # request “%r”
   r’(?P[0-9]+)’, # status %>s
   r’(?P\S+)’, # size %b (careful, can be ‘-’)
   r’”(?P.*)”‘, # referer “%{Referer}i”
   r’”(?P.*)”‘, # user agent “%{User-agent}i”
   r’(?P[0-9]+)’, # stuff at end
]

Now you'll want to start the server. You can specify a port number to listen on if you want:

kord@loggly> cd globe
kord@loggly:/globe> python globe.py 8001

http://0.0.0.0:8001/

Try hitting http://yourserver:8001/json and see if you get a response back. Here's an example of what you should see: http://www.loggly.com:8001/json. Here's the demo again, if you just want to skip to the good stuff. Additional work could be done to integrate the code into an Lightty or Apache install to make it more permanent. You can read more about doing that on Web.py's cookbook page.

Once we get the beta launched, you'll be able to make mashups like these with your own log files. We're looking forward to doing more coolcams like this with Loggly!

Not surprisingly, it took a TON of work to get the clip to its current state. Most of that work involved writing and editing the ’script’ to sound like the actor was talking to an audience. I’m no script writer, so it took a fair amount of work with Brenda (the voice actor to whom I’m married) to nail down what sounded natural when spoken, and what didn’t. There are still some rough spots, but here’s the final version of the first draft if you want to take a listen. The script is here, less some final edits.

Back in the day, when John Leestma and I were involved in producing the Splunk Videos, we would do outtakes of the different developers doing their thing during filming. You had to do something to keep things entertaining – a 5 minute video would take all day to film and occupy 1/3 of the available conference rooms. It’s really too bad we didn’t publish them, some of them were side splitting.

Speaking of cracking up, the wife and I have always enjoyed a good laugh together. Here’s hoping you get a hoot from our silly outtakes. All your logs are belong to us!

Now I’m working on Loggly fulltime, I realized we need a video which would explain the product in an easy to digest format, along the lines of the Twitter in Plain English video. It struck me I have all the bits I need to do this myself, including an amazing performer voice (albeit not using her operatic mode) and a kick ass microphone.

Videos are one of those things that help brand your identity, along with your blog posts, tweets, and of course, your product. The way in which you present your product to the people who use/consume it helps sear your identity forever in their minds. Case in point, Bluebottle Coffee. A friend of mine took me there a few weeks back and I was blown away by the preparation process. It was a thing of art really, much like the Blue Bottle Mic.

Maybe we should name ourselves Blue Bottle Logs. Whaddya think?