I’ve been talking about coolcams for years as a way to help quickly show off a product’s features. Coolcams aren’t meant to be useful – they exist simply to entertain and engage your audience. It’s like an elevator pitch for your product demo.

Last year I did a coolcam mashup based around Poly9’s Flash Globe and events from my web server’s log files. I never did get around to publishing the code, but the URL was accessed 100s of times by the sales guys I worked with. If it ever was down, or broken, I’d get an email from then in minutes. As it turns out, a lot of them used the globe to start conversations with their customers.

Loggly Globe
Flash forward to present day. I’ve completely rewritten the code and put it up on Loggly’s site to share with everyone. The way the globe works is pretty simple. Using the web.py framework, it starts a web server which does two things. First, it serves up the HTML to your browser, which includes the Poly9 globe object and the jQuery library. Second, it serves up a JSON object to the page which is parsed and sent to the globe object. The code that serves up the JSON object does a few magical things for you:

• tails yor web access log file for visits
• parses out the ip address, timestamp, etc. from the log event
• takes the ip address and does a geoip lookup on it
• removes duplicate visits from a single ip address
• wraps the whole thing up in a JSON object

While Loggly Globe is hard coded to parse our logs, it should be fairly easy to use it yourself. To get started, download the tarball for Globe 1.0, and then extract it somewhere on your server:

kord@loggly> tar xvfz globe_1.0.tar.gz

You may need a couple of Python libraries installed. Assuming you have easy_install installed you can run:

kord@loggly> easy_install web.py
…
kord@loggly> easy_install httplib2

You’ll want to edit the globe.py file and modify the location of the Apache log file to point to your local log file. You’ll also want to edit the regular expression extractions to match your log file format. Here’s a line out of our logs for reference, and the corresponding extractions, most of which were pulled from Random Encounter. Make any changes you need to match the regex up with your logs.

75.101.142.96 - - [11/Sep/2009:09:19:17 -0700] "GET / HTTP/1.1" 200 6196 "-" "collectd/4.4.2" 195546

parts = [
   r'(?P\S+)', # host %h
   r'\S+', # indent %l (unused)
   r'(?P\S+)', # user %u
   r'\[(?P.+)\]‘, # time %t
   r’”(?P.+)”‘, # request “%r”
   r’(?P[0-9]+)’, # status %>s
   r’(?P\S+)’, # size %b (careful, can be ‘-’)
   r’”(?P.*)”‘, # referer “%{Referer}i”
   r’”(?P.*)”‘, # user agent “%{User-agent}i”
   r’(?P[0-9]+)’, # stuff at end
]

Now you'll want to start the server. You can specify a port number to listen on if you want:

kord@loggly> cd globe
kord@loggly:/globe> python globe.py 8001

http://0.0.0.0:8001/

Try hitting http://yourserver:8001/json and see if you get a response back. Here's an example of what you should see: http://www.loggly.com:8001/json. Here's the demo again, if you just want to skip to the good stuff. Additional work could be done to integrate the code into an Lightty or Apache install to make it more permanent. You can read more about doing that on Web.py's cookbook page.

Once we get the beta launched, you'll be able to make mashups like these with your own log files. We're looking forward to doing more coolcams like this with Loggly!

The first pass of the audio for the Loggly intro video was finished today. The clip will be used to further refine the wording of the script and start assembling the various visual elements we’ll need for shooting the video. What we should end up with is something similar to the works by Common Craft.

Not surprisingly, it took a TON of work to get the clip to its current state. Most of that work involved writing and editing the ’script’ to sound like the actor was talking to an audience. I’m no script writer, so it took a fair amount of work with Brenda (the voice actor to whom I’m married) to nail down what sounded natural when spoken, and what didn’t. There are still some rough spots, but here’s the final version of the first draft if you want to take a listen. The script is here, less some final edits.

Back in the day, when John Leestma and I were involved in producing the Splunk Videos, we would do outtakes of the different developers doing their thing during filming. You had to do something to keep things entertaining – a 5 minute video would take all day to film and occupy 1/3 of the available conference rooms. It’s really too bad we didn’t publish them, some of them were side splitting.

Speaking of cracking up, the wife and I have always enjoyed a good laugh together. Here’s hoping you get a hoot from our silly outtakes. All your logs are belong to us!