Loggly Live Tail

Description

Live Tail provides comparable, but more powerful, capabilities than the classic tail command on most Linux/Unix systems (on Windows, the PowerShell command Get-Content provides similar functionality when invoked with the parameters -Tail and -Wait). Similar to tail -f, Live Tail allows you to monitor all the log data that is being sent to Loggly in a consolidated form and in near real-time. Pattern-based filtering and color coding help you to focus on what matters to you.

Screen Shot 2016-06-01 at 12.48.07 AM

You can also send Live Tail output to third party messaging and collaboration tools. This happens in parallel with the output written to the command line. See Sending Live Tail Output to Third Party Services for more information.

The Live Tail feature is part of the Enterprise subscription tier. If you don’t have an Enterprise subscription, you can upgrade by contacting your Loggly account manager. If you’re on a Standard or Pro subscription and not ready to upgrade your account but want to try Live Tail, you’ll be able to sign up for a special 14-day trial when you click on Live Tail in the Loggly menu bar. Live Tail is not available as a part of the Lite subscription. See our pricing page for details.

 

Installation and Operation

Live Tail in Your Browser

  1. Options
  2. Examples

 

Live Tail on Linux, Mac OS X, Unix

  1. Prerequisites
  2. Installation
  3. Basic Operation
  4. Command Line Options
  5. Examples

 

Live Tail on Windows

  1. Prerequisites
  2. Installation
  3. Basic Operation
  4. Command Line Options
  5. Examples

 

Sending Live Tail Output to Third Party Services

  1. Sending Output to HipChat
  2. Sending Output to Slack

 

Live Tail in Your Browser

The browser version of Live Tail lets you monitor and filter incoming log data straight from your web browser. It supports most of the features available in the command line versions including near real-time updates, pattern-based filtering using regular expressions, and color coding. In addition, Live Tail in the browser also gives you the option to group/ungroup similar events and you could choose between a light or dark theme.


Options

Matches Regex A log entry is printed if it matches any of the specified Regex patterns. Matching patterns are highlighted, and multiple capture groups are highlighted in different colors. This field is disabled when Live Tail is running.
Ignore Regex A log entry is ignored if it matches any of the specified Regex patterns.
Start Initiate Live Tail in the browser by clicking on the Start button. You could also enter the desired regex and hit enter key on your keyboard to start the Live Tail
Stop Terminate Live Tail in the browser.
Pause Click the Pause button to explore any event in the stream and view grouped items.
Group/Ungroup Events Group similar events to identify patterns quickly and reveal what matters. By default grouping is enabled.
Clear Click on the Clear button to clear the contents of the Live Tail screen.

Examples

To see the entire data stream with no filtering applied:

Match Regex = .*

To see only log entries that match the term “ERROR”:

Match Regex = ERROR

To see only log entries that match the terms “ERROR” and “DEBUG”:

Match Regex = ERROR|DEBUG

To see only log entries that match the term “ERROR” but ignore those that match “DEBUG”:


Grouping Events

Grouping similar events helps to identify the patterns quickly and reveal what matters. You also have the option to pause the stream to expand and view grouped items.

 

Live Tail Settings

Use Settings to select how events are grouped (Exact or String match) and select theme (Light or Dark mode)

Exact match vs String match: As the name suggests Exact match will check for matching letters [A-Z, a-z], numbers[0-9], whitespaces or special characters in your Live Tail events in order to form a group. If there is a mismatch for any of these, then the group will not be formed. However, String match will only check the letters [A-Z, a-z] and ignores any numbers, whitespace and special characters in your event in order to form a group.

For example, consider these two log event snippets:

X-Real-IP:54.173.157.5, X-Forwarded-For:54.173.157.5, Content-Type:application/json, tag:.... }
X-Real-IP:54.173.157.5, X-Forwarded-For:54.173.157.8, Content-Type:application/json, tag:.... }

Using string match, these events will be grouped together, since the string match ignores the numbers (highlighted in red) that don’t match. However, with exact match, these will not be grouped together. Both IP’s need to be identical for an exact match.
Light mode vs Dark mode: Select the theme that suits your preference. Light mode uses a light background and dark text, as shown in the first image above. Dark mode uses a dark background and light text, as shown in the second image above.

 

Filter and Highlight Events

Use Regex patterns to filter and highlight events within the Live Tail stream. This field is disabled when Live Tail is running.

 

Live Tail Command Line Client

Live Tail is available as a command line interface (CLI) program for Mac OS X, Linux and other Unix-type and Windows operating systems. It allows you to pipe output into commands (like grep, awk, sed, etc.) The sessions will time out after 1 hour if there is no activity.

Screen Shot 2016-01-13 at 5.39.19 PM

Prerequisites

As a prerequisite, the Java Development Kit (JDK) needs to be installed on your system. Live Tail will automatically download the correct version of Java if it can’t find it on your system, and place the installation file in the Live Tail installation directory. Please refer to these instructions on how to install JDK on your operating system.

Installation

  1. Download Live Tail from here or use
    curl -O 'https://www.loggly.com/install/tailclient-1.0.2-install.zip'
    
  2. Unzip the zip file using
    unzip tailclient-1.0.2-install.zip
  3. CD into the resulting directory and edit the configuration file conf/livetail.properties by adding your authentication code from the Live Tail instruction page in Loggly (you get there by clicking “Live Tail” in the Loggly menu bar). Alternatively, you can copy and paste the sed command from those instructions. The line should look like this:
    # You should put your Auth token
    tail.client.authtoken=<AUTHTOKEN>
    

Basic Operation

  • Change to the bin/ subdirectory in the Live Tail installation directory using
    cd bin/
  • Run the following command:
    ./livetail -m <matcher pattern> -i <ignore pattern>
  • CTRL+C ends the program.
  • See below for examples.

Command Line Options

Options
-h, --help Print a brief help message and exit.
-v, --version Print version and exit.
-at --authtoken <arg> Live Tail authentication token for customer. This can be omitted if the token is specified in Live Tail’s configuration file conf/livetail.properties (recommended).
-m, --matches <pattern|pattern|...> (Mandatory) A log entry is printed if it matches any of the specified Regex patterns.
-i, --ignore <pattern|pattern|...> A log entry is ignored if it matches any of the specified Regex patterns.
-bc,--background <arg> Background color highlight for matched pattern.
-tc,--textcolor <arg> Text color highlight for matched pattern.

Examples

To see the entire data stream, no filtering applied:

./livetail -m '.*'

To see only log entries that match the term “ERROR”:

./livetail -m 'ERROR'

To see only log entries that match the terms “ERROR” and “DEBUG”:

./livetail -m 'ERROR|DEBUG'

To see only log entries that match the term “ERROR” but ignore those that match “DEBUG”:

./livetail -m 'ERROR' -i 'DEBUG'

To pipe the entire data stream to the Unix grep command to search for lines that contain the term “level”:

./livetail -m '.*' | grep 'level'

Live Tail on Windows

Prerequisites

As a prerequisite, the Java Development Kit (JDK) needs to be installed on your system. Please refer to these instructions on how to install JDK on your operating system.

We recommend that you use Windows PowerShell to execute the following steps, but Live Tail will work using Command Prompt as well.

Installation

  1. Download Live Tail from here or (on Windows PowerShell only) use
    curl 'https://www.loggly.com/install/tailclient-1.0.2-install.zip' -Outfile tailclient-1.0.2-install.zip
    
  2. Once downloaded, unzip the file by right-clicking it in Windows File Explorer and select “Extract all”, or by using your favorite unzip method.
  3. In a Command Prompt or Windows PowerShell window, cd into the resulting directory tailclient-1.0.2-installtailclient-1.0.2. Edit the configuration file conflivetail.properties by adding your authentication code from the Live Tail instruction page in Loggly (you get there by clicking “Live Tail” in the Loggly menu bar).The line should be edited to look like this:
# You should put your Auth token
tail.client.authtoken=<AUTHTOKEN>

Basic Operation

  • Change to the bin subdirectory in the Live Tail installation directory using
    cd bin
  • Run the following command:
    .livetail.bat -m <matcher pattern> -i <ignore pattern>
  • CTRL+C ends the program. If asked if you want to terminate the batch job, hit Y (Yes)
  • See below for examples.

Command Line Options

Options
-v Print version and exit.
-at <arg> Live Tail authentication token for customer. This can be omitted if the token is specified in Live Tail’s configuration file conf/livetail.properties (recommended).
-m <pattern|pattern|...> (Mandatory) A log entry is printed if it matches any of the specified Regex patterns.
-i <pattern|pattern|...> A log entry is ignored if it matches any of the specified Regex patterns.

Examples

To see the entire data stream, no filtering applied:

.livetail.bat -m ".*"

To see only log entries that match the term “ERROR”:

.livetail.bat -m "ERROR"

To see only log entries that match the terms “ERROR” and “DEBUG”:

.livetail.bat -m "ERROR|DEBUG"

To see only log entries that match the term “ERROR” but ignore those that match “DEBUG”:

.livetail.bat -m "ERROR" -i 'DEBUG'

 

 

Interactive Mode

Live Tail can be put into interactive mode, which displays matching patterns and other parameters.

To enter interactive mode, type / + ENTER in a running Live Tail session. Type / + ENTER again to get back to data mode. /help will tell you what interactive commands are available.

/
#######################################
Loggly Live Tail: Now in Command Mode. To get in Data mode type / again . Type /help to see help
#######################################
/help
#######################################################################
Loggly Live Tail Command Help
To stop tail type "/" to enter in command mode or Data mode
To stop tail type "/stop"
To start tail type "/start"
To pause tail type "/pause"
To unpause tail type "/unpause"
To show matcher type "/show matcher"
To unpause tail type "/show ignore"
#######################################################################

Sending Live Tail Output to Third Party Services

It is possible to send Live Tail output to third party messaging and collaboration tools. This happens in parallel with the output written to the command line. Currently, Live Tail supports sending its data to a chat room in Atlassian’s HipChat and Slack.

Sending Output to HipChat

To send Live Tail’s output to a HipChat room simply add a line with the following syntax to the configuration file conf/livetail.properties:

tail.client.im.url=https://api.hipchat.com/v2/room/<HIPCHAT ROOM API ID>/notification?auth_token=<HIPCHAT AUTHORIZATION TOKEN>

First you need to create a chat room in HipChat to receive the data from Live Tail. Then simply log into HipChat’s web frontend (www.hipchat.com) and select the chat room from the “Rooms” menu. You will find the <HIPCHAT ROOM API ID> and <HIPCHAT AUTHORIZATION TOKEN> in the settings for your room.

HipChat and Loggly Live Tail

Sending Output to Slack

You can send Live Tail output to your Slack teams by using Slack’s Webhooks integration. From your team’s channel list, click on the name of your team, then click on Apps & Custom Integrations. Search for the Incoming WebHooks integration, then click Install. From here, you can choose an existing channel to send your Live Tail output to, or you can create a new channel.

slack_channel

After adding the integration, you’ll receive a Webhook URL similar to the following:

https://hooks.slack.com/services/T00000000/B00000000/ABcdefGHIjklmnOPQRstuvWX

Simply add this URL to your conf/livetail.properties configuration file:

tail.client.im.url=https://hooks.slack.com/services/T00000000/B00000000/ABcdefGHIjklmnOPQRstuvWX
Thanks for the feedback! We'll use it to improve our support documentation.


Top