How Loggly Works

The Loggly system has six phases:

  • Log transmission
  • Ingestion
  • Pre-Index analysis
  • Indexing
  • Post-Index analysis
  • Loggly analysis application

centralized-ill

Log Transmission

Your application and systems generate log events which can be delivered to Loggly using any modern syslog agent or any agent that uses the syslog protocol (RFC 5424), like syslog-ng, rsyslog, and NXlog.

Alternatively, your applications (server applications or even browser-side Javascript) can send events directly to Loggly using HTTP or HTTPS. Logs can be sent in cleartext or encrypted format (TLS). Significantly, Loggly does not use any proprietary agent software which means there’s nothing to download or worry about installing. To identify your events to Loggly, a customer-unique token is provided that is included in the syslog configuration.

Ingestion

The Ingestion phase is where Loggly receives your events, records that events were received and stores the events.

Pre-Index Analysis

Once the events are stored, Loggly immediately begins analysis. If it is determined to be a standard type that is recognized by the system (https://www.loggly.com/docs/log-types/), then Loggly parses the event into individual fields. For example, if an Apache web log is parsed then all of the fields including referring URL, status code, etc. will be identified as individual fields to be indexed. Loggly automatically extracts individual fields from your data. Standardized log formats such as syslog, Apache, Java, and nginx are supported as well as JSON structured data.

Indexing

The indexing phase creates fast, distributed-lookup indices for the full text of each event as well as individual parsed fields. This provides maximum flexibility for users to find and analyze events once they are made available in the application.

Post-Index Analysis

At this stage, Loggly performs any alert conditions that the user has set, executes any reports that are due to be issued and updates summary information for dashboard widgets that have been created.

Log Analysis Application

All search, trending, alerting, reporting, setup functions are available via the Loggly web application as well as via our RESTful API.

Top