Logstash Logs

You can send your Logstash logs to Loggly over HTTP. We will automatically parse the logs sent by Logstash in JSON format. You can send logs from any of Logstash’s inputs to Loggly, but we offer an example showing a standard input. These instructions were tested with version 2.1.1. For alternatives, please see the Advanced Options section.

Logstash Logging Setup

  1. Configure Logstash To Output To Loggly
  2. Download the logstash-2.1.1 .tar.gz file from here.
    Unzip and Untar the file

    sudo tar -xzvf logstash-2.1.1.tar.gz
    

    Move the folder to /opt/

    sudo mv logstash-2.1.1 /opt/
    

    Go to the folder and install the logstash-output-loggly plugin

    cd logstash-2.1.1
    sudo bin/plugin install logstash-output-loggly
    

    Create a logstash-loggly.conf file and add it to the root folder of the Logstash directory. We included a source field for logstash to make them easier to find in Loggly.

    vim logstash-loggly.conf
    

    If you are using version 1.x, then add the following content in the file

    input{
       file{
          path => "FILENAME"
          start_position => beginning
       }
    }
    filter{
       mutate{
          add_field => ["timestamp", "%{@timestamp}"]
       }
    }
    output{
       loggly{
          key => "TOKEN"
          host => "logs-01.loggly.com"
          proto => "https"
       }
    }
    

    If you are using version 2.x, then add the following content in the file

    input{
       file{
          path => "FILENAME"
          start_position => beginning
       }
    }
    filter{
       mutate{
          add_field => ["timestamp", "%{@timestamp}"]
       }
    }
    output{
       loggly{
          key => "TOKEN"
          tag => "logstash"
          host => "logs-01.loggly.com"
          proto => "https"
       }
    }
    

    Put this file in the root folder of the logstash directory

    Replace:

    • FILENAME: your file name with absolute path whose logs are to be monitored
    • TOKEN: your customer token from the source setup page

  3. Sending Logs to Loggly
  4. Run Logstash to send the files to Loggly. This command will run it in the background. Please run it inside the root folder for Logstash.

    bin/logstash -f logstash-loggly.conf &
    

  5. Verify Events
  6. Search Loggly for events with the Logstash in json.source field over the past 20 minutes. It may take a few minutes to index the event. If it doesn’t work, see the troubleshooting section below.

    tag:logstash
    

    Click on one of the logs to show a list of JSON fields (see screenshot below). If you don’t see them, please check that you are using one of our automatically parsed formats.
    Logstash Output 01

Advanced Logstash Logging Options

Troubleshooting Logstash Logs

If you don’t see any data show up in the verification step, then check for these common problems.

    How to check it:

  • Wait a few minutes in case indexing needs to catch up
  • Check to see if the logstash-loggly.conf is properly created and placed in the proper location. It should be in the root of logstash folder downloaded from the web
  • Check if you are running commands in the proper location. You should be inside the root of the logstash folder downloaded from the web
  • Check if the file path provided in the logstash-loggly.conf file is correct
  • You can’t send JSON data to the current Loggly output plugin because it escapes the data before inserting it into the message. You can fork and modify it as needed.
  • Run “sudo tcpdump dst logs-01.loggly.com and port 80″ to verify HTTP events are being sent to Loggly
  • Still Not Working?

  • Search or post your own question in the community forum.
Thanks for the feedback! We'll use it to improve our support documentation.


Top