Input Basics

[obsolete]

Loggly inputs are a logical way for you to segment your log files.  You should use names that clearly identify what kind of data is being handled by the input. Many different servers can send logs to one Loggly input.   For example, you may want to have an input called prod-apache, which could be all of your Apache log files for production servers.  Each server that sends logs is called a device within Loggly.

Each Loggly input can handle only one input type.  For example, HTTP(S) vs Syslog TCP vs Syslog UDP.  So, if you want to send your Apache logs over HTTP(S), those will need to have a different input than your syslog events.  There are no limits to the number of inputs that you can create on Loggly, so it's best to create different inputs for anything you want to segment your data by.

It's possible to perform a search across single, multiple, or all inputs.

Adding an Input

Firstly, only Account Admins can create inputs. Ask your account administrator to upgrade your account with the Admin permissions.

If you know exactly what type of input you want to add, simply visit the "inputs" dashboard.

Service Types

Syslog UDP vs Syslog TCP

The short answer is that you should use TCP whenever possible. UDP is not a reliable means of transferring data. Each packet of data is sent from your computer to our proxy server. If UDP is used, it could happen that the packet gets lost and therefore your log entry. In the case of TCP, if a packet is dropped on the way, TCP will resend the packet and make sure that your log record arrives at Loggly. In terms of security, you are safer using TCP as well. It is not easy to spoof TCP messages, whereas UDP messages are incredibly simple to spoof.

Syslog with Stripping

Loggly can operate Syslog in two modes. The normal mode is where we index exactly what we receive via syslog. The other mode, the "stripping" mode, is where we eliminate the syslog headers before we index and store the data. Let me explain with some examples: Assume you are sending your data via straight syslog. You configured your device to log via syslog to Loggly. In this case, you would like to preserve all the syslog headers that the original syslog added to the event. An example of an event looks as follows:
 

This is the normal case and you would use the input type of "Syslog UDP" or "Syslog TCP". If you configured "Syslog UDP Header Stripping" or the TCP analogous entry, you would end up with the following event being indexed in Loggly (note how the header is stripped off):

The case where you should be using the "Syslog UDP Header Stripping" is where you are tunneling events through syslog. For example, if you are reading from a file with SyslogNG and forward the content of the file to Loggly, you want to enable the stripping option. This is the case, for example, when you monitor Apache logs. The following shows what happens when you don't enable header stripping:

Notice how you end up with an additional syslog header. You don't want that. That is why you use the "Syslog UDP Header Stripping" option to preserve only the original part of your log entry:

Syslog on port 514

Port 514 is commonly used for sending logs to a remote location from a server or router.
Some syslog servers, such as those used in routers or other network devices, may not support sending log data over anything but UDP port 514.  This has the unfortunate side effect of preventing you from changing the destination to a port Loggly uses to listen for your data.
In order to get log data to Loggly, you'll need to first forward it to a local syslog collector that can listen on port 514.  That syslog collector will then be able to forward the log data to your personal port address at Loggly.

HTTP/HTTPS

There are a few reasons that you may choose to set up HTTP/S inputs for your logs:

  • You may prefer to have your application send logs directly to Loggly instead of through syslog (Python, Ruby, Apache)
  • Client-side logging on a web site (Javascript)
  • Debugging purposes from a script, using our API
  • Secure logging via syslog proxy (but sent over HTTPS)
  • Forward

Note: At this time it is not possible to add "Devices" (IP addresses) to HTTP(S) inputs (This is reserved for Syslog TCP, UDP, and TLS inputs).  By default HTTP(S) accepts all events from IP's which POST to its URI (https://logs.loggly.com/inputs/<unique-http-key>).  

Secure Syslog

Discovery Mode

When you create an input, it is automatically in discovery mode, which means that you can send logs to that port from any machine you like. After 15 minutes, the input will fall out of discovery mode. It basically blocks any connection to that port. Except for the machines that have been sending data to that port during the period that the input was in discovery mode. This helps prevent that other people can send data to your inputs.

At any point in time, you can click on the "discover" slider to put an input back into discovery mode for 15 minutes. The machines that are allowed to send data to an input are called "devices". You can see a list of devices that are allowed to send data to an input on the input detail page.

Troubleshooting

I fill out the add input form and click save, but my input doesn't show up in the list.

You're probably lacking the proper permissions to save an input. Please email your account owner for help.

Top