Why Should I Archive My Logs?

A good log management solution provides a glimpse into the state of a system over a period of time. It can help you identify performance trends, audit security events, and pinpoint periods of high demand. Archiving logs can drive cost savings while allowing you to extend the time frame in which you can use your log data to make informed decisions.

Compliance

For compliance reasons, archiving your logs ensures that you’re fully protected. Data retention policies can vary from several months to several years depending on the type of service you provide and standard or regulation with which you need to comply. Some might specify minimum log retention periods, but almost all will require an audit trail.

Trend Spotting

Logs are essential for identifying and troubleshooting short-term problems, but are less effective at identifying long-term trends. Older entries may get overwritten, deleted, or lost. Archives make it easier to identify patterns over a longer period of time than rolling log files.

How to Build Your Log Archive

Creating and maintaining a log archive can be a daunting task. Even a relatively small architecture can generate gigabytes of log data from operating systems, applications, and services. When considering what to archive, ask yourself these questions:

1. How much should I keep?
2. How can I organize and search the results?
3. How often do I expect to access archived log data, and is retrieval speed a concern?

Determining What to Keep

For auditing purposes, your archives typically store events that affect security, integrity, and performance. This may include:

• Logs that you are required to retain to meet a regulatory requirement
• Administrator activity
• User actions (logins, commands, etc.)
• Errors, exceptions, and warnings

With Loggly, you can create a filter that limits the type of logs stored in your archive.

Storing Logs

Even with a filter in place, the amount of space required for your archive will climb rapidly. Fortunately, Loggly allows you to specify an Amazon S3 bucket as a destination for new logs. Logs stored in Amazon S3 remain indefinitely or until you delete them. You can also define an access policy within Amazon S3 that limits access to your logs to certain users or IP addresses.

Once your archive is set up, it’s time to start logging. Loggly can help you build a comprehensive, compliant, and effective log archive.

The Loggly and SolarWinds trademarks, service marks, and logos are the exclusive property of SolarWinds Worldwide, LLC or its affiliates. All other trademarks are the property of their respective owners.