Send Amazon Cloudfront logs to Loggly

You can push your Amazon Cloudfront logs to Loggly using an AWS Lambda Script, originally created by Quidco.com. It converts the Cloudfront gzipped logs written to S3 into JSON format and then sends them to Loggly. See the list of fields of an access log on AWS Cloudfront Documentation.

AWS Setup

  1. Get the Lambda Code
  2. Clone the git repo

    git clone https://github.com/fgheorghe/cloudfront2loggly.git
    cd cloudfront2loggly

    Edit cloudfront2loggly.js with the proper Loggly customer token.

    logglyConfig = {
        tag: "aws-cloudfront",
        token: "TOKEN"
    };
    

    Replace:

    Install required npm packages.

    npm install

    Zip up your code

    zip -r cloudfront2loggly.zip cloudfront2loggly.js node_modules

    The resulting zip (cloudfront2loggly.zip) is what you will upload to AWS in step 2 below.

  3. Configure the Lambda Function
  4. Go to AWS Lambda Console Console. Click the “Create a Lambda function” button. (Choose “Upload a .ZIP file”). Fill the following details.

    Name: cloudfront2loggly
    Upload lambda function (zip file you made above in Step 1)
    Handler*: cloudfront2loggly.handler
    Role*: In the drop down click "S3 execution role". (This will open a new window to create the role, click Allow)
    Set memory at 128MB
    Set Timer to 10 seconds.
    

    Configure the Event Source to call cloudfront2loggly when logs are added to the S3 bucket. Go to the AWS Lambda Console . Make sure the cloudfront2loggly lambda function is selected, then click ‘Actions->Add event source‘. Then fill the following details.

    Event source type: S3
    Bucket: Choose the bucket that contains your Cloudfront logs.
    Event type: ObjectCreated (All)

  5. Configure Cloudfront Logging
  6. Goto the Cloudfront app. In your distribution settings, enable logging and select the S3 bucket for logs.

  7. Verify Events
  8. Search Loggly events with the tag aws-cloudfront over the past 20 minutes. It may take a few minutes to index the events. If if doesn’t work, see the troubleshooting section below.

    tag:aws-cloudfront

    cloudfront

    Advanced AWS Cloudfront Options

    Troubleshooting

    If you don’t see any data show up in the verification step, then check for these common problems.

    • Wait a few minutes in case indexing needs to catch up
    • Make sure you’ve included your own customer token
    • Make sure you have configured same roles as mentioned above.
    • Search or post your own Amazon Cloudfront logging questions in the community forum.
    Thanks for the feedback! We'll use it to improve our support documentation.


Top