Send Amazon Cloudwatch Metrics Stats to Loggly

Metrics are stats relating to the performance of your systems. You can push your Amazon Cloudwatch metrics to Loggly using our AWS Lambda Script. Please note these instructions are for Cloudwatch Metrics Logs, which are different from Cloudwatch logs. Follow the instructions below.

AWS Setup

1. Encrypt Loggly Customer Token

Create a KMS key using the steps mentioned in the link with alias name : logglyCustomerToken
Encrypt the Loggly Customer Token using the AWS CLI

aws kms encrypt --key-id alias/logglyCustomerToken --plaintext "TOKEN"

Replace:

NOTE: Always use the latest version of AWS CLI to get all the updated features. Please make sure that the AWS CLI is configured and is using the correct AWS Access Key ID, AWS Secret Access Key and Default region name using ‘aws configure’ command.

pip install --upgrade awscli
aws configure

2. Get the Lambda Code

Clone the git repo cloudwatch-metrics-to-loggly

git clone https://github.com/psquickitjayant/cloudwatch-metrics-to-loggly.git
cd cloudwatch-metrics-to-loggly

Install required npm packages.

npm install

Edit index.js to set up encrypted customer token. Copy the base-64 encoded, encrypted token from step 1’s CLI output (CiphertextBlob attribute) and replace it with the “your KMS encrypted key” in the script.

Zip up your code:

zip -r Cloudwatch-metrics-to-loggly.zip index.js node_modules

The resulting zip (Cloudwatch-metrics-to-loggly.zip) is what you will upload to AWS in step 3 below.

3. Configure the Lambda Function

Sign in to your AWS account and open IAM console – https://console.aws.amazon.com/iam/
In your IAM console:

– Create a new Role say, ‘cloudwatch-full-access‘.

– Select Role type as ‘AWS Lambda‘ from the AWS Service Roles.

– Attach policy ‘CloudWatchFullAccess‘ and save.

Go to AWS Lambda Console. Click the “Create a Lambda function” button. (Choose “Upload a .ZIP file”). Fill the following details.

Type the name and description of your function
Select Node.js in runtime.
Upload lambda function (zip file you made above in Step 2)
Handler*: index.handler
Set Role : cloudwatch-full-access
Set memory at 256MB
Set Timer to 2 minutes.

Configure the Event Source to call Cloudwatch-metrics-to-loggly. Go to the AWS Lambda Console . Make sure the Cloudwatch-metrics-to-loggly lambda function is selected, then click ‘Actions->Add event source‘. Then fill the following details.

Event source type: CloudWatch Events – Schedule
Name: Name for event. Like (custom-schedule-five-minute - rate(5 minutes))
Description: Description for events.
Schedule expression: Select rate(5 minutes).
Enable event source: select Enable Now.
Click on submit button.
Amazon CloudWatch Metrics

4. Verify Events

Search Loggly events with the tag CloudwatchMetrics over the past 20 minutes. It may take few minutes to index the events. If this doesn’t work, see the troubleshooting section below.

tag:CloudwatchMetrics

CloudWatchMetrics Logs

Advanced AWS CloudWatch Options

Troubleshooting

If you don’t see any data appear in the verification step, then check for these common problems.

    • Make sure you’ve included your own customer token.
    • You can use multiple ways to create roles in an AWS Service
    • Make sure you have configured the same roles as mentioned above.
    • Make sure you are using the latest version of AWS CLI.
    • Check the Cloudwatch Logs for the log group of your script to see if it’s running and if there are any errors.
    • Search or post your own Amazon CloudWatchMetrics logging questions in the community forum.
Thanks for the feedback! We'll use it to improve our support documentation.


Top