Python Logging Over HTTP/S

You can send Python logs to Loggly using the Loggly’s Python logging handler package. This is an asynchronous and non-blocking handler that uses requests-futures to send logs. It also supports JSON and multiline events. For alternatives, please see the Advanced Options section.

Python Logging Setup

  1. Install loggly-python-handler using pip
  2. Install the loggly-python-handler library from the Python Package Index using pip or from GitHub. This handler is better to use if you are hosted in a PaaS and don’t have access to the system syslog daemon, or if you are logging from a client-side script or app.

    sudo pip install loggly-python-handler
    

  3. Add HTTPSHandler to the Configuration file
  4. Add a configuration file python.conf with the following contents in it.

    [handlers]
    keys=HTTPSHandler
    
    [handler_HTTPSHandler]
    class=loggly.handlers.HTTPSHandler
    formatter=jsonFormat
    args=('https://logs-01.loggly.com/inputs/TOKEN/tag/python','POST')
    
    [formatters]
    keys=jsonFormat
    
    [loggers]
    keys=root
    
    [logger_root]
    handlers=HTTPSHandler
    level=INFO
    
    [formatter_jsonFormat]
    format={ "loggerName":"%(name)s", "asciTime":"%(asctime)s", "fileName":"%(filename)s", "logRecordCreationTime":"%(created)f", "functionName":"%(funcName)s", "levelNo":"%(levelno)s", "lineNo":"%(lineno)d", "time":"%(msecs)d", "levelName":"%(levelname)s", "message":"%(message)s"}
    datefmt=
    

    Replace:

  5. Use Configuration in python file
  6. Write the following code inside your Python source file to send logs to Loggly over HTTPS

    import logging
    import logging.config
    import loggly.handlers
    
    logging.config.fileConfig('python.conf')
    logger = logging.getLogger('myLogger')
    
    logger.info('Test log')
    

    It assumes that python.conf and your source file are placed in the same directory. Run your application and send some logs.

  7. Django Logging
  8. To setup Django logging use the following configuration in your settings.py file

    LOGGING = {
        'version': 1,
        'disable_existing_loggers': False,
        'formatters': {
            'verbose': {
                'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
            },
            'simple': {
                'format': '%(levelname)s %(message)s'
            },
            'json': {
                'format': '{ "loggerName":"%(name)s", "asciTime":"%(asctime)s", "fileName":"%(filename)s", "logRecordCreationTime":"%(created)f", "functionName":"%(funcName)s", "levelNo":"%(levelno)s", "lineNo":"%(lineno)d", "time":"%(msecs)d", "levelName":"%(levelname)s", "message":"%(message)s"}',
            },
        },
        'handlers': {
            'console': {
                'class': 'logging.StreamHandler',
                'level': 'DEBUG',
                'formatter': 'verbose',
            },
            'loggly': {
                'class': 'loggly.handlers.HTTPSHandler',
                'level': 'INFO',
                'formatter': 'json',
                'url': 'https://logs-01.loggly.com/inputs/TOKEN/tag/python',
            },
        },
        'loggers': {
            'django': {
                'handlers': ['console', ],
                'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
            },
            'APPNAME': {
                'handlers': ['console', 'loggly'],
                'level': 'INFO',
            },
        },
    }
    

    Replace:

    • TOKEN: your customer token from the source setup page
    • APPNAME: your Django application name

  9. Verify Events
  10. Search Loggly for events with the tag as python over the past 20 minutes. It may take few minutes to index the event. If it doesn’t work, see the troubleshooting section below.

    tag:python

    Python Logging

Advanced Python Logging Options

  • Python Syslog Handler – You can use Syslog Handler to send logs to syslog.
  • Django Logging – Step by step guide for setting up Django logging
  • LogglyAppEngine – If you use Google App Engine use this and set the endpoint to logs-01.loggly.com
  • Urllib2 – If you would like to send requests directly to Loggly over HTTP
  • Loggly Libraries Catalog – There are more libraries available to send Python logs to Loggly.

Troubleshooting Python Logs

If you don’t see any data show up in the verification step, then check for these common problems.

  • Wait a few minutes in case indexing needs to catch up
  • See our HTTP Troubleshooting Guide to verify HTTP events are being sent to Loggly.
  • Search or post your own Python logging configuration questions, or questions how to import logging to Loggly, logging levels, logging modules, and more in the community forum.
Thanks for the feedback! We'll use it to improve our support documentation.


Top