The 2017 DevOps holiday survival guide: Keep calm and carry on
The 2017 holiday season is almost here! So we thought of sharing a holiday survival guide with you. With the rise in mobile shopping and fully integrated fulfillment systems, expect another spike in your traffic this holiday season. If you are part of your company’s DevOps team, you have probably been preparing for months.
In this spirit of sharing, I have put together a holiday survival guide based on the collective wisdom of Loggly’s e-commerce customers and our very own DevOps team.
- Ask: Reach out to your Sales and Marketing teams to see if there are new campaigns that might generate more traffic. You don’t want to be caught off guard because a new product or coupon leads to a spike in customer interest on your website.
- Connect: Circle back with your cloud services and infrastructure partners on their preparations and how they plan to react to any downtime.
- Extras: Estimate your traffic volume for this year. Check your scaling plan. One of the beauties of modern infrastructure options and tools is how easily you can scale capacity up and down on demand.
- Backup: Talk to your Ops teammates. Finalize your escalation process and sync (and stagger) your vacation schedules. You want to make sure your PagerDuty schedules are updated and other stakeholders such as Customer Support are also aware.
- Monitoring: Set up dashboards for key metrics in advance. You want to get a complete view of your system when stuff happens. No firefighter creates a building plan when he or she is firefighting!
- Alerts: Tune your alerts. Be ready to sniff out those sudden changes. The holiday season is going to be busy as it is. You don’t want unnecessary alerts to add to the stress.
- Write it down: Document what needs to be done when an alert is triggered. The person who responds to the pager first may not always be the expert on something that has crashed. So make it easier for everyone to troubleshoot and for everyone to identify the owner of each service and/or piece of code. Pro tip: Put these in a tool where you can add annotations and have the ability to search.
- Playbooks: Always share your playbooks and keep them handy. Name and annotate every task when you can.
- Beaten path: Stress test the most popular user flows across your website such as new account creation, credit card payments, and password reset.
- Stay positive: Take a few deep breaths. Remember that this is a festive time of year. If everything is planned and organized, you are going to have a happy holiday season!
Pranay Kamat is a Product Manager at Loggly. His previous experiences include designing user interfaces, APIs, and data migration tools for Oracle and Accela. He has an MBA from The University of Texas at Austin and Master’s degree in Computer Science from Cornell University.