5 Best Practices to Optimize Log Management for DevOps, SREs, and IT Operations

1. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Taming Multi-Cloud, Hybrid Cloud, Docker, and Kubernetes:
Five Best Practices to Optimize Log Management for DevOps,
SREs, and IT Operations
EMA
Torsten Volk (Research Director)
Twitter: @torstenvolk
SolarWinds
Bradley Sisson (Technical Account Manager)
Email: bradley.sisson@solarwinds.com

2. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Agenda
Slide 2 © 2019 Enterprise Management Associates, Inc.
Taming Multi-Cloud, Hybrid Cloud, Docker,
and Kubernetes:
Five Best Practices to Optimize Log
Management for DevOps, SREs, and IT
Operations
Agenda
• Five critical questions to ask log
analytics vendors
• Balance log analytics
requirements between traditional
and modern apps
• Impact of log analytics on security
and compliance
• Increase DevOps efficiency and
lower operational risk

3. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Unified Monitoring of Data Center and Public Cloud
Rapid increase in application complexity between 2016 and today
Slide 3 © 2019 Enterprise Management Associates, Inc.
335
714 685
883
1,908 5,214 (+173%) 5,018 (-4%) 6,359 (+27%)
October 2019October 2018October 2017October 2016
Each dot represents a category of developer questions on stackoverflow.com (on Nov 15, 2019)

4. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Unified Monitoring of Data Center and Public Cloud
62% increase in AWS complexity
Slide 4 © 2019 Enterprise Management Associates, Inc.
2018 2019 YTD
12,957 issue categories 20,950 issue categories (+62%)
Data source: stackoverflow.com

5. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Unified Monitoring of Data Center and Public Cloud
Slide 5 © 2019 Enterprise Management Associates, Inc.
Top 10 fastest growing public cloud services
between January 2018 and today
The rise of DevOps and continuous delivery
brings further complexity
Data source: devopshelp.com

6. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Unified Monitoring of Data Center and Public Cloud
Slide 6 © 2019 Enterprise Management Associates, Inc.
335 714 (+113%) 685 (-4%) 883 (+29%)
October 2019October 2018October 2017October 2016
Application complexity directly translates into infrastructure complexity
Data source: serverfault.com

7. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Monitor Bare Metal, VMs, Kubernetes, and Serverless
Slide 7 © 2019 Enterprise Management Associates, Inc.
36 different databases 15 networking options 16 storage platforms
22 automation platforms 12 container runtimes 5 container registries
Data source: CNCF dataset

8. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Monitor Bare Metal, VMs, Kubernetes, and Serverless
Slide 8 © 2019 Enterprise Management Associates, Inc.
And 35 application management technologies… … and
• 12 API gateways
• 25 service mesh
platforms
• 21 Kubernetes
distributions
• And counting
Data source: CNCF dataset

9. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Monitor Bare Metal, VMs, Kubernetes, and Serverless
Slide 9 © 2019 Enterprise Management Associates, Inc.
Serverless
technologies grew
1,200% since 2016
Data source: stackoverflow.com

10. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Monitor Bare Metal, VMs, Kubernetes, and Serverless
Slide 10 © 2019 Enterprise Management Associates, Inc.
Serverless added over
1,000 more problem
categories
Data source: StackOverflow.com

11. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Jointly Monitor Logs and Metrics
Slide 11 © 2019 Enterprise Management Associates, Inc.
Issues related to monitoring logs and
metrics doubled since Q1, 2018.
Logs and metrics are often monitored
separately
Data source: stackoverflow.com

12. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Simplify Instrumentation for Developers
Consistency is critical for
code instrumentation
Recent progress:
OpenTracing and
OpenCensus merged into
OpenTelemetry
Slide 12 © 2019 Enterprise Management Associates, Inc.
Source: stackoverflow.com

13. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Jointly Monitor Metrics and Logs
Slide 13 © 2019 Enterprise Management Associates, Inc.
Metrics Logging
Data source: github.com

14. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Guidance for Operators
Rapid growth of
microservices
(orange) and
serverless
applications (red)
Result: Logs and
metrics becoming a
bigger challenge
Slide 14 © 2019 Enterprise Management Associates, Inc.
Source: stackoverflow.com

15. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Guidance for Operators
Slide 15 © 2019 Enterprise Management Associates, Inc.
Source: stackoverflow.com
Microservices have
added tremendous
operational
complexity

16. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Summary
Slide 16 © 2019 Enterprise Management Associates, Inc.
• Simplicity is key
• Eliminate all silos
• Applications and infrastructure are becoming
one
• Unified logs, metrics, and traces are the basis
for continuous compliance
• No excuses