Modern software systems now increasingly span cloud, on-premise, and remote embedded devices & sensors. These distributed systems bring challenges with data, connectivity, performance, and systems management, so for business success we need to design and build with operability as a first class property.
In this talk, we explore five practical, tried-and-tested, real world techniques for improving operability with many kinds of software systems, including cloud, Serverless, on-premise, and IoT:
- Logging as a live diagnostics vector with sparse Event IDs
- Operational checklists and 'Run Book dialogue sheets' as a discovery mechanism for teams
- Endpoint healthchecks as a way to assess runtime dependencies and complexity
- Correlation IDs beyond simple HTTP calls
- Lightweight 'User Personas' as drivers for operational dashboards
These techniques work very differently with different technologies. For instance, an IoT device has limited storage, processing, and I/O, so generation and shipping of logs and metrics looks very different from the cloud or Serverless case. However, the principles - logging as a live diagnostics vector, Event IDs for discovery, etc. - work remarkably well across very different technologies.
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
Practical operability techniques for teams - IPEXPO 2017
1. Practical Operability
Techniques for Teams
Matthew Skelton
Skelton Thatcher Consulting
skeltonthatcher.com / @SkeltonThatcher
IPEXPO Europe, ExCeL, London – 05 October 2017
2. Today
What is operability?
Modern logging
Run Book dialogue sheets
Endpoint healthchecks
Correlation IDs
User Personas for dashboards
5. Operability:
use modern logging, Run Book
dialogue sheets, endpoint
healthchecks, correlation IDs,
and user personas as
team collaboration techniques
31. Example: video processing
Discover processing bottlenecks
Trigger alerts via LogEntries /
HostedGraphite
Report on KPIs
Target areas for improvement
36. System characteristics
Hours of operation
During what hours does the service or system actually need to operate? Can portions or features of the
system be unavailable at times if needed?
Hours of operation - core features
(e.g. 03:00-01:00 GMT+0)
Hours of operation - secondary features
(e.g. 07:00-23:00 GMT+0)
Data and processing flows
How and where does data flow through the system? What controls or triggers data flows?
(e.g. mobile requests / scheduled batch jobs / inbound IoT sensor data )
…
41. endpoint healthchecks
Every runnable app/service/daemon
exposes /status/health
An HTTP GET to the endpoint returns:
200 – "I am healthy"
500 – "I am sick"
49. Synchronous HTTP:
X-HEADER e.g. X-trace-id
X-trace-id: 348e1cf8
If header is present, pass it on
(Yes, RFC6648, but this is internal only)
50. Asynchonous (queues, etc.):
Message Attributes, name:value pair
e.g. "trace-id":"348e1cf8"
AWS SQS: SendMessage() / ReceiveMessage()
Log the Correlation ID if present
51. Example: electronic trading
High speed, low latency
Trading options & derivatives
Connected to stock exchanges
Sub-millisecond timings
> £1 million per day traded
52.
53.
54.
55. Correlations IDs for trading
Evidence for timely operation
Help identify bottlenecks
Target areas for perf tuning
Identify race conditions
Increase operability
70. Operability
use modern logging, Run Book
dialogue sheets, endpoint
healthchecks, correlation IDs,
and user personas as
team collaboration techniques
71. Team Guide to
Software Operability
Matthew Skelton & Rob Thatcher
skeltonthatcher.com/publications
Download a free sample chapter
73. Resources
• Training: Practical Operability for Developers and Testers – led
by Matthew Skelton and Rob Thatcher – 1-day workshop –
http://www.unicom.co.uk/practical-operability-for-developers-
and-testers.html
• Team Guide to Software Operability by Matthew Skelton and Rob
Thatcher (Skelton Thatcher Publications, 2016)
http://operabilitybook.com/
• Run Book template & Run Book dialogue sheets
http://runbooktemplate.info/