EXTENT-2016: Software Testing & Trading Technology Trends 22 June, 2016, 10 Paternoster Square, London Delivering Quality from Chaos Jeremy Norwood, Programme Director, Tradition Would like to know more? Visit our website: extentconf.com Follow us: https://www.linkedin.com/company/exactpro-systems-llc?trk=biz-companies-cym https://twitter.com/exactpro #extent2016 #exactpro

1. Delivering Quality From Chaos
Jeremy Norwood

2. Project Delivery
“is providing a solution on time, on
budget and to the required quality that
solves the identified problem”

3. Quality
“is the degree of conformance to
explicit or implicit requirements and
expectations ”

4. Chaos
“is the property of a complex system
whose behaviour is so unpredictable as
to appear random, owing to great
sensitivity to small changes in
conditions”

5. Content
1. Contributors
2. Symptoms
3. Diagnosis
4. Survival
5. Summary/Q&A

6. CONTRIBUTORS – Industry Sector

7. CONTRIBUTORS – Product & Technology
Sector
Context
Product Diversity
Asset Class Variance
Exchange vs OTC
Cleared vs Not
Client Diversity
Institutional to
Individual
Human to Machine
Reg. Diversity
Regulated vs Not
Regional & Global
Differences
Software
Programming
Languages
Too Many to Name
Development
Methodologies
Immature & Evolving
Multi Dimensional
Challenges
Complex Logic,
Latency, Big Data
Hardware
Network
Global Data Super
Highway
Servers / Storage
Phys vs VM vs Cloud
Linux vs Windows
Embedded Hardware
& O/S Programming
Highly Skilled & Niche
Testing
Testing Coverage
Never Enough
Scenarios
Multi Layer
Func & Integration
Perf & Capacity
Security & DR
Test Tools
Swiss Army knife
required

8. CONTRIBUTORS – People & Culture
Culture &
Language &
Social
Company
Culture &
Values
Team
Behaviours &
Dynamics
Individual
Personalities
& Needs

9. SYMPTOMS – Recognising the Signs
• Overly Complex and Inflexible Solutions
• Low Innovation
• High Defect Rates and Poor Quality
• Unstable Operations and Costly to Maintain
Technology
• Long Hours
• Poor Collaboration Between Teams
• Low Level of Trust
• High Staff Turnover
People
• Missed Deliverables & Milestones
• High Levels of Rework
• Cost Overruns
• Low Levels of Continuous Improvement
Project

10. DIAGNOSIS – How Chaotic are Things?
Low Chaos =============================================> Extreme Chaos
1 2 3 4 5 6 7 8 9 10
Well Known/Defined Requirements Exploratory/Fluid
Low Speed High
Flexible/Long Term Deadlines Fixed/Short Term
Low Innovation High
Low Change & Uncertainty High
Low Dependencies & Interdependencies High
Dedicated Team Availability Multitasking
Available Sponsor Unavailable
Available Customer Unavailable
Low Technical Complexity High
Low Org Complexity & Politics High
Co-Located Geographical Diversity Dispersed
Low Market Complexity High

11. SURVIVAL – Be Realistic, Consider Future Obstacles
• Expect uncertainty
– Little or no relationship between cause and
effect
– Manage through small iterations, anticipation
and adaptation
– Have contingency plans ready to go
– Challenge assumptions

12. SURVIVAL – Keep it Simple and Be Agile
• Keep it Simple
• Be Agile
– “Having A Quick, Resourceful And Adaptable
Character”
– A way of thinking, not a defined process
Simple clear purpose and process give rise
to complex intelligent behaviours.
Complex rules and regulations give rise to
simple, stupid behaviours.
Dee Hock, Founder Visa Credit Cards

13. SURVIVAL – Synchronise Activities
• Apply Dynamic Planning & Control
– Act, Sense, Respond
• Use best fit Dev and Test lifecycle
– Ensure teams are synchronised
• Avoid Domino Effect
– Decouple activities and create Buffers
– Manage “technical debit” built up over time

14. SURVIVAL – Understand the Dependencies
Functional/
Non Functional
Project
Deliverables
Software/
Hardware

15. SURVIVAL – Use Data
• Consider all sources of data
• Analyse and identify patterns
– Don’t focus on root cause (no relationship between cause &
effect)
• Make small and quick decisions, check impact and adjust
– Remember chaotic environments are sensitive to small changes

16. SURVIVAL – Advanced Strategies
• Chaos Engineering/Testing
– Failure is unavoidable so deliberately introduce (via a
Monkey)
• Latency monkey, Security Monkey, Janitor Monkey etc
– http://principlesofchaos.org/
– http://testingsoftwarehelp.com/what-is-chaos-
testing-engineering/
• Extreme Project Management
– http://www.dougdecarlo.com/

17. IN SUMMARY
• Recognise and embrace chaos
– It’s in the DNA of Finance sector and Technology
• Understand your environment
– Consider all the influencing factors
• Combat complexity with agility
– Adopt an agile and flexible mind set
– Look ahead and anticipate uncertainty
– Use data to inform decision making and check impact