The value of software is only potential value until it is in users’ hands. There can be many roadblocks to software getting into those hands. These roadblocks tend to revolve around elaborate deployment pipelines stemming from Configuration Management Debt:
* Over-burdened release engineering and operations teams
* High coupling with centrally managed architecture element/component
* Source control practices that impact delivery velocity
* Too many variations/versions of the software supported in production
* Poor integration processes across architecture components and scaled team delivery
* Too many hand-offs between teams in order to release software to users
* Code changes feel too risky and takes too long to validate before releasing into production
* Poor documentation practices
In organizations that have effective configuration management practices it is common to see deployment pipelines that have a smaller number of hand-offs between teams, architectures that tend to be more malleable, and efficient validation processes. By focusing on reducing Configuration Management Debt it is simpler to identify aspects of the integration and release management process that need to be tackled in order to get working software in the hands of users sooner while reducing the bottlenecks in the organizational processes and practices.
In this session we will discuss specific approaches and examples on how reducing Configuration Management Debt leads to reducing other forms of software debt including:
* Smaller number of hand-offs: Platform Experience Debt
* Malleable architectures: Design Debt
* Efficient validation processes: Quality Debt
* More testable software: Technical Debt
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Reduce Time to Value: Focus First on Configuration Management Debt
1. Reduce Time to Value
Focus First on Configuration Management Debt
Chris Sterling, Product Owner, CenturyLink Cloud
2. Bio - Chris Sterling
2
Director of Product Management at
CenturyLink Cloud
Recently Launched AppFog:
https://www.ctl.io/appfog
Author of book “Managing Software Debt:
Building for Inevitable Change”
Supported change efforts to adopt Lean,
Agile & Continuous Delivery behaviors in
organizations of 10 up to 800+ people
Entrepreneur & Lean Startup Practitioner
Blog: managingsoftwaredebt.com
4. Types of Software Debt
Configuration Management Debt: Integration and release management become
more risky, complex, and error-prone.
Platform Experience Debt: The availability and alignment of people to business
objectives that involve software changes is becoming more limited or cost-prohibitive.
Design Debt: The cost of adding features is increasing toward the point where it is more
than the cost of writing from scratch.
Quality Debt: There is a diminishing ability to verify the functional and technical quality
of software: the “Break/Fix” mentality.
Technical Debt: These are the activities that a team or team members take shortcuts
on now that will impede future development if left as is.
4
5. Reducing Time To Value
5
Focusing on Configuration Management Debt first leads
to opportunities for reducing all forms of software debt
Reduce hand-offs and dependencies in Org to reduce
Platform Experience Debt
Malleable architectures with Pluggable UI,
Microservices and APIs to reduce Design Debt
Increase efficiency of validation processes to reduce
Quality Debt
Make software more testable to reduce Technical Debt
7. Case Study: Web Property
7
180+ person “Web 2.0” product organization
Waterfall SDLC that development uses to deliver in 6 month release
cycles
Want to use Agile methods to be more responsive to users and keep up
with other “Web 2.0” companies
Transitioned to Agile methods on 15 teams in 3 months
Changed release management strategy, added XP technical practices,
and implemented Scrum product development framework for scaled
coordination
Able to release every week to users within 4 months
Used streamlined deployment environment process to validate product
changes daily using Continuous Integration and automated promotions
8. Case Study: Ad Platform
8
700+ person Ad Platform organization
Extended dependencies out to 2000+ people throughout company
Millions of lines of code + Hundreds of unique apps & services
Continuous Integration server involved thousands of jobs & 1M+
builds per year
9-18 month overlapping release cycles
Found opportunity to not branch for each overlapping release
After 2 years can release to 20+ data centers globally every day
4 years later they release to production with no humans for nearly
100% of their company application & service deployments
10. Where Does CM Debt
Source control practices that impact delivery velocity
Too many variations/versions of the software supported in production
Over-burdened release engineering and operations teams
High coupling with centrally managed architecture element/
component
Too many hand-offs between teams in order to release software to users
Poor integration processes across architecture components and scaled
team delivery
Code changes feel too risky and takes too long to validate before
releasing into production
Poor documentation practices
10
15. Traditional Source Control
11
Main BranchDebt
Death March
{
Debt accrues quickly within stabilization periods
Version 1
Branch
Integrate for
Version 2
Code
Complete
32. Feature Team
“Feature Team” structure
Uses common Product Backlog
Integration is done in parallel
Requires high levels of communication
across teams to resolve integration
issues
Forces Teams to be more coordinated
Sprints should be synchronized
Cross team fertilization is a
requirement to successfully
deliver in parallel
23
33. Design Debt
Technical features that involve improving
software quality attributes can be prioritized
based on the cost of not addressing them.
34. Design Debt
Technical features that involve improving software
quality attributes can be prioritized based on the
cost of not addressing them.
36. Feature Toggles
Problem: Need to deliver changes to production on master
with features that are not sufficient or validated with customers
Introduce feature behind toggle
▶ ON: Show and allow access to feature
▶ OFF: Don’t show or allow access to feature
Should be secondary option to introducing smaller
capabilities that lead to sufficient feature
27
37. Microservices
28
Pluggable UI
API Routing
Service 1 Service 2 Service 3 Service 4
Identity Management
Authorization Authorization Authorization Authorization
Data Data Data Data
Messaging Platform
41. The Three Amigos
Quickly get testers,
coders, and business on
the same page before
building based on a
requirement
32
* The Three Amigos pattern originally coined by George Dinwiddie
http://www.stickyminds.com/s.asp?F=S17232_COL_2
42. The Three Amigos Pattern
33
As a Shopper I want to
receive updates on incredible
deals that are located near
my home so that I can save
money on my purchases
Acceptance Criteria:
Save Shopper’s location
Ask Shopper if they want to receive
localized deals daily
Send notification of incredible deals to
Shoppers located within 10 miles each
morning
Allow Shopper to stop receiving localized
deals
43. The Three Amigos Pattern
34
As a Shopper I want to
receive updates on incredible
deals that are located near
my home so that I can save
money on my purchases
What areas of the application will this
affect?
What is the overall design? (UI, API, UX,
etc…)
What are the details test cases for this user
story and it’s acceptance criteria?
What about negative test conditions?
What about boundary conditions?
How might we put existing functionality at
risk?
44. The Three Amigos
At minimum include tester,
coder & business rep in
discussion
Should only take 30 minutes to 1
hour for user stories
Focus on clarification and design
through testable inputs/outputs
Extend to Operations to reduce
Configuration Management Debt
35
47. Manual Regression Testing
Testing was taking 75 person hours during 2 full test
runs consisting of:
▶ Comprehensive manual regression testing
▶ Data conversion and validation
Cost for testing was $17,000 each iteration
38
48. Introducing Fit into Testing
After 8 iterations team had introduced healthy
amount of Fit fixtures and automated tests
Reduced 70+ hour test runtime down to 6 hours which
now included:
▶ Fit automated regression testing
▶ Data conversion and validation automated with Fit
fixtures
Reduced cost of testing each iteration from $17,000 to
$7,000
39
49. The Agile Regression Testing
40
* The Agile Triangle has been modified from Mike Cohn’s original version
50. The Agile Regression Testing
40
* The Agile Triangle has been modified from Mike Cohn’s original version
Automated Unit Tests
Make up largest portion of
regression tests and are
developed by programmers
51. The Agile Regression Testing
40
* The Agile Triangle has been modified from Mike Cohn’s original version
Automated Unit Tests
Make up largest portion of
regression tests and are
developed by programmers
Integration Tests
Automated &
Exploratory
52. The Agile Regression Testing
40
* The Agile Triangle has been modified from Mike Cohn’s original version
Automated Unit Tests
Make up largest portion of
regression tests and are
developed by programmers
Integration Tests
Automated &
Exploratory
Smoke++ Tests
Risk-based UI &
API Automated
Tests
53. Technical Debt
“For every [dollar] of competitive advantage gained by cutting
quality, it costs $4 to restore it; and software is an organizational
asset and decisions to cut quality must be made by executive
management and reflected in the financial statements.”
— Ken Schwaber, co-creator of Scrum
Source: http://www.infoq.com/presentations/agile-quality-canary-coalmine
57. Early Warning Dashboard
45
Measuring Early Warning Signs:
• Design Debt in Duplication (DRY)
• Technical Debt in Code Complexity
• Quality Debt in Bug DB (Break/Fix)
• Other Custom Thresholds
58. Reducing Time To Value
46
Focusing on Configuration Management Debt first leads
to opportunities for reducing all forms of software debt
Reduce hand-offs and dependencies in Org to reduce
Platform Experience Debt
Malleable architectures with Pluggable UI,
Microservices and APIs to reduce Design Debt
Increase efficiency of validation processes to reduce
Quality Debt
Make software more testable to reduce Technical Debt