This document provides an overview of managing software debt in practice, with a focus on quality debt. It discusses the different types of software debt including technical, quality, configuration management, design, and platform experience. It emphasizes the importance of asserting quality through practices like having a clear definition of done and implementing test automation. The document outlines how one team significantly reduced their testing costs from $17,000 to $7,000 per iteration by introducing the Fit framework and automating their regression tests.

1. Managing
So)ware
Debt
in
Prac2ce
Quality
Debt
Focus

2. First
Things
First...
Get Rally!
www.rallydev.com
@csterwa
#swdebt
2

3. Chris
Sterling
• Director
of
Engineering
at
Rally
So)ware
in
Kirkland,
WA
office
• Author
of
Book
Managing
So)ware
Debt:
Building
for
Inevitable
Change
• Consults
on
so)ware
technology,
Agile
technical
prac2ces,
Scrum,
and
effec2ve
management
techniques
• Innova2on
Games®
Trained
Facilitator
Email:
csterling@rallydev.com
Web:
hWp://www.rallydev.com
• Cer2fied
Scrum
Trainer
Follow
me
on
TwiWer:
@csterwa
Blog:
hWp://www.geYngagile.com
Hashtag
for
presenta2on:
#swdebt
• Open
Source
Developer
3

4. Agenda
• Managing
So)ware
Debt
• The
3
Amigos
PaWern
Overview
• Test-­‐Driven
Design
• So)ware
Debt
Types
• Monitoring
Quality
• Technical
• The
Power
of
2
Scripts
• Quality
• Con2nuous
Integra2on
• Configura2on
Management
• Automated
Promo2on
• Design
• Advanced
Quality
Metrics
• Pla]orm
Experience
Trending
and
Analysis
• Asser2ng
Quality
• Wrap
Up
• Defini2on
of
Done
• So)ware
Debt
Management
Strategy
• Test
Automa2on
• The
No
Defect
Mindset
4

5. Managing
So)ware
Debt
Overview
Let
me
tell
you
a
poten2ally
familiar
story…

7. Lack
of
emphasis
on
so)ware
quality
aWributes
contributes
to
decay

8. Principle:
No
maWer
what,
the
cost
of
addressing
so)ware
debt
increases
with
2me.

9. Types
of
So)ware
Debt
Technical,
Quality,
Configura2on
Management,
Design,
and
Pla]orm
Experience

10. Why
not
just
call
it
all
Technical
Debt
• Technical
debt
tended
to
focus
more
on
programming
aspects
of
so)ware
delivery
and
le)
out
full
so)ware
development
life
cycle
• Each
type
of
so)ware
debt
can
be
managed
and
monitored
using
different
tools
and
approaches
• Focusing
on
managing
each
type
of
so)ware
debt
simplifies
crea2on
of
an
overall
strategy
that
promotes
holis2c
perspec2ve
10

11. Types
of
So)ware
Debt
• Technical
Debt:
These
are
the
ac2vi2es
that
a
team
or
team
members
take
shortcuts
on
now
that
will
impede
future
development
if
le)
as
is.
• Quality
Debt:
There
is
a
diminishing
ability
to
verify
the
func2onal
and
technical
quality
of
so)ware:
the
Break/Fix
mentality.
• Configura2on
Management
Debt:
Integra2on
and
release
management
become
more
risky,
complex,
and
error-­‐prone.
• Design
Debt:
The
cost
of
adding
features
is
increasing
toward
the
point
where
it
is
more
than
the
cost
of
wri2ng
from
scratch.
• Pla]orm
Experience
Debt:
The
availability
and
alignment
of
people
to
business
objec2ves
that
involve
so)ware
changes
is
becoming
more
limited
or
cost-­‐prohibi2ve.
8

12. Asser2ng
Quality
Teams
must
focus
on
asser2ng
sustainable
quality
to
support
future
customer
needs
in
a
2mely
manner

13. Defini2on
of
Done
So)ware
developments
assert
what
finished,
working
so)ware
entails
to
support
predictable
delivery
into
the
future

14. Defini2on
of
Done
-­‐
Assert
Quality
" Acceptance defined criteria for each " Tested on FE
user story
" Integration test written & passes
" Unit tests written and passed
" Test code reviewed
" Code compiles with no errors and no
" Environment requirements documented
warnings
" New code doesn t break existing code
" Interface document updated/added and
checked in to SVN
" Test case review (Dev to review test
" Acceptance criteria verified complete
case written)
" Architectural impact assessed and
" All P1-P3 bugs for the story are closed
artifacts updated if necessary " Test approves user story
" Comments in code " Story demonstrated to product owner
and accepted on Target Platform
" Error codes added
" Code reviewed by peer
" Code checked in with reference to
US#/Task#
14

15. Release
Defini2on
of
Done
• Every
release
should
have
clear
quality
criteria
• With
a
Release
Defini2on
of
Done
you
can
understand
targets
beWer
• Measure
the
gap
between
the
teams
Defini2on
of
Done
and
a
Release
Defini2on
of
Done.
• This
gap
is
a
source
of
quality
issues
and
represents
significant
risk
to
schedule

16. Case
Study:
Test
Automa2on
Reduces
Cost
of
Change

17. Manual
Regression
Tes2ng
• Tes2ng
was
taking
75
person
hours
during
2
full
test
runs
consis2ng
of:
• Comprehensive
manual
regression
tes2ng
• Data
conversion
and
valida2on
• Cost
for
tes2ng
was
$17,000
each
itera2on
17

18. Introducing
Fit
into
Tes2ng
Process
• A)er
8
itera2ons
team
had
introduced
healthy
amount
of
Fit
fixtures
and
automated
tests
• Reduced
70+
hour
test
run2me
down
to
6
hours
which
now
included:
• Fit
automated
regression
tes2ng
• Data
conversion
and
valida2on
automated
with
Fit
fixtures
• Reduced
cost
of
tes2ng
each
itera2on
from
$17,000
to
$7,000
18

19. The
Agile
Regression
Tes2ng
Triangle*
Smoke++
Tests
Risk-­‐based
UI
&
API
Automated
Integra2on
Tests
Tests
Automated
&
Exploratory
Automated
Unit
Tests
Make
up
largest
por2on
of
regression
tests
and
are
developed
by
programmers
*
The
Agile
Triangle
has
been
modified
from
Mike
Cohn’s
original
version
19

20. The
3
Amigos
PaWern*
Quickly
get
testers,
coders,
and
business
on
the
same
page
before
building
based
on
a
requirement
*
The
Three
Amigos
paWern
originally
coined
by
George
Dinwiddie
hWp://www.s2ckyminds.com/s.asp?F=S17232_COL_2

21. The
Three
Amigos
PaWern
As a Shopper I want to
receive updates on incredible
deals that are located near
my home so that I can save
money on my purchases
Acceptance Criteria:
• Save Shopper’s location
• Ask Shopper if they want to receive
localized deals daily
• Send notification of incredible deals to
Shoppers located within 10 miles each
morning
• Allow Shopper to stop receiving
localized deals
21

22. The
Three
Amigos
PaWern
As a Shopper I want to
receive updates on incredible
deals that are located near
my home so that I can save
money on my purchases
• What
areas
of
the
applica2on
will
this
affect?
• What
is
the
overall
design?
(UI,
API,
UX,
etc…)
• What
are
the
details
test
cases
for
this
user
story
and
it’s
acceptance
criteria?
• What
about
nega2ve
test
condi2ons?
• What
about
boundary
condi2ons?
• How
might
we
put
exis2ng
func2onality
at
risk?
22

23. The
Three
Amigos
PaWern
• At
minimum
include
tester,
coder
&
business
rep
in
discussion
• Should
only
take
30
minutes
to
1
hour
for
user
stories
• Focus
on
clarifica2on
and
design
through
testable
inputs/
outputs
23

24. Acceptance
Test-­‐Driven
Development
24

25. Release
Management
If
releases
are
like
giving
birth,
then
you
must
be
doing
something
wrong.
-­‐
Robert
Benefield

26. Case
Study:
Enterprise
Agile
Adop2on
• 180+
person
Web
2.0
product
organiza2on
• Waterfall
SDLC
that
development
uses
to
deliver
in
6
month
release
cycles
• Want
to
use
Agile
methods
to
be
more
responsive
to
users
and
keep
up
with
other
Web
2.0
companies
• Transi2oned
to
Agile
methods
on
15
teams
in
3
months
• Changed
release
management
strategy,
added
XP
technical
prac2ces,
and
implemented
Scrum
product
development
framework
for
scaled
coordina2on
• Able
to
release
every
week
to
users
within
4
months
• Used
streamlined
deployment
environment
process
to
validate
product
changes
daily
using
Con2nuous
Integra2on
and
automated
promo2ons
26

27. The
Power
of
2
Scripts:
Deploy
&
Rollback
27

28. Tradi2onal
Source
Control
Management
Code
Complete
Version
1
Integrate
for
Branch
Version
2
Debt
Main
Branch
Death
March
{
Debt
accrues
quickly
within
stabiliza2on
periods
28

29. Flexible
Source
Control
Management
Version 1
Version 2
Main Branch
{
Not Easy! Must have proper infrastructure to do this.
29

30. Scaling
Con2nuous
Integra2on
End-­‐to-­‐End
&
Load/Stress
Integrated
Component
Valida2on
Component
Valida2on
30

31. Automated
Promo2on
to
Environments
31

32. Advanced
Quality
Asser2ons
Using
Automated
Tools
and
Dashboards

33. Con2nuous
Integra2on
33

34. Quality
Dashboard
-­‐
Sonar
34

35. Quality
Dashboard
-­‐
Sonar
35

36. Quality
Dashboard
-­‐
Sonar
36

37. Quality
Dashboard
-­‐
Sonar
37

38. Early
Warning
Signs
Early
Warnings:
• Broken
Builds
• Broken
Automated
Tests
• Broken
Custom
Thresholds
38

39. Early
Warning
on
Quality
Dashboard
Early
Warnings:
• Design
Debt
in
Duplica2on
(DRY)
• Technical
Debt
in
Code
Complexity
• Quality
Debt
in
Bug
DB
(Break/Fix)
• Other
Custom
Thresholds
39

40. The
No
Defect
Mindset
What
he
needs
is
some
way
to
pay
back.
Not
some
way
to
borrow
more.
-­‐-­‐
Will
Rogers
39

41. Ken
Schwaber
For
every
[dollar]
of
compe22ve
advantage
gained
by
cuYng
quality,
it
costs
$4
to
restore
it;
and
so)ware
is
an
organiza2onal
asset
and
decisions
to
cut
quality
must
be
made
by
execu2ve
management
and
reflected
in
the
financial
statements.
hWp://www.infoq.com/presenta2ons/agile-­‐quality-­‐canary-­‐coalmine

42. Case
Study:
Field
Support
Applica2on
• 2000+
users
access
applica2on
each
day
• Applica2on
supports
mul2ple
perspec2ves
and
workflows
from
Field
Support
Opera2ons
to
Customer
Service
• Team
of
5
people
delivering
features
on
exis2ng
Cold
Fusion
pla]orm
implementa2on
• Migra2ng
Architecture
to
Spring/Hibernate
in
slices
while
s2ll
delivering
valuable
features
• 36
2-­‐week
Sprints,
33
produc2on
releases,
and
only
1
defect
found
in
produc2on
• So,
what
was
the
defect
you
say?
Let
me
tell
you…
40

43. Can
We
Afford
a
No
Defect
Policy?
• This
team
worked
on
legacy
codebase
inherited
from
another
vendor
• Other
vendor
had
been
slowing
down
month
a)er
month
and
cost
of
development
was
increasing
• In
first
itera2on
this
team
was
able
to
deliver
more
than
other
vendor
was
able
to
in
previous
2
months
• A)er
24
itera2ons
this
team
was
10
2mes
faster
delivery
than1st
itera2on
• Acceptance
Test-­‐Driven
Development
and
Con2nuous
Integra2on
were
greatest
technical
factors
to
support
team
in
these
results
• Can
you
afford
not
to
have
a
No
Defect
policy?
41

44. Thank
you!
Ques2ons
and
Answers
[Time
permiYng]

45. Chris
Sterling
Director
of
Engineering
at
Rally
So)ware
in
Kirkland,
WA
office
Author
of
Book
Managing
So)ware
Debt:
Building
for
Inevitable
Change
Consults
on
so)ware
technology,
Agile
technical
prac2ces,
Scrum,
and
effec2ve
management
techniques
Innova2on
Games®
Trained
Facilitator
Email:
csterling@rallydev.com
Web:
hWp://www.rallydev.com
Cer2fied
Scrum
Trainer
Follow
me
on
TwiWer:
@csterwa
Blog:
hWp://www.geYngagile.com
Hashtag
for
presenta2on:
#swdebt
Open
Source
Developer