1. Limbo Lower Now: An Agile
Approach to Defect
Management
ACCU 2011
Lisa Crispin
With Material from Janet Gregory
1
2. Introduction
My defect tracking background:
As programmer – no DTS
Tech support – DTS developed in-house
Traditional QA – vendor DTS, customized
Agile teams – variety of approaches
2
Copyright 2010: Lisa Crispin
3. Your Defect Tracking?
How about you?
How many use a DTS?
How many don’t use one?
How low is your “defect limbo” bar?
3
Copyright 2010: Lisa Crispin
4. Takeaways
Defects and technical debt
When to use a Defect Tracking System
Alternatives
Defect metrics
Setting the bar lower
4
Copyright 2010: Lisa Crispin
5. Traditional View
Bugs must be documented
Especially those found in production
Provide information for root cause analysis
Knowledge base – did similar bug occur
earlier?
DTS provides useful metrics
5
Copyright 2010: Lisa Crispin
6. Lean Development / Agile View
Defect queues are queues of rework –
waste (Poppendieck)
Find, fix test-first, and forget
Better yet, prevent
zero bug tolerance
Bugs => Technical Debt
Small experiments
6
Copyright 2010: Lisa Crispin
7. Choices
An agile approach:
Understand the problem
Do what works for your team
Focus on goals: bug prevention
Start simple, add as needed
Explore alternatives
7
Copyright 2010: Lisa Crispin
8. Why Log Bugs – or not
What are the advantages of using a DTS?
What are the downsides to using a DTS?
8
Copyright 2010: Lisa Crispin
9. Some Advantages of Using DTS
Knowledge base
Place to keep supplemental docs
Prioritizing
Traceability
Distributed/large teams
Customer ease, visibility, convenience
Metrics
Redundant with test-first, frequent releases
9
Copyright 2010: Lisa Crispin
10. Some Disadvantages of using DTS
Overhead
Duplicates test
Impedes communication
Waste
Inaccurate or inadequate reports
Caught during development
Won’t get fixed
Doesn't work towards goal
zero defects, bug prevention
10
Copyright 2010: Lisa Crispin
11. Defect Metrics
Traditionally, tell us state of quality, release
readiness
Using tests to drive incremental, iterative
development tells us this
Trends more important than numbers
Who cares how many bugs found in
development?
May be useful for team goals
Careful how you use metrics
11
Copyright 2010: Lisa Crispin
12. Some Other Perspectives
What we can learn from bugs
Alternatives to logging bugs
When tracking defects makes sense
12
Copyright 2010: Lisa Crispin
13. Bugs as the “Hidden Backlog”
Acceptance tests = desired behavior
Defect reports = misbehavior (Antony
Marcano)
Ignored defects = missed features
Misbehavior not addressed
Defects prioritized over new features
Misbehavior over desired features
Information can get lost in DTS
13
Copyright 2010: Lisa Crispin
14. Choosing to Not Log a Bug
Unit-level bugs
Write a test, not a bug
Higher-level regression failures
Test usually enough
Bugs found during development
Fix immediately
Log on task card
14
Copyright 2010: Lisa Crispin
15. Choosing to Log a Bug
Found in later iteration
Log if not fixed immediately
Legacy bugs
Write new story/feature
Don't log if it won't be fixed
Found in production
Fix immediately or
Prioritize, estimate, plan
Non-bug production issues
“production support requests”
15
Copyright 2010: Lisa Crispin
16. Other Approaches To Log or Not?
How do you decide?
16
Copyright 2010: Lisa Crispin
17. Alternative Ways to Track Bugs
Cards
DTS
Tests
17
Copyright 2010: Lisa Crispin
18. Logging Bugs on Cards
Story, task or bug cards – physical or online
Tangible
Visible – color coding
Attach screen prints
Use when:
Disciplined team
Bugs need high visibility
18
Copyright 2010: Lisa Crispin
19. Logging Bugs in DTS
Use a DTS when
Team is distributed
Audit requirements
Release notes
Not all bugs will be fixed immediately
Legacy system with lots of defects
19
Copyright 2010: Lisa Crispin
20. Logging Bugs with Tests
Use the test written to reproduce the bug and
verify the fix when:
Everything learned from fixing bug is
captured in test
Test can be automated
Team is disciplined and writes tests for
every bug found
20
Copyright 2010: Lisa Crispin
21. When to Fix Bugs
Fix now
cheaper
prevents technical debt
Estimate, prioritize and fix later
complex, high effort
legacy code, may destabilize
customer decision
Never fix
about to be rewritten
low risk
don't pretend you'll fix if you won't!
21
Copyright 2010: Lisa Crispin
22. Lowering the Bar
Exploring alternatives
Preventing bugs in the first place
22
Copyright 2010: Lisa Crispin
23. Experiment !
Start a new project with no DTS, and see
what you need.
Set rules - “no more than 10 bugs at once”
Use color-coded cards and stickers
Fix all bugs
Treat bugs as stories
23
Copyright 2010: Lisa Crispin
24. Make It a Team Problem
Understand your needs
Choose solution by team consensus
Cards and stickers (real or online)
Story board
Online DTS
Combo: DTS and cards
Ask other teams
24
Copyright 2010: Lisa Crispin
25. Focus On Preventing Defects
Teams have achieved zero-defects goal
Whole team collaborates on ways to
improve quality
Use retrospectives
Try small experiments
25
Copyright 2010: Lisa Crispin
26. Let’s Try an Experiment: Part 1
Pair up with the person next to you
One of you take the “tester” role, the other
is “programmer”. Stand back to back.
Testers: Take the mini-schedule out of your
badge & open to p. 10
Programmers: take a piece of paper and
pen. You cannot ask questions or turn
around.
Testers: Get the programmer to draw what
you see on p. 10 – 11 only by describing
the shapes & where they are. Do not turn
around!
26
Copyright 2010: Lisa Crispin
27. Let’s Try an Experiment: Part 2
Programmer: Get a new piece of paper.
This time, you are allowed to ask
questions.
Tester: You may now watch what the
programmer draws
Point out “defects” immediately, and
answer questions.
(Don’t show the actual pages, we’re
trying to simulate real-life coding here!)
27
Copyright 2010: Lisa Crispin
28. Bug Prevention!
Make testing integrated part of
development
Practice TDD and Specification by Example
Gain domain expertise
Exploratory testing by testing professionals
Consider ripple effects of code changes
Look at problem areas, experiment with
solutions
28
Copyright 2010: Lisa Crispin
29. Action!
Think of one thing your team can do to
prevent defects, starting next week.
Write it on an index card along with your
email address.
Exchange cards with the person next to
you.
Email each other every week for 4 weeks
with encouragement, questions, support.
Please let me know your results!
lisa@lisacrispin.com
29
Copyright 2010: Lisa Crispin
30. How Low Can You Go?
Experiment to find defect management
approach that fits best
Zero defects is achievable
But go for baby steps
Focus on bug prevention
Do what works for your team
Test/code together
Use retrospectives
30
Copyright 2010: Lisa Crispin
31. Agile Testing
Agile Testing: A Practical Guide for Testers and Agile
Teams
By Lisa Crispin and Janet Gregory
www.agiletester.ca
Book signing after this session
at the bookstore
Copyright 2010: Lisa Crispin
31
32. All Proceeds to Charity!
Beautiful Testing: Leading Professionals Reveal How
They Improve Software
Edited by Tim Riley, Adam Goucher
Includes chapter by yours truly
Copyright 2010: Lisa Crispin
32
33. Specification by Example
How successful teams deliver the right
software
Gojko Adzic
Case studies from > 50 teams
33
Copyright 2010: Lisa Crispin
Copyright 2008 Janet Gregory, DragonFire