20140925 fistb keynote

Beyond Agile
Testing to Lean
Development
James O. Coplien
Gertrud & Cope
BEYOND AGILE TESTING TO LEAN DEVELOPMENT - Jim Coplien
• Testing is a business function that should be focused on risk
• Lean defines testing as waste — why do it?
• Automated tests should reflect requirements, but testers shouldn't test requirements
• Testers are the team's conscience
50 Minutes
1 20140925FiSTBKeynote.key - 27 Sep 2014

2-1 20140925FiSTBKeynote.key - 27 Sep 2014

http://www.ambysoft.com/essays/agileTesting.html

What
we
Wish
Agile
Testing
Were
(Big thanks to Michael Bolton and James Bach)

Agile Testing
means “Test-first”
… and almost always
at the unit level
Specification by example.
“Context-driven testing is sometimes considered a ‘flavor' of agile software development.” — http://
searchsoftwarequality.techtarget.com/definition/context-driven-testing

Agile Testing
at the unit level
Specificatio
n
by
E
xam
ple
Contextualtesting
Autom
ation
JUnit
TDD
"At the basis of the test automation
pyramid is unit testing. Unit testing
should be the foundation of a solid test
automation strategy and as such
represents the largest part of the
pyramid.” — Mike Cohn

Agile Testing
at the unit level
Specificatio
n
by
E
xam
ple
Contextualtesting
Autom
ation
JUnit
TDD
Processes and tools
over
Individuals
and Interactions

Part I: (Unit)
Testing is Muda

Unit Testing is One of the
Least Efficient Ways we
Know to Remove Defects
Removal Step Efficiency
Formal Inspections 93%
Static Analysis 55%
System Test 36%
Functional Test 35%
Component Test 32%
Unit Test 32%
Desk Check 27%
Prototyping 20%
Acceptance Tests 17%
Regression Tests 14%
(“Software Defect Origins and Removal Methods,” Draft 5, 28 December 2012, http://www.ifpug.org/Documents/Jones-
SoftwareDefectOriginsAndRemovalMethodsDraft5.pdf

Why doesn’t unit
testing work?
We can’t test things of business
value
Not knowing what to test, we test
many things that don’t need testing
Automated crap is still crap

Unit tests can’t test
things of business value
In an OO world you can’t tell, from the code,
whether a method will ever be executed — or in
what sequence
There is rarely any explicit link from unit tests
to anything of business value
What’s amazing is that we continue to test…
Paths that will never be exercised
Methods that are never used
Ungrounded method invocation sequences

Not knowing what to test,
we test things that don’t
need testing
We test scenarios that will never be
exercised in the real world
We test methods that will never be
invoked

Automated crap is
still crap
Automation for its sake is waste
Design requires intelligence: intelligence cannot be
automated
Autonomation instead of automation
You still need exploratory and experience-based testing
Get manual procedures working first and then
automate the mundane parts
Design for Test like they do in hardware

Automated testing is not
testing
http://qz.com/196200/toyota-is-becoming-more-efficient-by-replacing-robots-with-humans/ Max Niesen, April 7, 2014
James Bach: “Testing Requires Inference.” “Exploratory inference may sound like a strange idea. It means one idea leading
to another in ways you can’t predict in advance” [and, therefore, can’t script].

The software is trained
to pass the tests
"In software testing, it gets progressively
harder and harder to find defects. If we
run the same test or use the same
technique over and over, the effort
becomes less effective each time we do it,
because we find only the easy-to-find
bugs. The software is trained to pass the
test. …” — Ross Smith, Microsoft

That kind of test thinking,
leads to a bottom-up
procedural architecture
Citation: Siniaalto and Abrahamsson, Does Test-Driven Development Improve the Program Code? Alarming results from a
Comparative Case Study. Proceedings of Cee-Set 2007, 10 - 12 October, 2007, Poznan, Poland.

Part II: Why do we
unit test?
Because we can
It doesn’t require teamwork
We don’t have to talk with the
business
We get to interact with tools
instead of people
Our decisions are emotional,
not rational
Last point: Most tests are irrational: i.e., you can never presume software correctness from the tests. But we do feel better by
having checked, so we test anyhow.

Nokia Test Score 2:
Testing
No dedicated QA - 0

Unit tested - 1

Feature tested - 5

Feature tested as soon as completed - 7

Software passes acceptance testing - 8

Software is deployed - 10

Part III: You Will Have Bugs!
Scenarios we
test in the lab
!
!
! !
!
!
Scenarios
exhibiting
faults
!
Scenarios
invoked in
the field

Testing waste
Scenarios we
test in the lab
!
!
! !
!
!
Scenarios
exhibiting
faults
!
Scenarios
invoked in
the field
Type 2: Weinberg’s Law of Decomposition
Type 4: Keeping tests that never fail

Testing waste
Scenarios we
test in the lab
!
!
! !
!
!
Scenarios
exhibiting
faults
!
Scenarios
invoked in
the field
Type 1
Muda
Type 1: Defects

Testing waste
Scenarios we
test in the lab
!
!
! !
!
!
Scenarios
exhibiting
faults
!
Scenarios
invoked in
the field
Type 2
Muda
Type 1
Muda
Type 1: Defects
Type 2: Stuff
customers
don’t ask for

Testing waste
Scenarios we
test in the lab
!
!
! !
!
!
Scenarios
exhibiting
faults
!
Scenarios
invoked in
the field
Type 2
Muda
Type 1
Muda
Type 4
Muda
Type 1: Defects
Type 2: Stuff
customers
don’t ask for
Type 4:
Overprocessing

Testing waste
!
!
!
Scenarios
exhibiting
faults
!
Scenarios
invoked in
the field
Type 1
Muda
Type 4
Muda
Type 1: Defects
Type 2: Stuff
customers
don’t ask for
Type 4:
OverprocessingType
1 Muda
(field)

Testing waste
!
!
!
Scenarios
exhibiting
faults
!
Scenarios
invoked in
the field
Type 1
Muda
Type 4
Muda
Type 1: Defects
Type 2: Stuff
customers
don’t ask for
Type 4:
OverprocessingType
1 Muda
(field)
Removing tests
leaves field faults
undetected!

Reduction of Waste
Type 1: Better testing (e.g., going beyond
contextual testing to exploratory & experience-
based testing) (and better yet, better analysis
and design, where the real payoffs are)
Type 2: Just get rid of the tests
Type 4: Remove redundant and tautalogical
tests
Type 1 in the field: Ship your tests!

Don’t take the tests
out when you ship!
Remember, you will have bugs in the
field!
Deployment is your largest test driver
Ship your tests as part of the code
IN case of a failure, it saves you
from Type 9 muda (unsafe execution)
Be customer-focused

Defensive
Programming
… is largely the incorporation of
tests in the code
Long known for high-reliability
systems
The value of asserting the obvious
Design by Contract

Assertions make code
readable!
- (void) prepareForSegue: (UIStoryboardSegue*) segue
sender: (id) sender
{
assert(segue != nil);
if ([[segue identifier] isEqualToString:
@"SegueAvatarPickerToPickOneLoginPlayer"]) {
assert((UI_USER_INTERFACE_IDIOM() == UIUserInterfaceIdiomPhone));
// The field parentViewController_ (instance variable) is
// maybe null if we are in iPhone mode and doing a real
// segue, instead of a popover as we do on the iPad. So
// get the right object from the segue record.
KnowsyPlayerPickerCommonViewController *parentViewController =
[segue destinationViewController];
[parentViewController setKnowsyData: knowsyData_];
} else {
// iPad - no special processing - should be no popover
assert(parentViewController_ == nil);
}
}

Agile testing isn’t
about tools!
It’s about the test effort being customer-
focused
It’s about partners instead of customers
It’s about “stopping the line” when
discovering a defect — even in deployment
It’s about quality at every step
These are all Lean ideas

Part IV: Beyond agile
testing, Lean testing
When a test finds a bug, it’s about
fixing the process — not the bug
The long-term payoff is higher!
No bug-tracking systems: we fix
problems NOW.
The tester should be the team’s
conscience
Conscience… Curmudgeonly and like Emperor’s New Clothes — not in the sense of standing in the way. That’s business’s
job.

Other Lean-isms
Deming: Quality is inversely
proportional to the number of
testers
GM kept around its cars for 1 - 6
weeks for test and repair: Toyota
drove them off the end of the
assembly line onto the car carriers.
No unit - subsystem - system - integration - acceptance test leveling.

Conclusion
Be humble about what tests can accomplish
It’s not about testing, but value
Design tests to assess properties of business value
Go beyond unit testing to defensive
programming and design-by-contract
As agile testers, focus on process first and
product second

20140925 fistb keynote

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (19)

Destacado

Destacado (20)

Similar a 20140925 fistb keynote

Similar a 20140925 fistb keynote (20)

Más de James Coplien

Más de James Coplien (6)

Último

Último (20)

20140925 fistb keynote