Rik Marselis and Geoff Thompson presented about why people in an IT organization would want to use methods for improving their test process. Rik was the project leader for the creation of TPI NEXT. Geoff was one of the founders of TMMi. Both have contributed to ISTQB. Both models can be used for test process improvement. This presentation gives an overview of the reasons why one would want to improve. After that Geoff and Rik both elaborated on the details of the methods and then did a comparison. This presentation was given at a conference for test experts of the Siemens group.

1. Restricted © Siemens AG 2014. All rights reserved
Why Would You Want
to Improve Your Process?
13th Exchange of Experience: Testing | Nuremberg, March 13, 2014
Rik Marselis (Sogeti)
Geoff Thompson (Experimentus)

2. Restricted © Siemens AG 2014. All rights reservedPage 2 March 13, 2014 Rik Marselis, Geoff Thompson
Introducing Rik & Geoff
NL
UKISTQB
Passion for quality and
for “doing IT right”
Enjoy sharing knowledge:
presenting and writing
Projectleader
TPI NEXT
3 daughters
2 daughters
1962 1961
Co Founder TMMi
Director of
ExperimentusFellow of
Sogeti-Labs
Happily married
Founder ISTQBVice-chairman
TestNet

3. Restricted © Siemens AG 2014. All rights reservedPage 3 March 13, 2014 Rik Marselis, Geoff Thompson
What is “success”?
Wikipedia: Achievement of a Goal;
the opposite of Failure
When is your business successful?
Project ready in time?
IT Quality meets standards?
Business value is created?
Clients are happy!!

4. Restricted © Siemens AG 2014. All rights reservedPage 4 March 13, 2014 Rik Marselis, Geoff Thompson
Why do we need to improve?
Because software fails
and we need to be as
efficient as possible in
trying to stop these!
London Stock Exchange
LA Airport flights grounded
AT&T take out ⅓ of US ‘phones
London Underground free travel with Oystercard
British Passport failure
Intel Pentium chip maths division wrong
Software failure at Stanstead closed check in
Therac 25 radiation overdosing
Airbus A380 incompatible software
Patriot Missile System – Dhahran
Mars Climate Orbiter & Polar Lander

5. Restricted © Siemens AG 2014. All rights reservedPage 5 March 13, 2014 Rik Marselis, Geoff Thompson
We want to do the right thing
in the right way

6. Restricted © Siemens AG 2014. All rights reservedPage 6 March 13, 2014 Rik Marselis, Geoff Thompson
To do the right thing in the
right way can mean...
• Saving money both in the short and long term
• Better risk management
• Accelerated time to market
• Reduction in delays to the software development life cycle
• Better career opportunities
• Improved working life of employees

7. Restricted © Siemens AG 2014. All rights reservedPage 7 March 13, 2014 Rik Marselis, Geoff Thompson
Business drivers
Source - IT priorities for battling the economic slowdown: UK CEO Survey | 2008 –
BearingPoint/HP

8. Restricted © Siemens AG 2014. All rights reservedPage 8 March 13, 2014 Rik Marselis, Geoff Thompson
To avoid going the wrong way!

9. Restricted © Siemens AG 2014. All rights reservedPage 9 March 13, 2014 Rik Marselis, Geoff Thompson
Opening the gate – getting rid
of barriers to quality work

10. Restricted © Siemens AG 2014. All rights reservedPage 10 March 13, 2014 Rik Marselis, Geoff Thompson
Is your process this sort of
Ferrari?

11. Restricted © Siemens AG 2014. All rights reservedPage 11 March 13, 2014 Rik Marselis, Geoff Thompson
When you would like it to be
this sort?

12. Restricted © Siemens AG 2014. All rights reservedPage 12 March 13, 2014 Rik Marselis, Geoff Thompson
V-model, outdated?
• Start/stop model
• Create in one go (no increments)
• Doesn’t stimulate collaboration

13. Restricted © Siemens AG 2014. All rights reservedPage 13 March 13, 2014 Rik Marselis, Geoff Thompson
Application Lifecycle model
Most people accept “Testing” as a fact of life
But often they are not happy about it
All activities
are relevant,
they may be
done
sequential or
in parallel
Applies to both
Waterfall
and
Agile

14. Restricted © Siemens AG 2014. All rights reservedPage 14 March 13, 2014 Rik Marselis, Geoff Thompson
So we performed an assessment to assess the test maturity
Conclusion: There’s nothing really wrong with testing
The fixing and rework is the real problem!
Example; Client said:
“Testing costs too much time & money”

15. Restricted © Siemens AG 2014. All rights reservedPage 15 March 13, 2014 Rik Marselis, Geoff Thompson
We introduced:
“the fixing-phase”
Now something interesting happened;
the stakeholder asked:
“Why do we need a fixing phase”?
“Can’t they build it right first time”?
Fixing phase
Source: Jerry Weinberg

16. Restricted © Siemens AG 2014. All rights reservedPage 16 March 13, 2014 Rik Marselis, Geoff Thompson
Why do we need this fixing?
Defect injection
58% of defects originate
from early lifecycle activities !!

17. Restricted © Siemens AG 2014. All rights reservedPage 17 March 13, 2014 Rik Marselis, Geoff Thompson
What often happens
Requirements Design Development Testing Acceptance Implementation Maintenance
Requirements Design Development Testing Acceptance Implementation Maintenance
Defect inserted in activity
Defect detected and fixed in activity

18. Restricted © Siemens AG 2014. All rights reservedPage 18 March 13, 2014 Rik Marselis, Geoff Thompson
The right quality,
not too less, not too much
Three key principles
Frontload the process with quality
measures; methods, tools, training,
reviews, …
People are fallible  early reviews
And remember:
Quality can’t be ‘tested in’ at the end

19. Restricted © Siemens AG 2014. All rights reservedPage 19 March 13, 2014 Rik Marselis, Geoff Thompson
No faults forward
to the next activity
Requirements Design Development Testing Acceptance Implementation Maintenance
Requirements Design Development Testing Acceptance Implementation Maintenance
Defect inserted in activity
Defect detected and fixed in activity

20. Restricted © Siemens AG 2014. All rights reservedPage 20 March 13, 2014 Rik Marselis, Geoff Thompson
When and What to improve
When:
As soon as you can
The next new project
Something that is big enough (and
important enough) to make a difference
Does timing even matter?
Everything
A bit at a timeWhat someone else tells me needs to change!
Use a model to help understand what's needed
On the next business critical project
Just tinker
Something that gives the biggest bang for the buck
Once you have a business case
Only what you have a business case for
What:

21. Restricted © Siemens AG 2014. All rights reservedPage 21 March 13, 2014 Rik Marselis, Geoff Thompson
To model or not to model that
is the question?
Non-model based approach
involves process
improvements by adapting
what already exists and
augmenting with process
components picked up at
conferences/training.
These processes typically
tend to be home grown
untried and untested.
Model based process
improvement incorporates
process improvement with
international standards and
frameworks such as the SEI’s
CMMI frameworks, ISO
frameworks, ITIL, TPI, TMMi,
PCMM and various others.
Using such models allows
organisations to compare their
practices with internationally
recognized industry good
practices.

22. Restricted © Siemens AG 2014. All rights reservedPage 22 March 13, 2014 Rik Marselis, Geoff Thompson
Do you feel lucky...

23. Restricted © Siemens AG 2014. All rights reservedPage 23 March 13, 2014 Rik Marselis, Geoff Thompson
How!
Measure
Prioritise
& Plan
Define /
Re-define
Operate
Validate
Evolve
Initiate
• Action Planning
• Target Setting
• New process
definition/refinement
• Training
• Pilot
• Full deployment
• Monitoring of results
against targets
• Evolution of programme
approach & targets
• Assess Process/
monitor against plan
• Obtain agreement
to progress

24. Restricted © Siemens AG 2014. All rights reservedPage 24 March 13, 2014 Rik Marselis, Geoff Thompson
But this success is not just
process….
Measure/Prioritise
and Plan/ Define and
Redefine Operate/Validate Evolve
Enthusiasm
Maximum
Hype
Actual Use
Disillusionment
Practical Benefit
RIP

25. Restricted © Siemens AG 2014. All rights reservedPage 25 March 13, 2014 Rik Marselis, Geoff Thompson
It takes time before the actual
improvements are realized
When ambitions for change
are set high, It takes too
much time before the actual
savings are realized

26. Restricted © Siemens AG 2014. All rights reservedPage 26 March 13, 2014 Rik Marselis, Geoff Thompson
Improve gradually
parallel and step-by-step improvement,
with small but measurable effects

27. Restricted © Siemens AG 2014. All rights reservedPage 27 March 13, 2014 Rik Marselis, Geoff Thompson
Improve where it helps most
Don’t elevate the peaks; start improving by filling the valleys

28. Restricted © Siemens AG 2014. All rights reservedPage 28 March 13, 2014 Rik Marselis, Geoff Thompson
The basic assumption for
process improvement
+ =
People Process Result
+
+
+
=
=
=




29. Restricted © Siemens AG 2014. All rights reservedPage 29 March 13, 2014 Rik Marselis, Geoff Thompson
So why would you improve?
Measure increased success!!

30. Restricted © Siemens AG 2014. All rights reservedPage 30 March 13, 2014 Rik Marselis, Geoff Thompson
Thank you !!

31. Restricted © Siemens AG 2014. All rights reserved
Comparing the models
Q&A session

32. Restricted © Siemens AG 2014. All rights reservedPage 32 March 13, 2014 Rik Marselis, Geoff Thompson
The models compared
http://www.nl.capgemini.com/bronnen/models-to-improve-your-test-process www.istqb.org

33. Restricted © Siemens AG 2014. All rights reservedPage 33 March 13, 2014 Rik Marselis, Geoff Thompson
Comparison
TMMi TPI

34. Restricted © Siemens AG 2014. All rights reservedPage 34 March 13, 2014 Rik Marselis, Geoff Thompson
Comparisons
TMMi
Staged model
Advantage of staged:
simplicity
TPI
Continuous and staged
model
Advantage of continuous:
specific improvement
focus

35. Restricted © Siemens AG 2014. All rights reservedPage 35 March 13, 2014 Rik Marselis, Geoff Thompson
Comparison - differences
TMMi
• Managers Model
• Developed by over 500 resources
worldwide
• Uses ISTQB terminology – which
includes TMap terminology
• Aligned to CMMI
• Accreditation of methods and assessors
• Metrics and full lifecycle focussed
• Defect Prevention and Quality control
focus
• Potential improvement recomendations
• Free download of model
(www.tmmifoundation.org)
TPI NEXT
• Engineers Model
• One organization in different countries
• Uses TMap terminology
• Independent of (but usable with) SPI-
models
• Business Driven approach
• Improvement suggestions & enablers
• Free Excel-tool (www.tpinext.com)

36. Restricted © Siemens AG 2014. All rights reservedPage 36 March 13, 2014 Rik Marselis, Geoff Thompson
Similarities
• Books
• Certification
• TMMi professional
• TPI NEXT certified
• Certification of organizations
• Referenced in ISTQB Advanced
& Expert level
• Fit well with CMMI
• Encourage structured
improvement of test processes

37. Restricted © Siemens AG 2014. All rights reservedPage 37 March 13, 2014 Rik Marselis, Geoff Thompson
Ask Geoff and Rik
(now is your chance!)
Q & A