Zum vierten Mal in Folge hat SwissQ zum Jahresende bei der Schweizer IT Community die aktuellen Trends und Benchmarks erfragt. Daraus entstanden ein repräsentativer Überblick zum Stand des Software-Testings in der Schweiz im Jahr 2012 und ein Ausblick auf die wichtigsten Trends der kommenden Zeit.

1. SwissQ Testing Trends &
Benchmarks Switzerland 2012
Software Testing Trends Switzerland 2012. Where are we now – where are we going to?

2. TABLE OF CONTENTS SwissQ Testing Trends & Benchmarks 2012 2
3 EDITORIAL
4 TRENDWAVE 2012
6 KEY MESSAGES
7 PROJECTS
8 MATURITY
9 ORGANISATION
11 PROCEDURES
12 TESTING TOOLS
13 FRAME OF SURVEY
14 REQUIREMENTS
15 AGILE

3. EDITORIAL SwissQ Testing Trends & Benchmarks 2012 3
For the fourth time in a row, SwissQ questioned the Swiss IT community about current trends and benchmarks. Not only did we receive
overwhelming feedback from over 300 participants, we also personally interviewed many IT decision makers from various companies,
segments, and regions about the current testing trends. The outcome is a representative overview of the most important trends.
For the first time this year we are not only presenting a trends and benchmarking analysis of Software Testing, but are also taking a look
at Requirements Engineering and Agile. The corresponding reports will be published separately to highlight their importance.
The group of participants mainly included test managers and line managers, but The same as last year, SwissQ offers companies the opportunity to present the
also test engineers and project leaders working in the area of IT, finance, insu- SwissQ Testing Trends & Benchmarks to their employees. The target audience
rance, manufacturing, or in public companies. For the first time this year we put consists of members of the testing community, software engineers, project leaders,
a special focus on Agile and Requirements Engineering and have thus enhanced requirements engineers, and the management. The presentation allows you to
the group of participants accordingly. A quarter of the participants of this survey, take an outside look at the current changes, as well as to boost the motivation for
therefore, work in Requirements Engineering. In the end, these 300+ question- and significance of testing, and profit from ideas for further developments.
naires served as the basis for the analysis and interpretation of the current state
in the year 2012 and the future development. Having said this, we hope that the present Trends & Benchmarks will inspire you
to actively pursue new challenges and take take the actions that best suit your
The benchmarks depicted in a multitude of informative charts and diagrams form business.
the backbone of this report and allow you to position your company in comparison
to others.
As before, we use the SwissQ Trend Wave® to show the evolution of the main
topics. It shows in four phases how single trends will most likely develop over time
and in turn allows you to appraise the influence of the trends on your business.

4. TRENDWAVE 2012 SwissQ Testing Trends & Benchmarks 2012 4
INTRODUCTION GROWTH MATURITY DECLINE
PRIORITY
Automated Test Process Improvement
Regression Tests
ALM Tools
SCRUM
Agile Testing Regression Tests
Load and Performance Tests
Central
Test Organisation
Explorative Testing
Risk Based Testing
ISTQB FL & AL Level
TDD
Security & Penetration Testing
Outsourcing
ISTQB Expert Level
Session Based Testing
Mobile Testing
Cloud Testing ATDD
TIME
INTRODUCTION – This topic has been GROWTH – This topic is more and MATURITY – Most companies are DECLINE – The topic has already
identified and some companies are more accepted and many companies working on the implementation been implemented by most of the
deploying initial implementations. are considering it. The first tools are or have already completed it. companies, with the exception of
However, it cannot be foreseen being developed and consultancy The knowledge of this topic is individual latecomers. Often, there
whether this trend will positively firms offer services for the same. often widespread, resulting in is no more added value in acquiring
advance and whether testing will Often risks are associated due to sub-topics being raised. further knowledge in these areas,
be considerably influenced. limited implementation experience. since it will become obsolete shortly.

5. TRENDWAVE 2012 SwissQ Testing Trends & Benchmarks 2012 5
The aim of the trendwave is to show what topics will influence the testing discipline as well as the people and the organisations
dealing with it. It thus helps readers to specifically prepare for, and actively participate in these changes.
The technologies that changed the consumer market in the last couple of years – The current focus remains on two areas that have continued to gain significance
smart phones and tablets, social media, cloud, and highly topical, voice control over the past few years, one of them being Agile Testing as most of the partici-
– are increasingly being used in the business world as well. The new information pants of this survey are at least partly working with agile methods. The second
and communication channels accompanying this development lead to higher area is Requirements Engineering which is also directly involved in one of the
consumer expectations as the following example illustrates: biggest challenges in testing: the generally poor quality of requirements.
Be it coincidence or fate, the parallel development of these two subjects can
only be of advantage to us.
During the après-ski break, we started a heated argument about the need of
a separate ski theft insurance. I got my smart phone and opened the app of my We can therefore assume that Mobile Testing and especially Cloud Testing for many
insurance company to check my current coverage. I was startled to realize that businesses and software testers will only appear on the radar once the biggest
I wasn‘t insured against ski theft! I immediately asked Siri to find a provider and obstacles in the transition to Agile Testing and the improvement of the quality of
took out a policy online. I was lucky I did that because I just saw a guy in a yellow requirements are overcome. Furthermore, test automation – still not on a
ski suit taking off with my skis! satisfying level – also remains in the focus.
Still, it is conceivable that the “mobilization“ of IT will increase rapidly in the
Such seemingly utopian scenarios are on the verge of their break through today. near future, so the question to pose is: shouldn‘t we jump on the bandwagon
They are not only challenging business analysts and software engineers, but tes- right now before catching up will be too cost-intensive and time-consuming?
ting as well will become a more complex and costly endeavour.
Many companies have recognized these trends and are already including them in
their current projects and products, others are gaining experience and building up
more knowledge. Most businesses though still think that it is too early to deal with
the subject but should be careful not to miss the opportunity to catch up on it.

6. KEY MESSAGES SwissQ Testing Trends & Benchmarks 2012 6
1 More than half of the companies
apply agile development practices.
Thereby 84.5 % of the respondents
use Scrum as their preferred agile
method.
2 New technology and process driven
processes like mobile testing,
agile testing or session based test
management are (still) getting
little attention.
3 The focus of education and
training has shifted towards
requirements engineering and
agile, closely followed by the ISTQB
advanced level certifications.
4 While 40% of the respondents rate
the maturity level of the testing
process as good or excellent, 75 %
of the respondents rate the maturity
level of the requirements engineering
5 Not surprisingly, poor
requirements are mentioned as
the biggest challenges in software
testing, Followed by late
involvement of the testers.
6 The interdisciplinary collabora-
tion between business analysts,
software engineers, and testers
gained further importance. But the
significant mind shift hasn’t
as medium or less. happened yet.
7 The most investments are made
in test automation or the creation
and extension of regression tests.
In contrast there are just marginal
investments in hiring of new
8 Even though 80% of all participants
automate their functional tests,
almost half of them automated less
than 10 % of all functional tests.
This is a step forward though, as
9 The trend towards integrated tool
solutions has increased, parti-
cularly thanks to HP ALM and
partly MS TFS. Tools covering only
one discipline find themselves
testing employees. last year almost 2/3 of the in a difficult position.
participants had automated less
than 10 %.

7. PROJECTS SwissQ Testing Trends & Benchmarks 2012 7
Project Type
70 % of all projects are new developments or updates of existing software.
>50%
of the respondents describe the starting situation of their
12 % projects as satisfying or insufficient related to
8% New development
Estimation
39 % Update of existing software
Planning
10 % Migration
Definition of requirements
Implementation of standard Realistic expectations
software
Operation, support, maintainance,
31 % re-design, ...
Project Outcome
Just over a third of all projects are finished with the expected functionality,
within the expected timeframe and within budget.
Project Size (in Swiss Francs)
40 %
51 %
35.1 %
30 %
40 % 39.2 %
39.2 % 25.1 %
20 %
18.1 % 17.5 %
20 % 10 %
4.1 %
10.8 % 0 %
Project Proj. finished Proj. finished Project Project
finished in time, with budget with major extended / stopped
0 % budget and and / or time functional rescheduled
up to 1 Mio up to 20 Mio more than 20 Mio functionality overruns changes

8. MATURITY SwissQ Testing Trends & Benchmarks 2012 8
Maturity of Test Process Reputation of Testing
In average the respondents rate the maturity of their companies testing
process as medium or good. This rating has remained constant in the last
years. It’s a contrast to the maturity in requirements engineering (RE)
Testing is strategic for 18 %
which is rated substantially worse. company success
50 % Testing is important for
49 %
reliable software
40 % Testing is a necessary evil 19 %
Testing has low priority 13 %
30 %
Testing
RE Testing is unnecessary, we can 2 %
save the costs for testing
20 %
0 % 10 % 20 % 30 % 40 % 50 %
10 %
0 %
The Biggest Challenges
ea
k
ea
k
um od nt
w W i Go lle
ry ed ce Poor Availability
Ve
M Ex Requirements of test
environment
Testers are
involved
63 % 30 %
Auto-
too late
mation
38 %
40% 75%
33 %
Not enough
of the respondents rate the of the respondents rate budget /
resources
maturity of the testing process the maturity of the
as good or excellent. requirements engineering 30 %
as medium or less.

9. ORGANISATION SwissQ Testing Trends & Benchmarks 2012 9
Who Performs Testing Activities Education
The focus of the employees training is moving from ISTQB certification
Test Case Design – which most of them already have – to the topics of requirements
engineering and agile.
I already have it It is planned In the not too distant future
IT Tester ISTQB Foundation Level
36 %
Req. Engineer / ISTQB Advanced Level
Business Test Manager
Developer
Analyst IT Tester End User ISTQB Advanced Level
21 % 16 % Test Analyst
19 %
Developer ISTQB Advanced Level
7.4 %
Technical Test Analyst
IREB® CPRE Foundation Level
Project Management
(IPMA, PMI, ...)
Test Execution
Certified Scrum Master
Agile Testing
Certified Product Owner
Certified IT Process and
Quality Manager
IT Tester Agile Requirements Engineering
34 % End User
26% ISTQB Expert Level
Developer
Req. IREB CPRE Advanced Level
IT Tester
Engineer/BA Elicitation & Consolidation
17%
14% Developer IIBA CBAP (Certified Business
9 %
Analysis Professional)
IREB CPRE Advanced Level
Requirements Modeling
0 % 20 % 40 % 60 % 80 % 100 %

10. ORGANISATION SwissQ Testing Trends & Benchmarks 2012 10
Use of External Resources Testing Effort
The use of external resources in testing slightly increased compared The variance of the testing effort is still wide. By average the testing effort in
to 2011. The external resources are mostly used in-house. proportion to the total effort is about 15 % to 20%. In proportion to the deve-
Outsourced testing stagnates compared to Managed Service lopment effort this value is more than 50 % (not represented in the graph).
(e.g. automated regression tests) which have expanded.
30 %
5.9 % 30 %
10,4 % No external resources
33.8 % Acquisition of external resources 20 %
21 %
(Insourcing)
19 %
Acquisition as a service 17 %
(Managed Service)
50 % Outsourcing to external resources 10 %
8 %
4 % 1 %
0 %
< 5 % 5 – 10 % 10 – 15 % 15 – 20 % 20 – 30 % 30 – 50 % above
Testing effort in proportion to total effort
Investments
Investments are Investments remain Investments are
increasing constant decreasing
Test automation
Test environments Outsourcing testing activities
Test management tools Hiring new testing employees
Creation and extension of
Test data management
regression tests
Training and education
Structured testing process / methods

11. PROCEDURES SwissQ Testing Trends Benchmarks 2012 11
Software Development Process Implementation of Agile Methods
More than half of the companies use agile processes. Many of them
are using a combination of different development processes e.g. agile
Drivers of Agile Methods
in combination with waterfall.
Agile processes are mostly promoted by the directly involved people.
60 %
39.2 %
52.1 %
Agile 40 %
39.4 % 38.0 %
36.6 %
51 %
Waterfall 10 %
19.8 %
40 %
0 %
Developer Departement Head Project CEO
Iterative mgr. / Team development Manager
Hermes lead
22 % RUP 12 %
16 %
Reasons for Agile Methods
84.5% Better handling
Accelerate Time-to-Market
of the respondents use Scrum as their favorite
agile method. of changing
priorities Productivity
SwissQ Agile Trends Benchmarks 2012 Collaboration between
There are many additional charts and details in the IT and business
SwissQ Agile Trends Benchmarks Report 2012.

12. TESTING TOOLS SwissQ Testing Trends Benchmarks 2012 12
Test Management Tools Test Automation
More than 60 % of the respondents work with HP QC (since version 11 the
name is changed to ALM). Many of them are working with more than one Tools used
80%
tool, mostly in combination with MS Excel or Word. MS Test Manager has a
The market leader by far is HP Quick Test Professional, followed
growing share, due to its tight integration with MS Team Foundation Server.
by proprietary tools. All in all the market is highly fragmented.
Almost 30 tools were mentioned.
of the respondents automate
their functional tests.
HP QC / ALM 62 %
Selenium
MS Office 44 %
Tosca
Canoo Webtest
Proprietary Tools
Ranorex Lisa 48%
of the respondents have
Open Source 20 %
Quick Test Professional automated less than 10 %
of their functional tests.
Rational Functional Tester In 2010, the value was 60 %.
QF Test
Others
Proprietary developement 8 %
MS TFS 2010 Test Manager 5 %
Cost Savings by Test Automation
Most of the respondents expect low savings by test automation.
Inflectra Spira 2 % In one third of the cases giving a statement to this question was not possible.
IBM Rational TestManager 2 %
33.3 %
No tool 1 %
22.6 % 23.7 %
0 % 20 % 40 % 60 %
10.2 % 2.8 %
7.3 %
Costs up to 10 % up to 20 % up to 50 % up to 80 % No
increased statement
possible

13. FRAME OF SURVEY SwissQ Testing Trends Benchmarks 2012 13
Industrial Sector Responsibilities
More than 60 % of the respondents work either in the IT or in the More than 50% of the respondents describe their job with more than
financial sector. Compared to the last years their proportion has decreased, one role. Especially test managers don’t work 100% as test managers,
demonstrating that the subject has arrived in other industries too. but also take responsibility for other roles.
IT 36.1 % 30 %
Finance, Insurance 28.4 %
Manufacturing 7.4 %
Public and semi-public companies 7.4 % 20 %
Traffic and Transportation 5.6 %
Telecommunication 4.0 %
MedTech 3.7 % 10 %
Others 7.4 %
0 % 10 % 20 % 30 % 40 %
0 %
IT Employees ge
r
na ana er
r A
ge / B nee age
r r
Te
st
er
na
ge
r
ne
er
a e n gi a n a ngi
A bit more than half of the respondents work in companies with more t M n M gin st E ct M s M re E
es io n e e nt a
than 500 IT employees. T is ts E T oj e
iv Pr m ftw
/ D men u ire So
t q
en uire Re
rtm eq
pa R
2001– ... 33.0 % De
501 – 2000 17.6 %
60% 33%
251 – 500 13.6 %
51 – 250 15.4 %
11 – 50 14.2 %
of the respondents mainly of the respondents are
1 – 10 6.2 % work in projects. line managers.
0 % 5 % 10 % 15 % 20 % 25 % 30 % 35 %

14. SWISSQ REQUIREMENTS ENGINEERING SwissQ Testing Trends Benchmarks 2012 14
TRENDS BENCHMARKS REPORT 2012
Do you want to know more? You can download the detailed report with further analyses about requirements engineering from www.SwissQ.it
SwissQ Requirements Engineering Trends Benchmarks 2012
There are many additional charts and details in the SwissQ Requirements
Engineering Trends Benchmarks Report 2012.
Future Investments
Investments Investments remain Investments
increase constant decrease
Education and Training
for Employees
Better Cooperation
33 % 54 % 13 %
80% 36%
of the projects check their do not check requirements
of Business and IT
33 % 53 % 14 %
requirements for operational for their need.
Standardisation of the accuracy, implementability,
internal RE-Processes 25 % 61 % 14 % and completeness.
Elaboration / Definition
of the RE-Role
24 % 60 % 16 %
Development of
Templates and Guidelines 22 % 61 % 17 %
2/3
Hiring new
RE-Employees 22 % 55 % 23 %
Establishing specific
RE Tools 21 % 64 % 15 %
of the respondents
Establishing internal allocate less than 1 day
RE-Divisions/-Departments 17 % 63 % 20 %
for stakeholder analysis.
Outsourcing
RE-Activities 11 % 48 % 41 %
0 % 20 % 40 % 60 % 80 % 100 %

15. SWISSQ AGILE TRENDS BENCHMARKS REPORT 2012 SwissQ Testing Trends Benchmarks 2012 15
Do you want to know more? You can download the detailed report with further analyses about agile trends from www.SwissQ.it
SwissQ Agile Trends Benchmarks 2012
There are many additional charts and details in the SwissQ Agile Trends
Benchmarks Report 2012.
Main Reasons for the Failure of Agile Projects
Only
40%
Lacking Experience with
agile Methods
52 %
Corporate Philosophy is not connectable
with agile Values
45 %
are content with the
implementation of their External Pressure to follow a
agile approach. traditional Approach
41 %
Lacking Support of the Management 38 %
Lacking / Insufficient Training / Coaching 36 %
1/3 Lacking Interconnections between
organisational units
35 %
of the respondents have Lacking Team Motivation 22 %
2 or more years experience
with agile methods.
Others 12 %
0 % 10 % 20 % 30 % 40 % 50 % 60 %

16. ABOUT US
SwissQ supports its clients in the development and implementation of IT-solutions and
assures that the end users get the functionality they really need. This is achieved by
unambiguously determining requirements and risk-based testing the implementation.
Our vision is to improve the added value of IT through requirements management and
software testing. Along with providing high-quality services, we pursue this vision
by establishing independent platforms, like the Swiss Testing Day and the Swiss
Requirements Day, which facilitate the exchange of know-how and experiences.
In addition to that we help bright minds to expand their knowledge in our trainings.
© by SwissQ Consulting AG | Stadthaus-Quai 15 | Switzerland-8001 Zürich
www.SwissQ.it | info@SwissQ.it | Phone +41 43 288 88 40 | Fax +41 43 288 88 39
Twitter: @SwissQ | Facebook: swissqconsulting