This a research made for study purpose ( Master degree in computer science) by Nora Alriyes

1. 1
King Saud University
College of Computer & Information Sciences
Department of Computer Science
The productivity of testing in software development
life cycle
CSC 569
By
Nora Abdul-Aziz Alriyes
434920502
Email(nora.alriyes@gmail.com)
Under the supervision of
Dr. Saleh Hammami
31/12/2015

2. 2
ABSTRACT
This paper introduces software testing as productive phase in SDLC based on many experiences in
the market place. Organizations have faced many testing challenges in term of limited time and
resources against the rapid development of new generation software. In our research we intent to
discover testing as more productive than detecting phase and answer three questions about testing:
Who test my software? How test my software? and what to test in the software. In order to answer
these questions, we identified the advantage and disadvantages of many strategies based on case studies
or true experiences of companies. For example the value of testing of sole tester and how we could
enhance this role; we also discovered google strategy of testing and we found many important aspects
should be taken into account for tech organizations; Finally, to answer the question: what to test in the
software we proposed the benefits of testing at many models like: waterfall model, Risk based model
and iterative model based on the needs of varied software companies.
Keywords—Testing ; Google; software testing; Testers; Testing Roles;

3. 3
TABLE OF CONTENTS
I. Introduction................................................................................................................................................................................ 4
II. Related work.............................................................................................................................................................................. 4
A. Who tests my software ?........................................................................................................................................................ 4
B. How to test my software?...................................................................................................................................................... 4
C. What to test in my software? ................................................................................................................................................. 5
III. Disussion ............................................................................................................................................................................... 5
IV. Conclusion............................................................................................................................................................................. 6
V. References.................................................................................................................................................................................. 6

4. 4
I. INTRODUCTION
Testing is fundamental phase in Software development life
cycle (SDLC) which is used to validate the software based on
user needs. Or it can be defined as : “the process of executing a
program with the intent of finding errors”.[3]. So, the primary
goal of this phase is detecting issues that could interrupt the
users’ goals.
Nowadays, with complexity of the software and the
demanding of business trade, testing becomes more challenging
practice.
To create a software, commonly many companies and teams
uses waterfall model [Figure 1] with testing as the last phase
before ship the software to the customer. Often Testers dilemma
when they discovered important issues in late stages, then it
takes more time correcting them by the developer, and
sometimes this situation can lead also to accumulated wrong
code design that takes much more time to be rolled back. This
situation is being faced by many practitioners and impact
negatively on customer satisfaction. [3]
Our research goal is to discover the effectiveness of tester
role, also to find new ways to make testers more productive in
SDLC model.
[Figure 1] Waterfall model [5]
II. RELATED WORK
A. Who tests my software ?
The authors in [1] emphasized that testing is socio-Technical
rather than purely technical.
They tried to fill the gap between the researches and the real
practitioners in the technical market. The three researchers
discovered testing organization and testing value in three
successful software companies. They discovered the defects
database and extensively did many interviews with employees
about the test processes for each company. They classified the
issues based on employee’s duties, and identify the importance
of these bugs and fixing rate.
[Figure 2] represents the average results they found in for all
the three companies. They found that Testers have the lowest
important defects for the customers. In contrary, Managers, sales
employees and consultants found the most important issues from
user point view.
In addition, the defects found by developers have the highest
fix rate, but testers have the lowest fix rate defects.
Finally, they conclude, that the responsibility of testing any
product is getting important with those whom close more to the
real users, while they focused more on the customer view point.
And the testing process by involving by managers and sales
employees; that would improve the product more than individual
tester. [1]
Despite of that author in [4] raised that many testers lacked
the appropriate vision for the product. As well as the challenge
to test high coverage of the combination of the hardware and
software. Coverage is referred to: Extent of testing for the
feature of software to cover all the functional code; this concept
is a way to measure how much percent the tester can cover the
functional code and prove that it worked properly.
[Figure 2] The average results of defects importance and fix rate
in[1]
B. How to test my software?
Google one of the companies who invents their own creative
way of testing. When we think about google we think about
enormous successful applications and products. At google
testers is a part of their centralized organization called
“Engineering productivity”. At google product team who owns
the quality not testers, while we all know that developers can’t
test! The author: James Whittaker raised that as:” Quality is a
development issue not a testing issue, to the extent to embed
testing practice inside development”. [2]
They invented two types of testing roles in addition to the
developer role they called as following: SWE (Software
Engineer), SET (Software Engineer in Test) and TE (Test
Engineer).
SWE is a traditional developer role who builds he functional
code, create design description and data structure. SWE also
responsible for TDD unites tests, fault tolerance designs, failure
recovery …etc.
SET is the owner of testing feature. SET’s responsible about
the testability of the code, they review the design to ensure that
the code is clean, consistent and work long. They focus more on
testing as a developer then as a user.

5. 5
TE is the responsible about functional defects, whom
prevents the user from bad designs, security, privacy and
confusing user experience. TE also spend a time write a code as
automation script and the code that drives usage scheme. So,
they focus more on testing as user then as a developer. [2]
One of the keys that google depend on with few testers is
building few features then release it in the moment it’s useful.
They are not attempt to ship many features at once. This idea
supports by their types of tests. They have only three types of
testing:
 small test: or it called unit test, automated and used to
exercise the code for only one function. It wrote by SWE
or SET in development environment.
 Median test: it can be manual or automated in testing or
real environment; this type of testing is to ensure that
neighbors of functions incorporate with each other
successfully.
 Large test: covers three or more feature and represents
real user scenario, SWE, SET and TE all responsible on
writing large tests to ensure that software works in the
way users expect.
TE’s who can decide where to focus on and they interpret
test results and execute the most tests scripts. They are playing
crucial role, the authors of book [2] point them as: ”They are
product experts, quality advisers and analyzers of risk”.
Google invents these testing techniques of what they
believed that it is more productive than the traditional testing
practices in a way to get over produce buggy systems. [2]
C. What to test in my software?
Testing practices are must be varied between tech companies
based on their needs and resources.
Waterfall is still popular model to develop software. Testing
engineers who used traditional modeles have faced many
challenges against rapid development cycles. The authors in[3]
tried to state some of solution to adapt with waterfall testing
model.
The main problem with waterfall model that software
development life cycle (SDLC) is only a part of overall software
life cycle Figure [1]. Most software products sustain under
maintenance phase to correct the accumulated defects during the
development phase which could take time more than the cycle
itself. [3]
The authors in [3] points to many important testing
challenges:
1. The limited time and resources
2. The delay to deliver the releases which impacts
negatively on customer satisfaction.
3. Finding the most important defects in the last stages.
Their goal is to form a robust model for development/Testing
without compromising business needs and overcomes the above
challenges in testing.
The first model they proposed for testing is Risk Based
Testing Model (RBTM): “it is an approach recommended for
projects with schedule limitation”. It’s basically prioritize what
develop and test first from software requirements based on: Risk
of failure for a function and the impact of that failure. [3]
It tests first the highest risk functions then medium, finally
low risk functions. As testing practice, we must categorize every
test case by priority, important for the user and the risk of failure.
[Figure 3] is an example of prioritize test cases in RBTM
model from [3] study. A factor called Functional Point (FB) to
estimate the test cases and calculated as FP= Test case weight
*Test case Points.
And every test case must be selected or not based on risk of
failure. The total effort estimation for this model decreased by
30 percent. in the RBTM than in waterfall model.
Also in [3] mentioned ITM (iterative test model) as a useful
approach by increment development or agile method. This
method can be useful when specify groups of functions for every
cycle and if it can be as a start point for the next cycle.
The authors Evaluated the three models: Waterfall, iterative
and Risk based testing model in matter of time and cost and
conclude that the lowest execution time and testing cost was in
RBTM than iterative model and waterfall model for more than
30 percent.
Figure [4]: example of prioritize test cases in RBTM model
[3].
III. DISUSSION
The case study in [1] clarified a strong view point that testing
practices and the defects by solo tester or external testing
company don’t have a strong impact on the product or for the
user. While the high important defects are raised those whom
engaged more with customer and the real users those whom
socially and technically aware of the product goal.
However, the study [1] was limited on only three companies;
the approach that used by these companies, software types and
size all are important factors to take into account and evaluate
the tester role. We also suppose that the number of products
delivered by each company and the rate of customer satisfaction
are also important to evaluate testing impact.
Nevertheless, we do believe that testers have crucial role to
validate systems functions, but they can be more effective when

6. 6
they became more involved with users socially and with
developers technically.
In the other hand google shows solid organizational roles of
testing which covered by all product team members, SWE, SET
and TE all have crucial role to make each other productive more
than the traditional roles. It is also a way to prevent he product
of accumulating a wrong steps of system design that can’t be
rolled back.
Google tried to ensure that problem of test coverage is
resolved by testing every feature with small test, then make sure
that all these tested features are working probably with medium
and large test; also by making tow roles of testing the code that
can be a solution for the many challenges raised in [4].
Although, many flaws have been raised by google inventor
against their model of testing, first, providing tow testers would
lead the developer to depend totally on the testers to not think
about testing his own code; that’s can be against their goal of
testing:” Quality must be built in not bolted on”.
In addition, we agreed with the authors of [3] that TE’s must
not spent enormous time on testing and more enable the users to
try their product as soon as it gets useful; the real users will
always found issues can’t be found by a tester.
Another side to discuss about google strategy of testing is
that google made their own products by specific requirements
internally for the general audience, while most tech companies
have their commitments with specific customers with limited
time and resources to deliver their product. And we suppose that
not all companies can adapt with google strategy with that
limitations.
We had discovered in [3] that the authors recommend an
efficient model instead of waterfall model, that kind of model
can be adaptive with many companies used Waterfall or agile
methods; Risk Based Testing Model (RBTM) can adapt within
any traditional model of SDLC. It would also decrease the test
time. Also we believed that it will make the tester find the
important issues in earlier stages. Besides, Iterative model
recommended more than RBTM for the projects need more user
involvement with a quick feedback.
IV. CONCLUSION
In this research we presented many testers challenges and
discovered testing models and answer three questions; Who and
how and what to test in the software. We found that the solo
tester is not effective enough without socially technically
engaging with the user, product developers. While, the most
important testing tasks must be with the one who’s understand
the social, organizational and technical context of the product.
In order to answer the question how to test any product? We
found that no tester could test and cover effectively many feature
once, this could lead to accumulate many wrong steps in code
design which is takes enormous time be rolled back. Revising
the code design for every feature and make it able to be tested
by automated scripts is very effective way to make sure that the
code is consistent and clear and can work long. Developers need
to be more productive by testing their feature one by one and
gradually test neighbor of functions then the whole system. So
they can prevent to build buggy systems need more time and
effort than the SDLC cycle itself.
Risk based testing model got over on many challenges of testing
in waterfall model which is basically could prioritize functions
based on risk to be tested first then we can avoid find the most
important issues in late stages. Finally, we can emphasize that
it is always better to release the software as soon as it becomes
useful for the real users; they would always find issues more
than any tester.
V. REFERENCES
[1] Mäntylä, Mika V., Juha Itkonen, and Joonas Iivonen. "Who tested
my software? Testing as an organizationally cross-cutting activity."
Software Quality Journal20.1 (2012): 145-172.
[2] Whittaker, James A., Jason Arbon, and Jeff Carollo. How Google
tests software. Addison-Wesley, 2012.
[3] Sivakami, K., and K. Mohan Kumar. "Comparative Study on
Software Testing Models." (2014).
[4] Kaner, Cem. "Fundamental challenges in software testing."
Colloquium at Butler University. 2003.
[5] Travis Morris. WHAT BEING AGILE MEANS TO ME. Web. 3 Aug
2015.<http://www.travdes.com/wp-
content/uploads/2015/08/maxresdefault.jpg >