- The document discusses adopting a shift left testing approach to overcome challenges with traditional testing being done late in the development cycle. Shift left testing involves testing earlier and more frequently throughout development. - Model-based shift left testing was used, which involved testers in requirements gathering and continuous testing of prioritized modules as they were developed. This allowed defects to be identified earlier. - The results showed that 30% of defects found were critical coding issues identified during development. More defects were found during integration testing compared to traditional testing later in the cycle. - The approach improved collaboration between developers and testers and allowed defects to be addressed sooner, improving quality while reducing delays from issues found late. Future work involves further aligning

1. International Journal of Emerging Technology and Advanced Engineering
Website: www.ijetae.com (ISSN 2250-2459, ISO 9001:2008 Certified Journal, Volume 9, Issue 8, August 2019)
57
Experiences in Shift Left Test Approach
Azura Othman1
, Noorul Ahlami Kamarul Zaman2
, Rajashekara Swamy3
Corporate Technology Division, MIMOS Berhad., Kuala Lumpur, Malaysia
Abstract – One of the key challenges in traditional testing
method is testing activity being pushed towards the end of
development lifecycle. As a result, the time given for testing is
short but the expectation is to have maximum test coverage.
In addition, late identification of defects in development cycle
can result in major re-work and further delays. Based on
these experiences, we have attempted to adopt Shift-Left
testing approach, which is considered as one of the best
practices in the industry. In reference to [1], the method is to
perform testing earlier in the lifecycle, test early and often.
[1] There are four methods of Shifting Testing to the Left.
These are referred to as Traditional Shift Left testing,
Incremental Shift Left testing, Agile/DevOps Shift Left
testing, and Model-Based Shift Left testing. This paper
explains the experiences in applying Model-Based Shift Left
testing method.
Keywords - Shift Left Testing, Model-Based Shift Left
Testing, Continuous Testing, System Test
I. APPROACH USED
Traditional testing approach has several challenges. The
key challenges are:
1. Complete the testing was impractical due to schedule
constraints imposed
2. Lack of skilled test resources with domain specific
knowledge
Shift left test approach was chosen to overcome those
challenges or limitations. Activity started with planning, a
discussion between Project Lead, Business Analyst,
Developer and Tester on the test approach to be used. The
discussion helped in prioritizing the user requirement that
needed more attention in terms of testing. Based on the
prioritized requirements effort and time estimate for the
testing was done.
Figure 1 illustrates Shift Left approach involving testers
much earlier in the software development life cycle. Early
involvement allowed tester to understand the requirements,
software design, architecture, coding and complete
knowledge about the system. Testing was not done towards
the end but started early and done continuously. As a result
of this approach, defects were detected at the early stage.
Figure 1: Shift Left Testing [2]
Figure 2 shows shift left model moves testing to the left
side. Testing begin immediately instead of waiting a long
time as per the traditional approach. The defects identified
earlier and in time. As a result, the time and resource were
used effectively.
Figure 2: Shift Left Approach [2]
Tester involvement started in the requirement gathering
activity phase. This was to prevent misunderstanding of the
requirements gathered.

2. International Journal of Emerging Technology and Advanced Engineering
Website: www.ijetae.com (ISSN 2250-2459, ISO 9001:2008 Certified Journal, Volume 9, Issue 8, August 2019)
58
Since then, test execution was performed continuously,
discovered the defect and highlighted the improvement of
the system. As defects and improvements were identified
earlier before the system development completed and
before deployment for User Acceptance Test, has resulted
in an efficient testing and improving the quality of the
system.
II. TESTING APPROACH
The team adopted following steps to commence in
implementing shift left testing.
Figure 3: Testing Approach
Tester were involved in project discussion between
project lead, business analyst and development team to
identify the user requirements. Requirements were
associated accordingly with the core module. Prioritization
of the modules for testing, which was based on critical
functionality, was essential to the whole process. Tester
reviewed and analyzed the requirements for better clarity,
testability and acceptance criteria definition.
Test lead proceed with the time, effort estimation for the
complete cycle of testing activities. After that, the test lead
assigned the test resources for each release build in order to
meet the pre-defined timeline.
Test cases were created for the defined critical
functionalities. These were categorized as highest priority
test cases. Major defects have been prevented while
entering into the user Acceptance Test cycle. Test
execution emphasized on the module functioning as per the
user requirements.
As per Figure 4, test execution process consisted of
executing the test scripts or test cases created for the
specific module.
Figure 4: Test Execution Process [3]
Test execution started with the core module that has
been defined earlier. Once the core modules were tested
and function well, other related modules taken up. Closed
interaction, frequent and regular discussion between
developers to speed up the testing cycle. Tester produced
the test result at every cycle of released and analyzed the
results.
JIRA was used in managing requirements, test cases and
defects. Usage of this integrated tool by developers and
testers, helped to trace the test cases to requirements and
defects to requirements.
III. RESULTS ANALYSIS
The defects and improvements captured during the test
execution were associated accordingly with the fault
category and the root cause.
Figure 5: Defect Raised by Severity
27%
55%
18%
Defect by Severity
Critical Major Minor

3. International Journal of Emerging Technology and Advanced Engineering
Website: www.ijetae.com (ISSN 2250-2459, ISO 9001:2008 Certified Journal, Volume 9, Issue 8, August 2019)
59
Figure 5 provides summary of test results by defect
severity. Defect severity is a classification of defect to
indicate the degree of negative impact on the quality of the
system. 27% of defects were categorized as critical, 55%
major and 18% were raised under minor severity. Critical
defects proved to be a showstopper for other functionalities
to behave as per the requirement.
Figure 6 shows the defect by fault category. 91% of the
issues were related to the functionality, while another 9%
were related to Usability. Functionality failed due to the
syntax error, improper validation, and not operate as it
supposed to. In addition, usability defects were associated
to graphical user interface, consistency, simplicity, easy to
navigate and understandable.
Figure 6: Defect by Fault Category
Figure 7 indicates the percentage of defects by root
cause and severity. Two major causes were identified, one
due to coding and the other due to improper
implementation. Coding issues were identified due to
limitation of open source component used during the
development. The remaining defects, which were due to
improper implementation, raised as minor severity.
Figure 7: Percentage by Root Cause
A review session with the project lead, business analyst
and developer was conducted. The team went through each
of the defect and discussed in details. Each defect was
categorized accordingly, whether it was defect or
improvement.
IV. SUMMARY
Figure 8 shows that majority of the defects were due to
the coding of the functionality, which were raised during
the development phase. This indirectly helped to improve
the code quality. The number of defects found during
System Integration Test increased by 50% versus the
defects found during System Test. These were related to
the upstream and downstream process flow dependencies
issues. The value moved to downstream services not in
synced.
User feedback given during User Acceptance Test were
classified as improvement. Improvement focused on
usability, user's ease to use the system and the flexibility in
handling the system.
91%
9%
Defect by Fault Category
Functionality Usability
0
10
20
30
40
50
60
70
80
90
100
Coding Issue Improper
Implementation
[VALUE]
%
0
[VALUE]
%
0
[VALUE]
%
[VALUE]
%
Defect by Root Cause

4. International Journal of Emerging Technology and Advanced Engineering
Website: www.ijetae.com (ISSN 2250-2459, ISO 9001:2008 Certified Journal, Volume 9, Issue 8, August 2019)
60
Figure 8: Number of Findings
In summary, the following are key pre-requisites for the
shift left testing approach.
1. Module prioritization to be released for testing
parallel with development activity
2. Teamwork between developer and tester. Tester
involved in all critical stages of the program
Some of the key learnings in applying shift left testing
approach are:
1. Shift left testing was a team effort between developer
and tester, it cannot be achieved by tester alone
2. Since testing was done by focusing on one module at
a time, higher number of defects were detected in
each module as compared to testing the entire
application at once. As show in Figure 7, 30% critical
and 60% of major defects caused by coding issues
3. Developer and tester decided to execute the test in the
development environment before the build being
released to test environment
4. Differences in configuration between development
and test environment influence the functionality and
behavior of the system
Overall, this approach eliminated waiting time for
development to be completed and new release build with
new features. Collaboration with development team was
greatly improved.
Tester's involvement from user requirement phase gave
better idea and saved a lot of time discussed on what the
system meant to do. Added with continuous testing reduced
defects in the end.
The final Test Summary Report determined the tested
system was viable enough to proceed with the next cycle of
acceptance test by the User.
V. CONCLUSION AND FUTURE WORK
In the future, we are planning to implement the above
testing approach by conforming requirement prioritization,
plan release cycle, test case development, test execution
and test results analysis in parallel to the development
activities. Testing earlier and continuously, thereby,
resulting in efficient testing as well as improving the
quality of the system. Shift left testing has helped us in the
following areas.
1. Cross-team collaboration between development and
testing, it bound effectively
2. Bridging the gap between developers and testers to
speed up the testing
3. Reduced the idle time and test resources used
effectively
4. Identified the improvement and verified the defects
right from the beginning in the development cycle
5. Uncovered implicit requirements
6. Focused on particular module, hence the complexity
of testing effectively managed, and thereby increasing
the coverage
7. Prevented project from delay due to late testing and
insufficient testing time allocated
Acknowledgment
The authors sincerely thank MIMOS Berhad, Malaysia,
for supporting the Software Engineering Lab in the
implementation of the Shift left testing approach for the
project.
REFERENCES
[1] Shift Left Testing. Wikipedia https://en .wikipedia.org/wiki/Shift
_left testing
[2] Shift Left Testing; A Secret Mantra for Software Success - June 19
2019
[3] International Standard ISO/IEC/IEEE 29119-2. Software and system
engineering – Software testing. Part 2: Test process
[4] The Shift Left Principle and DevOps by Rick Weaver - June 14 2016
[5] 10 Steps to Get Started with Shift Left Testing. Testing Whiz - June
13 2017
[6] Test Early and Often. Microsoft - September 28 2012
15
9
18
7
2
4
13
%
of
Finding
Cycle
Number of Findings Raised
Defect
Improvement

5. International Journal of Emerging Technology and Advanced Engineering
Website: www.ijetae.com (ISSN 2250-2459, ISO 9001:2008 Certified Journal, Volume 9, Issue 8, August 2019)
61
[7] How “Shift-Left” Testing Can Help Your Product Quality? Harshit
Paul - December 28 December 2018
[8] Shift Left Approach and Practices. Paul Bahrs - 6th
November 2014
[9] The Shift-Left Approach to Software Testing. Arthur Hicken -
December 10 2018
[10] Shifting Testing Left Is a Team Effort. Mark Franssen, Jan Jaap
Cannegieter - June 11, 2018