Lean and efficient test process framework (TestSmart) 1.Provide quality gates across the Application Lifecycle to prevent the problems faced currently 2.Incrementally builds the missing requirements and test cases 3.Improved defect containment resulting in increased customer satisfaction 4.Reduces overheads and consequently cycle time

1. © 2013. Maveric Systems Limited
Hari Babu T
Principal, Test Process Consulting
Essential Building Blocks
of a Lean & Efficient Test
Process
This presentation and supporting material is owned by Maveric Systems and is for sole use of intended audience only.
The presentation may contain content that is confidential or proprietary and may not be distributed or copied without the permission of Maveric Systems.

2. © 2013. Maveric Systems Limited 2
Testing Challenges
Solution to these Challenges
Building Blocks of a Lean & Efficient Test Process
Summary
Agenda

3. © 2013. Maveric Systems Limited 3
Challenges
Testing Challenges
▸ Large number of applications in production
▸ Lack of test case definitions/obsolete test case definitions for applications
in production
▸ Large number of Change Requests, Enhancements & Production issue
resolutions incorporated into applications in production
▸ Lack of requirements definitions for applications in production
▸ Multiple levels of testing with either overlapping or missing testing coverage
▸ Inadequate impact analysis
▸ Absence of regression test packs based on impact
▸ Up gradation or Replacement of application is required every few years
▸ Cumbersome paperwork/Non-integrated tool set

4. © 2013. Maveric Systems Limited 4
Agenda
Testing Challenges
Solution to these Challenges
Building Blocks of a Lean & Efficient Test Process
Summary

5. © 2013. Maveric Systems Limited 5
MTS – Solution to Challenges
Solution
Lean and efficient test process framework (TestSmart)
▸ Provide quality gates across the Application Lifecycle to prevent the problems
faced currently
▸ Incrementally builds the missing requirements and test cases
▸ Improved defect containment resulting in increased customer satisfaction
▸ Reduces overheads and consequently cycle time

6. © 2013. Maveric Systems Limited 6
Agenda
Testing Challenges
Solution to these Challenges
Building Blocks of a Lean & Efficient Test Process
Summary

7. © 2013. Maveric Systems Limited 7
Single Stage Tool
Aided Design
Test Cube
Notation
Organization
Level Test
Strategy
Integrated
toolset
Requirements
Taxonomy
Define and use
“Build Quality”
Metrics
Test Organization
aligned with test
strategy
Configuration
Management
across
Application Life
Cycle
Building Blocks of a
Lean & Efficient Test
Process
Building Blocks of a Lean and
Efficient Test Process

8. © 2013. Maveric Systems Limited 8
Test Organization
aligned with test
strategy
Building Blocks of a Lean and
Efficient Test Process
Define and use
“Build Quality”
Metrics
Configuration
Management across
Application Life
Cycle
Organization Level
Test Strategy
Building Blocks of a
Lean & Efficient Test
Process
Requirements
Taxonomy
Integrated toolset Test Cube Notation
Single Stage Tool
Aided Design

9. © 2013. Maveric Systems Limited 9
Organization Level Test Strategy
▸ Lack of clarity on the levels, types
of testing that are to be performed
by the vendor/development and
testing teams
▸ Majority of the defects are
detected at the UAT stage
▸ Test effort is not used optimally or
compromised under pressure
Why needed?
▸ Clear identification of test levels
and types applicable and the
responsibility for each of them
▸ Applicability of the various test
types based on size, impact and
criticality of the feature being
changed
▸ Coverage type and coverage
requirements for each test level are
identified
What is addressed?

10. © 2013. Maveric Systems Limited 10
Test Organization Aligned with
Test Strategy
▸ Testers are not assigned to be
responsible for a test type
▸ Testers do not have the required
skill set for the assigned
test type
Why needed?
▸ Testers perform the intended test
type
▸ Testers skills are assessed and
improved based on the test type
▸ Better quality of testing
What is addressed?

11. © 2013. Maveric Systems Limited 11
Requirements Taxonomy
▸ Requirements are documented in
word documents and not in a
particular order
▸ Difficult to extract, update
requirements and build traceability
(bidirectional and requirements
traceability)
Why needed?
▸ Requirements, Test Cases, Defects
and Incidents are organized based
on the Requirements taxonomy
▸ Facilitates easy traceability,
storage, retrieval and maintenance
▸ Requirements and test cases are
updated on the existing baseline
and rebase lined (not to be
duplicated)
What is addressed?

12. © 2013. Maveric Systems Limited 12
Single Stage Tool Aided Design
▸ Very little understanding of formal
test design techniques
▸ Difficult to gauge the quality and
coverage of manual test design
▸ Multiple levels of test design
resulting in redundant work and
difficult to maintain artifacts
Why needed?
▸ Quality and coverage is assured
and consistent
▸ Test Design effort can be reduced
by upto 50% (over the applications
life span)
▸ Unified framework for test case
documentation
▸ Significant reduction of test case
maintenance effort
What is addressed?

13. © 2013. Maveric Systems Limited 13
Test Cube Notation
▸ No defined approach to develop,
use and manage test sets
▸ Testers consider the test set as a
product which arises automatically
when specifying test cases
▸ Switch from “Project” to “BAU”
mode and vice versa resulting in
discrepancy between the
application & the test set
Why needed?
▸ Specify test cases & execute test
cases on the basis of prioritizing
▸ Report progress of test
specification and/or test execution
quickly and adequately
▸ Plan and estimate test projects
accurately
▸ Compose regression test sets
quickly and flexibly
▸ Incorporate changes of the test
object easily in the test set
What is addressed?

14. © 2013. Maveric Systems Limited 14
Integration of Toolset
▸ Mostly standalone tools used for
Incident Management, Change
Management, Configuration
Management and Test
Management
▸ Toolset is incorrectly configured
correctly
▸ No roadmap for toolset
▸ Same data replicated in multiple
systems with inconsistent data
Why needed?
▸ Single stage data entry
▸ Reduction of data inconsistencies
▸ Better traceability and MIS
▸ Reduced documentation overheads
What is addressed?

15. © 2013. Maveric Systems Limited 15
Improved Configuration
Management
▸ Traceability missing between
objects tested and objects moved
to production
▸ Requirements/Test set not under
configuration management
resulting in test set becoming
quickly obsolete
▸ Test environments not under
configuration management
resulting in inconsistencies
between production and test
environments
Why needed?
▸ Clear identification of baselines
▸ Build numbers used as a basis to
track the movement of test objects
between development  testing
 production environments
▸ Anytime availability of the latest/
previous baseline requirements/
test set
▸ Clear traceability to the test
environment used reducing
environment related issues
What is addressed?

16. © 2013. Maveric Systems Limited 16
Define and Use “Build Quality”
Metrics
▸ No mechanism to gauge the quality
of the builds being provided by
development/vendor
▸ Pressure on the organizations
business and testing teams to
ensure timely release to production
leading to a number of
requirements/defect fixes being
deferred
Why needed?
▸ A transparent mechanism to
measure the quality of the builds
▸ Development contracts are
managed better resulting in
decreased need to defer defects
What is addressed?

17. © 2013. Maveric Systems Limited 17
Agenda
Testing Challenges
Solution to these Challenges
Building Blocks of a Lean & Efficient Test Process
Summary

18. © 2013. Maveric Systems Limited 18
Summary
How TestSmart maps to lean principles
OrgLevelTest
Strategy
TestOrganization
alignedtotest
strategy
Requirements
Taxonomy
SingleStageTool
AidedDesign
TestCubeNotation
Integrated
toolset
ConfigurationMgmt.
acrossapplicationlife
cycle
Defineanduse“Build
Quality”Metrics
Eliminate waste  
Amplify learning  
Decide as late as
possible
 
Deliver as fast as
possible
  
Empower the team  
Build integrity in   
See the whole 

19. © 2013. Maveric Systems Limited
Thank You
haribabut@maveric-systems.com