1. White Paper
Assurance - Not just about the bugs - Part 2
In the last white paper we discussed the difference between assurance and testing, how to
‘professionalise’ your assurance and how to build your team
In this white paper we talk about getting the environment right to enable the assurance team to bring
value; including the resources and tools they may need to do their job more effectively.
Common Processes for Common Projects
Most companies run assurance projects, which while they differ considerably in detail from project to
project, many of which could be run in much the same manner, at least at a high level. For example,
waterfall and staged-waterfall projects largely have the same sequence of test phases, performing
similar tasks in each, subject to the same controls as each other, irrespective of content. By
acknowledging the existence of these patterns, companies are implicitly acknowledging that it is
possible to define a limited set of test strategies that their projects could then adapt and follow to
achieve assurance. Further, these adaptations could be done rapidly and easily, and would ensure that
little or nothing gets forgotten in planning out the assurance of a system.
Many companies sign up to the idea that test strategies are a good idea, and alarmingly few actually
do anything more than that. The end result is that companies reinvent the wheel with depressing
regularity. By establishing a small set of test strategies, a company gains:
• Reduced start-up time in defining test plans – much of the test plan becomes a reference to
the test strategy that describes the type of project to be run. Each project’s test plan need only
describe variations form a common test strategy
• Improved service delivery – by using test strategies that amount to “pre-proven” methods, so
it is less likely that key considerations get missed and more likely that the right set of test and
assurance events is planned in and budgeted for, right from the start of the project
• Repeatable processes – well documented, well tried, proven processes for how projects run
their testing lead to numerous consequential benefits. Learning curves are reduced as each
project becomes, at least in terms of process, much like another. Project assurance teams get
to focus on the content of the project, instead of continually reinventing the same process by
which to assure its delivery of an operable business
• Consistent reporting – by driving both the outcomes for reporting and the method by which it
is implemented both for progress and outcome reporting, so you establish expectations on all
sides for how reporting will be run and what information will be delivered. By establishing
expectations, so the project by project negotiations are reduced, best practices are established,
and delivery is accelerated.

2. White Paper
Test Tools
Many companies still attempt to run complex test and assurance programmes without support from
either test automation frameworks, or in some cases, even test management tools.
The days of test automation tools being little more than interesting toys, delivering limited actual
business value, are firmly over. As Agile methods have become established, test automation tools have
similarly matured, and are now an essential part of the armoury of any assurance team.
Test management tools are similarly an established part of the assurance process, providing as they do,
facilities for the management of test assets, test results and often, facilities to analyse and report on
results, along with facilities to distribute reports.
Frequently, tools for test asset management, defect management, test automation management and
requirements management can be linked together to provide for a seamless view of the overall
assurance process. Picking the wrong tool set can be expensive, but picking the right tool set can drive
massive improvements in productivity.
Test Environments and Test Data
Common thieves of both time and effort in assurance are the provisioning of test environments and
the provisioning of test data.
Test environments are as essential to the live operation of a system and its assurance as the
provisioning of the live environment itself. Often, the simplest method for provisioning a test
environment is to design and deliver it alongside the environment for live implementation, so
increasing the likelihood that the test environment will match or at least closely align to the live
environment.
It’s equally worth remembering that although we often use the term “test environment”, in fact most
projects need a range of test environments to reflect the needs of the assurance process at the various
delivery stages in the road to live. A test strategy helps here; by defining the planned method by which
projects of a given type will be tested, so it is possible to predict the range and design of test
environments that will be needed, now and in the future, when creating the initial plans for assurance
of the business that will use the system.
Closely aligned to the provisioning of a test environment is the provisioning of test data. Again, planning
in the creation and management of test data on a system-by-system basis, rather than project by
project, can lead to significant savings of both time and money, as the processes need only be
established once, then modified as the needs of assurance change with time. Similarly, the best time
to consider test data management for a new system is when the system is being designed; at that point,
design and implementation decisions can be made, taking into account the need to assure the
operation of the business. Data management processes and structures can then be designed to include
the creation and management of test data, instead of leaving it to become an expensive after-thought,
as is often the case on projects.

3. White Paper
Considering test environment and data management from a wider perspective, there’s a shopping list
of issues that project teams must consider, including questions such as:
• Are sufficient environments available?
• Are they big enough to meet the needs of pre-live testing?
• Are they small enough to meet the needs of developer testing?
• From where can I source test data that accurately reflects data in live?
• If I’m using data derived from live, how will I anonymise that data without damaging referential
integrity?
• Where and how will I store a “safe” copy of data so that when data is changed (and damaged)
by testing, I can rapidly and easily refresh my environments?
• How can I copy targeted data out of live, to drive issue investigation and management?
By addressing these questions on a system-by-system basis, rather than leaving each new project to
reinvent the wheel, productivity is maximised and overheads are kept to a minimum. By planning for
these questions right from the start, when designing the system, costs can be reduced even further.
Training
Too often, training is seen as an expensive overhead. There is however the saying that “If you believe
training is expensive, try having your system developed by an enthusiastic amateur”. The alternative
view taken is “Well, we can always hire the skills when we need them”. The problems implicit in that
view however, are legion:
• There’s a considerable financial premium and lead time in hiring new permanent employees
• There is a considerable learning curve once you have employed someone
• Given that the IT industry has both weathered the storms of the economic downturn in the
European Union very well, and given that the growth in IT hiring is ahead of that in many other
arenas, fully trained, experienced assurance personnel are at a premium, and can command a
salary package in line with their scarcity.
These and other factors conspire to lead to the position where there simply isn’t a wide range of
permanent people with the right qualifications and experience seeking work.
Contract employees may fill the gap in the short term, providing for highly specialist skills when they
are needed, but similarly, this can be an expensive proposition in the long term.
Bringing in an assurance team under a managed service arrangement in the mid-term, alongside
growing in-house staff to replace them in the longer term, may represent the best return on
investment. By bringing in a managed service team, the immediate issue of resourcing assurance is
dealt with. The team provides the assurance service so badly needed to ensure that new and modified
systems and processes are implemented into live successfully. But somewhat counter intuitively, such
a team can also provide the framework of best practice implementation, mentoring and support
beyond training to allow permanent members of staff to replace them in the mid-term. There will
always be a need for specialist people brought in under contract to provide for specific needs, but a
managed service team can provide the framework into which an in-house team can be grown as they

4. White Paper
learn to implement the skills picked up in the classroom.
Conclusions
There really is no substitute for a well-trained, empowered employee with the right tools, data and
environment to hand, following processes that are proven to work. In the end, the long-term solution
for most companies will be a blend of permanent people supported by the right partner specialists to
build skills, implemented in an assurance framework that meets the needs of the business now, while
supporting long-term growth.
Although the cost of investing in assurance may seem considerable, steps can be taken to ensure that
overheads are reduced as much as possible, and time to full productivity reduced to a minimum.
The alternative is to allow businesses to continue as is, potentially placing security of business
operability going forward in the hands of the ill equipped and the under-resourced. Confidence in a
business on the part of their customers has taken years to build, but can be lost in a matter of days.
The investment in assurance may seem large, but can be dwarfed by the costs of failing to invest.
Contact Details
If you would like to know more about TSG or any of our solutions please contact us on
+44(0) 20 7469 1500 or email enquiries@testing-solutions.com.