1. In next 3 min …
Renata Čičman, Tieto
Aistė Stikliūtė, Webmedia
UAT WITH
SALT
The taste of UAT in large projects
© 2011 Tieto Corporation

2. Aistė Stikliūtė Renata Čičman
LinkedIn:http://lt.linkedin.com/in/aistestikliute LinkedIn: http://lt.linkedin.com/in/renatacicman
Blog: http://luni.net84.net/ Blog: http://reci7422.projektas.in
Twitter: @RenataCicman

3. IN THE NEXT 30 MIN …
COMPLAINTS FINDINGS
SARCASM GOOD EXAMPLES
STOLEN IDEAS ADVANTAGES OF
LESSONS LEARNED
OUR IDEAS
© 2011 Tieto Corporation

4. HOW TO START?
Do not wait for someone to tell You what to do
Determine company culture and Collect existing processes
existing systems and procedures
Understand the Identify
Business Case stakeholders, requirements, ri
sks, constraints, assumptions
© 2011 Tieto Corporation

5. WHAT TO BE CONSIDERED?
Context in which the UAT testing
is to be performed
Consistency in testing across
the testing teams
Schedule
Methods and
techniques, metrics, tools
Communication
© 2011 Tieto Corporation

6. BEST PRACTICES
There are good practices in
context, but there are no best
practices.
Understand your context and
then make your approach
(Michael Bolton @ DevelopSense,
http://www.developsense.com/presentations/
User%20Acceptance%20Testing%20-
%20STAR%20East%202006.pdf)
© 2011 Tieto Corporation

7. ENVIRONMENT –
Human resources
POSSIBLE ON TIME?
How to make environment come on time:
• Rehearsals
• Set up the environment in advance
• Pre-test the environment
• Good configuration management is the key!
© 2011 Tieto Corporation

8. CASE STUDY
2 environments with different versions of the system
• Same test data (periodic loads from production)
• Same schedule
• No other systems integrated, no other teams involved
© 2011 Tieto Corporation

9. HOW MANY /
Human resources
WHAT METRICS TO USE
Per team / area:
• Percentage of test cases executed
• Defects found (per priority/severity)
• Defects found (per root cause)
•…
© 2011 Tieto Corporation

10. HOW MANY /
WHAT METRICS TO USE
How many: depends on what
decisions you’ll want to make
No ambiguities!
Performance metrics?
Probably not, but...
© 2011 Tieto Corporation

11. CASE STUDY (2 METRICS)
© 2011 Tieto Corporation

12. CASE STUDY (MANY METRICS)
© 2011 Tieto Corporation

13. TEST CASES: WHO SHOULD
DEFINE?
• Customer knows what’s important to him
• Vendor knows how to use the system
• Solution:
• customer gives guidelines
• vendor writes test cases
• customer can add test cases
© 2011 Tieto Corporation

14. HOW TO SELECT WHAT TO TEST?
Critical
success
Typical factors???
scenarios
High risk
scenarios
Business
benefits???
Existing
requirement
documentation
© 2011 Tieto Corporation

15. TRAININGS FOR UAT
TEST ANALYSTS
• Purpose of UAT
• Ownership of the testing
• Rely on "End-User" documentation
• Use of the Testing Tool/Repository
• Activities involved
© 2011 Tieto Corporation

16. SUPPORT FOR UAT
TEST ANALYSTS
• Inexperienced Testers
• Compliance Issues
• Poor Release Control
• Unstable Test environment
• Poor Planning
• Incomplete Test Scripts
• ...
© 2011 Tieto Corporation

17. BUSINESS SUPPORT
FOR TEST ANALYSTS
• Gaps in requirements
• 3rd parties performing acceptance testing
© 2011 Tieto Corporation

18. ‘PILOT’ – THE WAY
TO PREVENT FAILURE
• Is our UAT strategy working?
• Are our trainings effective?
• Gaps in testing scripts? In requirements?!
• Should we use UAT environment for ‘Pilot’?
• Should testing results from ‘Pilot’ be considered as part
of UAT?
© 2011 Tieto Corporation

19. PHASES FOR UAT?
ITER. WTF.
How to split:
• By functionality to be rolled-out first
• By resources availability
When to split:
• Time frame is long enough
• By diversity of activities
What ever approach you choose it must be
thoroughly planned in advance
© 2011 Tieto Corporation

20. CASE STUDY
From staged
UAT1 UAT2 UAT3 Reh. Go-Live
to continuous
UAT2 cont. Defects retesting
UAT1 cont.
No env. UAT3 cont.
© 2011 Tieto Corporation

21. COMMUNICATION
• How to communicate with UAT test analysts?
• How to communicate progress?
• What channels should be used?
© 2011 Tieto Corporation

22. DEFECTS TRACKING
• There shouldn’t be many new defects during UAT.
• But they need special attention.
© 2011 Tieto Corporation

23. DEFECT ANALYSIS
Why we find defects in UAT still?
• Business-knowledge related defects,
like missed requirements
• Technical issues because of newly
setup environment
• Defects were missed in
functional testing
© 2011 Tieto Corporation

24. POSTPONING FIXES
ACCEPTABLE OR NOT?
• Should UAT defects be of higher
priority by default?
• When should UAT defects be closed?
© 2011 Tieto Corporation

25. CASE STUDY
Why these bugs are
found only now???!
Why are you testing things Why TC that I tested 5 times
that are not in TC ??? now suddenly FAILS?!
© 2011 Tieto Corporation

26. CHALLENGES
• Environmental (technical) issues
• Scope control
• Business users engagement
• Plan changes
© 2011 Tieto Corporation

27. ENVIRONMENTAL
(TECHNICAL) PROBLEMS
• super-fast communicating
and escalating
• Rescheduling
• planning pre-prod or prod
tests
• postponing UAT
• releasing without testing in
UAT
© 2011 Tieto Corporation

28. SCOPE CONTROL
• Keep scope unchanged as much as possible
• Apply bureaucratic view for any potential changes
• Testing scope can be reduced but more often vice
versa
© 2011 Tieto Corporation

29. SCOPE TENDS TO GROW…
© 2011 Tieto Corporation

30. CASE STUDY
Scope change during UAT preparation
© 2011 Tieto Corporation

31. CASE STUDY
Scope change during UAT execution
© 2011 Tieto Corporation

32. BUSINESS ENGAGEMENT
Need whipped cream or a whip? :)
Vs.
© 2011 Tieto Corporation

33. BUSINESS ENGAGEMENT
UAT timeframe vs.
UAT scope vs.
business availability
© 2011 Tieto Corporation

34. SIGN-OFF
• Sign-off time causes sudden diligence :)
• The key: sign-off strategy confirmed in advance
© 2011 Tieto Corporation

35. CONDITIONAL SIGN-OFF
• Reduces the number of sign-off iterations
• Planning the follow-up:
- Due date
- Resources for verification
© 2011 Tieto Corporation

36. YOUR QUESTIONS?
Aistė Stikliūtė Renata Čičman
LinkedIn:http://lt.linkedin.com/in/aistestikliute LinkedIn: http://lt.linkedin.com/in/renatacicman
Blog: http://luni.net84.net/ Blog: http://reci7422.projektas.in
Twitter: @RenataCicman
© 2011 Tieto Corporation