Agile User Acceptance Testing - Incorporating UAT into Agile

Incorporating UAT into Agile
“Do It Right The First Time”
#AgileAcceptanceTesting
#AgileUAT

XBOSoft
Dedicated to Software Quality Improvement
Founded in 2006

We speed products to market with our expert:
•  Software QA consulting
•  Software testing

Global team with offices in San Francisco & Beijing
© XBOSoft, Inc. 2019 All Rights Reserved @xbosoft @philiplew
© XBOSoft, Inc. 2019 All Rights Reserved

House Rules
§  Participants other than the speakers are muted
§  Ask questions in the GoToWebinar/Zoom control panel on the right
side of your screen or through Twitter @XBOSoft
§  Questions may be asked throughout the webinar - we’ll try to answer
them at the end
§  You’ll receive info on recording after the webinar
§  Ask questions and participate (I’ll ask questions tooJ
§  Webinar Hashtag: #AgileAcceptanceTesting #AgileUAT

Meet Our Speaker
Philip Lew
CEO and Founder, XBOSoft
• Relevant specialties and passions
o  Software quality process, evaluation,
measurement and improvement
o  Software quality in use / UX design
o  Mobile User Experience and usability
o  Cycling and travel

Meet Our Special Guest Speaker
Cheney Ma
Product Manager, Go2Group
• Relevant specialties and passions
o  Test planning, setting up QA processes, procedures and
tools, defect tracking and analysis, and test reporting.
Wealth of experience in working with both Waterfall
and Agile development methodologies
o  Certified Project Management Professional (PMP).
o  Product manager for Go2Group’s SynapseRT test
management plug in for JIRA.
o  Atlassian Certified Professional JIRA Administrator
(GURU)

Software
Builder
Software
Software
Builder
End User
Business
Needs
The Need For
User Acceptance Testing
Business
Needs
Software
Sponsor
End User
Software
Builder
Software
@xbosoft @philiplew

Where is Testing in Agile?
@xbosoft @philiplew

With Agile What Happened?
Many “Check” and Don’t Test
•  Who is verifying/
checking?
•  Who is testing?
•  Who is accepting?
Testing Verification

Testing Versus Verification
•  Is trying to break = Does
it work?
•  Are you testing or
verifying?
•  Testing is not equal to
Verification
Testing
Verification
@xbosoft @philiplew

Problems with Traditional “UAT”
•  Get a bunch of users who come from different backgrounds
•  Usually at the end of development
•  Limited period of time
•  Not testers
•  Test from their own viewpoint
– Usually only cover most common scenarios
– Some important data or information validation may be
missed
@xbosoft @philiplew

Let’s Look at Agile
•  “Individuals and interactions over processes and
tools
•  Working software over comprehensive
documentation
•  Customer collaboration over contract negotiation
•  Responding to change over following a plan”

Where’s the Beef?
•  “Satisfy the customer”
•  “Valuable software”
•  “Changing requirements”
•  “Face to face
conversation”
•  “Motivated individuals”
•  “Technical excellence”
•  “Regular intervals”
Where’s the
Quality and Testing?

PUTTING QUALITY INTO AGILE
IMPLEMENTING AGILE REQUIREMENTS
“Valuable software” achieved through “Satisfying the customer”
“Changing requirements” facilitated by “Regular intervals”
@xbosoft @philiplew

Requirements
User
Stories
Acceptance
Criteria
Acceptance Tests
User Acceptance Testing
UAT in Agile

DEVELOPING EPICS AND USER STORIES
Using Jira with SynapseRT - Demonstration
@xbosoft @philiplew

Epics and Stories
•  An Epic is a high level requirement that may contain several
User stories
•  User story needs to be broken down smaller pieces
– These smaller pieces represent work that needs to get done.
– Called tasks with details on feature description
•  Sub tasks can be used to define further
•  Enter a work estimate for user story or for task
•  Let’s look at one user story in detail

Representing Requirements: User Stories
●  From the user's perspective, avoid use of technical terms and
implementation details.
●  General form of a user story:
○  As a < role>, I want <goal/desire> so that <value / benefit>.
○  Role is the user class of people who will be doing the task
(eg: accountant, receptionist, bank teller)
○  Action explains the specific outcome. (eg: log in to my
email, pull up a patient’s X-ray, sell shares of stock)
●  Value or benefit is explained from the user’s point of view.
@xbosoft @philiplew

Building User Stories
Examples of Value/Benefit
●  As a shareholder, I want to sell 50 shares of stock so that I can get the
money in my account.
●  As a webmaster, I want to count how many people visit my site every
day so that my sponsors will know the popularity of my site.
●  The value is specific, but we don’t know what happens after.
●  We don’t know that the shareholder may want to use the money to
buy a car or pay rent; Webmaster may be trying to get new
sponsors, or raise the rates on existing sponsors.

@xbosoft @philiplew

User Stories Cannot Be Lonely
•  Description of an objective that a user should be
able to achieve, or a feature that a user should be
able to utilize, when using a software application.
– As an Administrator, I want to be able to create User
Accounts so that I can grant users access to the system.
•  User Story cannot stand alone.
•  Must be accompanied by Acceptance Criteria with
Acceptance Tests.

ACCEPTANCE CRITERIA
Is the User Story Done or not Done? Pass or Fail?
@xbosoft @philiplew

User Stories and Acceptance Tests
Peas in a Pod
•  User stories MUST be coupled with acceptance tests to ensure
traceability.
– "We write the acceptance tests with the requirements and
keep them with story card for the lifecycle of the
requirements.”
– “Requirements are what you’re building and Acceptance
tests help you measure what you built.”
– “Acceptance test is the condition which your customer will
accept the product you built. ”

Acceptance Criteria Should Include
•  Functional Criteria: Identify specific user tasks, functions or
business processes that must be in place. Example:
–  “A user is able to access a list of available reports.”
•  Non-functional Criteria: Identify specific non-functional conditions
the implementation must meet, such as design elements. Example:
–  “Edit buttons and Workflow buttons comply with the Site Button Design.”
•  Performance Criteria: Sometimes performance is critical to the
acceptance of a user story and should therefore be included.
Example:
–  Measured as a response time, spelled out as a threshold such as within 2
seconds.

Acceptance Criteria
•  Set of statements, each with a clear pass/fail result, that
specify both functional (e.g., minimal marketable
functionality) and non-functional (e.g., minimal quality)
requirements applicable at the current stage of the project.
•  Represent “conditions of satisfaction” with no partial
acceptance: Yes or No, Met or Not or Not Met
•  Define the boundaries and parameters of a User Story/feature
and determine when a story is completed and working as
expected.
•  Purpose is to add certainty to what the team is building.

Acceptance Criteria
•  In simple language that customer would use without
ambiguity as to what the expected outcome is:
– what is acceptable and what is not acceptable.
•  Testable: easily translated into one or more test
cases.
•  May reference other User Stories or design
documents to provide details, but should not repeat.
•  Relatively high-level yet enough detail to be useful.

Acceptance Criteria
•  Acceptance Criteria should state intent, but not a
solution
– “A manager can approve or disapprove an audit form”
– “A manager can click an ‘Approve/Disapprove’ radio
button to approve an audit form”
•  Criteria should be independent of the implementation
and platform

●  Start with basic examples, then expand through exploration.
●  Use “Given - when - then” language to give the tests a consistent form.
●  Stay focused on the story - avoid expanding the scope.
●  Functional Acceptance Tests vs User Acceptance Testing (UAT)
○  Functional testing means the implementation meets the letter of the
requirements.
○  UAT is more about the intent of the requirements, and how it might
work in conjunction with the rest of the system.
●  Both functional testing and UAT can be included in the Definition of Done.
“A good specification with examples is effectively an acceptance
test for the described functionality.” - Gojko Adzic
Writing Acceptance Criteria
@xbosoft @philiplew

User Acceptance Criteria
As the treating doctor I want to remove a prescription from the lookup list
because the drug is discontinued so that my staff doesn't select it by
accident.
●  Given an existing reference reference prescription, when it is
removed then it shall no longer appear in the lookup list
●  Deleting reference prescriptions shall not affect patient prescriptions
●  Given two similar references prescriptions, “aspirin” and “aspirin
50mg”, when “aspirin” is deleted it shall not affect the similar
prescription “aspirin 50mg.”
●  Staff shall be able to subsequently create a new reference.
prescription with the same information.
@xbosoft @philiplew

User Story à Acceptance Criteria
User Story
•  As an Administrator, I want to be able to create User Accounts so that I can grant
users access to the system.
Acceptance Criteria
•  If I am an Administrator, I can create User Accounts.
•  I can create a User Account by entering the following information about the User:
a. Name, b. Email address, c. Phone Number d. Reports to (from a list of “Active”
Users)
•  The system notifies me that it sent an email to the new User’s email address,
containing a system-generated initial password and instructions for the person to
log in and change their password.
•  I am able to verify with the intended recipient of the email that it was received.
•  Exercise: Change these into the proper language
•  What else should be in the acceptance criteria?

User Story à Acceptance Criteria
User Story
•  As an Administrator, I want to be able to create User Accounts so
that I can grant users access to the system.
Acceptance Criteria (What can’t be done!)
•  I cannot assign a new User to report to an “Inactive” User
•  I cannot assign a new User to report to a User if it creates a cyclical
relationship (e.g., User 1 reports to User 2 who reports to User 1
•  What else should not be able to be done?

Agile Requirements/Acceptance
Model Example
Team defines
acceptance criteria
to exit the sprint.
____ ties
acceptance tests
to criteria.
Acceptance testing
is done by ______.
Acceptance by
Whom?
•  User stories determined for the iteration.
•  User stories have associated acceptance criteria.
•  Acceptance criteria have associated tests.
•  In addition to the product owner, “Acceptance by whom”
sometimes is just as important as the word “testing”.

Agile Acceptance
Acceptance By Whom?
•  Importance of the Product Owner
•  Product owner puts User into Acceptance Testing
– Must show up
– Must be clear on criteria
– Must be clear on acceptance tests tied to criteria
•  Product owner pulls others that represent the end user/
customer into the conversation
•  Who could others be?

Put the User in Acceptance Testing
•  In an agile Agile team with high levels of collaboration, we can
forget that ultimately, its the customer or user that will decide
if what was built is actually wanted and therefore used.
– Best practice : Include customer focus group or customer
panels in creating acceptance criteria and tests
•  Acceptance criteria and tests are not just created by the
Quality person on the team.
– Anyone that has the voice of the customer in mind can do it.

CONNECTING USER STORIES WITH CRITERIA
AND TESTS
@xbosoft @philiplew

User Stories with Tests
•  Test description (test objective)
– Acceptance Criteria
•  Test steps
•  Expected results
•  Associated requirement (User Story) for traceability
•  Test Plan – Execution History

One More Thing on Acceptance Tests
•  Reusable and independent whenever possible
•  Design from the start with reusability and automation in mind
for the sustainability of the product
– (white box-static, API and UI)
“We don't have automated acceptance tests which would
be nice. At one time I was working toward that, but that
was deprioritized by management.”
@xbosoft @philiplew

PLANNING TESTS FROM YOUR REQUIREMENT
(USER STORY)
@xbosoft @philiplew

Managing Requirements Versions
•  User Stories are always changing!
•  Track what happens between versions

MANAGING REQUIREMENTS BASELINE
@xbosoft @philiplew

Managing Requirements Baseline
•  Decide for a set of requirements to be implemented in
a specific sprint
•  You don’t want the change in requirements for future
to influence the current development iteration
•  So you snapshot the set of requirements for developers
to work on while product managers or product owners
may be refining or changing the story.

Agile Requirements
•  Requirements are negotiable by the team during spring
planning
– Team breaks the requirements into tasks
– Team can negotiate decomposing large stories into smaller ones
•  During the project you generally stick to quality and timeline
as the two fixed points, and features are the flexible post of
the project.
– Product owner / customer / team work together to negotiate quality
and timeline

Managing Requirements
Keep It Short Term (accurate)
•  Agile requirements need an agile process
•  The refining process happens regularly and accurately
• Backlog not refined too far out (2-3 sprints)
• Stories with Acceptance criteria not too far out (less than 3
sprints, maybe even current sprint)
•  People don't remember discussions after about 2 or 3 weeks.
•  Anything agreed to in a big spec, or future story far out is lost –
forgotten
•  Hassle to maintain long specs over time. You have to keep
returning to the spec & tweaking it, etc.

TRACEABILITY - USER STORIES WITH TESTS AND
THEN DEFECTS
@xbosoft @philiplew

Traceability
• Create tests from requirements (user stories)
• Track defects from requirements
• Find covered requirements from test cases
• Find impacted requirements from defects
• Traceability matrix report

TEST PLANNING – ENSURING THAT USER
STORIES ARE TESTED
@xbosoft @philiplew

Test Planning
• Create a test plan
• Input requirements (user stories) into the test plan
• Test assignment and execution
• Report defects if found

TAKEAWAYS
Let’s Sum up

Putting UAT into Agile
Use today’s concepts to transform your Agile process:
1.  Put the User back into Acceptance Testing
• Get users involved in developing user stories and criteria
• Clearly define Product Owner’s Role
2.  Put “acceptance” back into Acceptance Testing
• Write clear acceptance criteria that are yes/no pass/fail
3.  Put “testing” back into Acceptance Testing
•  Testing means more than verification

•  Agile UAT requires agile people:
– Product owner has the final answer on priorities
– Don’t plan too far out
– Multiple people reviewing / refining stories (product
owner + QA + dev) and updated
• QA makes the stories testable before any dev occurs
• QA can help ID unknowns in stories / tasks
UAT in Agile Requires Agility

Implementing Your Agile UAT
Cheney Ma
cma@go2group.com
@xbosoft @philiplew

Post your webinar questions on Twitter @XBOSoft
Registrants will receive an email with information on where to view the recording and slides from today’s
webinar.
Join us to keep updated on all our webinars, reports and white papers:
facebook.com/xbosoft
+xbosoft
linkedin.com/company/xbosoft
Check out our blog: http://xbosoft.com/software-quality-blog/
Download our free white papers: https://xbosoft.com/resources/white-papers/
Email us with ideas for future webinars or questions regarding our services! services@xbosoft.com
Q&A
www.xbosoft.com
@xbosoft @philiplew

Agile User Acceptance Testing - Incorporating UAT into Agile

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Agile User Acceptance Testing - Incorporating UAT into Agile

Similar a Agile User Acceptance Testing - Incorporating UAT into Agile (20)

Más de XBOSoft

Más de XBOSoft (20)

Último

Último (20)

Agile User Acceptance Testing - Incorporating UAT into Agile