ATDD - Acceptance Test Driven Development

Acceptance Test
Driven Development
Naresh Jain
naresh@agilefaqs.com
http://blogs.agilefaqs.com

Licensed Under Creative Commons by Naresh Jain
1

Warmup Scenarios

2

Warmup Scenarios
Pick one scenario and in relation to your scenario,
what are the speciﬁc observable results that will
tell you that the activity has been successfully
completed?

2

Warmup Scenarios
completed?
Going out for Movie (THX sound and Digital projection)

2

Warmup Scenarios
completed?
Going out for meal (one veg.)

2

Warmup Scenarios
completed?
Going shopping ($50)

2

Warmup Scenarios
completed?
[10 Minutes]

2

Warmup Scenarios
completed?
[10 Minutes]

Present back to the group your ﬁndings. [3 minutes per group]

2

What is a Story?
Story is a smallest piece of functionality that add business value

Story Title - Actor Action Context

As a .. <user who requires this feature>
I want .. <do something>
So that... <user goal/business justiﬁcation>

Ron Jeffries’ 3 Cs - Card, Conversation and Conﬁrmation

3

Story Example

Title: Keen Reader subscribes to a blog
As a keen reader of your blog
I want to subscribe to your blog
So that I can stay up-to-date with the new posts

4

Another Story Example

Title: Social Networking Enthusiast uploads proﬁle picture
As a Social Networking Enthusiast
I want to upload my proﬁle picture
So my friends can see how I look and recognize me

5

What makes a good Story?

6


Stories should follow the INVEST principle:

6


Independent

6


Independent
Negotiable

6


Independent
Negotiable
Valuable

6


Independent
Negotiable
Valuable
Estimate-able

6


Independent
Negotiable
Valuable
Estimate-able
Small

6


Independent
Negotiable
Valuable
Estimate-able
Small
Testable

6

Stories are fundamental unit of activity

7


Business Goals

7


Business Goals

Inception

7


Business Goals Product Backlog

Inception

As a ____, I want to
be able to ____ so
that ____

Might have an initial estimate
(perhaps for both analysis
and development), and an
expression of technical and
business confidence that this
is real and achievable

7


Business Goals Product Backlog

Inception
Release planning

As a ____, I want to
be able to ____ so
that ____

is real and achievable

7


Business Goals Product Backlog Release Backlog

Inception
Release planning

As a ____, I want to As a ____, I want to
be able to ____ so be able to ____ so
that ____ that ____

I will know this is done
when _______
is real and achievable More detailed estimate, and a
specific acceptance test – low
confidence stories might be
“spiked” or prototyped

7


Business Goals Product Backlog Release Backlog

Inception
Release planning Sprint planning

As a ____, I want to As a ____, I want to
be able to ____ so be able to ____ so
that ____ that ____

I will know this is done
when _______
is real and achievable More detailed estimate, and a

7


Business Goals Product Backlog Release Backlog Sprint Backlog

Inception
Release planning Sprint planning

As a ____, I want to As a ____, I want to As a ____, I want to
be able to ____ so be able to ____ so be able to ____ so
that ____ that ____ that ____
Possible automation
of the acceptance
test

I will know this is done I will know this is done
when _______ when _______
and development), and an Development team
expression of technical and breaks out the detail
To do this I must:
business confidence that this of work needed to
1) _____
is real and achievable More detailed estimate, and a pass test
2) _____

7

Acceptance Criteria

Is a set of conditions that the Story must meet for it to be
accepted as complete
Is typically provided by the customer or product owner.

Is not a replacement for conversation.
Is the results of the conversation

Acceptance Criteria are NOT tests

8

Writing Acceptance Criteria

Acceptance Criteria should contain:
ACTOR
VERB – DESCRIBING A BEHAVIOR
OBSERVABLE RESULT

To accommodate pre-conditions Acceptance Criteria can be expressed as

Given [Precondition]
When [Actor + Action]
Then [Observable Result]

9

Example
Social Networking Enthusiast uploads proﬁle picture

10

Example
Given the user has a valid facebook account and a digital picture on her computer,
When she uploads a picture in facebook,
Then her the picture should be visible to all her friends in her network.

10

Example

Given an user is trying to ﬁnd a friend on facebook,
When the user searches for a person using their name,
Then their proﬁle picture should be displayed along with other details.

10

Example

Given an user is trying to find a friend on facebook,
When the user searches for a person using their name,
Then their profile picture should be displayed along with other details.

As owner of facebook,
I want users to upload authentic, personal profile picture,
So facebook's reputation remains intact and facebook stays out of legal hassles.

10

Acceptance Criteria & Tests: Deﬁnition

Acceptance Tests

11


Acceptance Tests

Acceptance Criteria

11


Acceptance Tests

Acceptance Criteria
+

11


Acceptance Tests

Acceptance Criteria
+ Examples (data + scenarios)

11


Acceptance Tests

Acceptance Criteria
+ Examples (data + scenarios)

Acceptance Tests
11

Tasks
Team members further break down each story into tasks that need
to be completed to meet the acceptance criteria for the story.

12

Tasks

To accomplish this story:
we start off with a simple upload and image display

12

Tasks

restrict user to only upload certain image types (gif, jpg and png)

12

Tasks

ﬁgure out where to store the image. (performant and fault-tolarent)

12

Tasks

scale down (size, resolution, etc.) of the image

12

Tasks

scale down (size, resolution, etc.) of the image
and so on...

12

Demo
Roman Numerals to Decimal Conversion Example

13

Demo
Real World Domain Forwarding Server

14

Thinking in Tables

15

Thinking in Tables

Only Tables Execute

Ignored

Executed

16

Thinking in Tables

Foundational Table Structure
Name of Fixture

Interaction with Application

Table structure depends on type of Fixture

17

Thinking in Tables

3 Foundation Fixtures

Column Fixture
Row Fixture
Action Fixture

18

Thinking in Tables

Column Fixture

19

Thinking in Tables

Row Fixture
Analogous to comparing
against rows in a
database table

20

Thinking in Tables

Action Fixture
Think GUI window Counter Window
Counter:
public class CountFixture extends Fixture {
private int counter = 0; Counter: 6

public void count() { Count
counter++;
}

public int counter() {
return counter;
}
}

21

FitLibrary

Extension to FIT
Written by Rick Mugridge
Adds some handy Fixtures

22

FitLibrary

FitLibrary Fixtures

ArrayFixture for ordered lists
SetFixture for unordered lists
SetUpFixture
Supports
Graphics
Tree structures
Nested Tables

23

FitLibrary

DoFixture

• Broken tables

• Highly readable

• Flexibility

24

Tools

FIT

Framework for Integrated Tests
Created by Ward Cunningham
Open Source
The most accepted solution for agile acceptance testing

25

Tools

FitNesse

Environment build around FIT
Makes everything easier
Created by Object Mentor, Inc.
Open Source

26

FIT FitNesse
- Stand alone web server
- Is a wiki
- Tests written in HTML
- Tests written in wiki text
- Tests are executed on the command line
- Tests are executed from within the wiki
- Tables are executed
- Translates tests into HTML
- Non-table markup is ignored
- Uses FIT to execute tests
- Tables map to Fixtures
- Supports test suites
- Fixtures are code that is aware of the
- Supports variables in tests
system
- Supports test refactoring
- Supplies foundational Fixtures
- Written in Java
- Implementations ported to many
- Supports FIT implementations in any
languages
language

27

Acceptance Criteria
and Tests:
A Critical Piece of Agile

28

Acceptance Criteria & Tests: A Critical Piece of Agile

Traditional Approach

1 May 1 Jul 1 Sep 1 Nov

Analysis

Design

Implementation
ERD
DFD
DD
ST

29

Key Questions
Business Facing

Are we building the right product?

Are we building the product right?

Technology/Implementation Facing


30

Brian Marick’s Test Categorization
Business Facing
Supports Programming

Critique product


31

It Helps to Think of Tests this way...
Business Facing
Drives Development

Critique product


32

Business Facing
Drives Development

Critique product
Unit Testing



32

Business Facing

Acceptance Testing
Drives Development

Critique product
Low-ﬁ prototypes

Unit Testing



32

Business Facing

Acceptance Testing Exploratory Testing
Drives Development

Critique product
Low-ﬁ prototypes UI and Usability Testing

Unit Testing



32

Business Facing

Acceptance Testing Exploratory Testing
Drives Development

Critique product
Low-ﬁ prototypes UI and Usability Testing

Performance Testing
Unit Testing
System Tests


32

Avatars of TDD
Business Facing
Drives Development

Critique product
Inside Out

Outside In


33

Test Driven Development

TDD Rhythm - Test, Code, Refactor

34

Add a Test


34

Add a Test

Run the Test


34

Add a Test

Pass
Run the Test


34

Add a Test

Pass
Run the Test
Fail

34

Add a Test

Pass
Run the Test
Fail
TDD Rhythm - Test, Code, Refactor Make a little
change

34

Add a Test

Pass
Run the Test
Fail
change

Run the Test

34

Add a Test

Pass
Run the Test
Fail
change

Fail
Run the Test

34

Add a Test

Pass
Run the Test
Fail
change

Fail
Run the Test
Pass

34

Add a Test

Pass
Run the Test
Fail
change

Fail
Run the Test
Pass
Refactor

34

Acceptance Test Driven Development

35


Story

35


Acceptance
Criteria

Story

35

Iteration

Acceptance
Criteria

Story

35

Iteration
Automated
Acceptance Acceptance
Criteria Tests

Story

35

Iteration
Automated
Criteria Tests

Story Automated
Unit Test

35

Iteration
Automated
Criteria Tests

Story Automated
Unit Test

Automated
Acceptance
Tests

35

Iteration
Automated
Criteria Tests

Story Automated
Unit Test

Automated
Acceptance
Tests

Exploratory
Testing

35

Iteration
Automated
Criteria Tests

Story Automated
Unit Test

Automated
Acceptance
Tests
Acceptance
Exploratory
Criteria
Testing

35

Iteration
Automated
Criteria Tests

Story Automated
Unit Test Automated
UI Tests

Automated
Acceptance
Tests
Acceptance
Exploratory
Criteria
Testing

35

Iteration
Automated P
Acceptance E
Acceptance R
Criteria Tests
F
O
Automated R
Story M T
Unit Test Automated E E
UI Tests N S
C T
E S
Automated
Acceptance
Tests
Acceptance
Exploratory
Criteria
Testing

35

Mike Cohn’s Testing Pyramid

36

GUI
Tests

36

Small in Number
GUI Tools: Selenium, Sahi, Watir, Abbot, Frankenstein
Tests

36

Small in Number
Tests

Acceptance
Tests

36

Small in Number
Tests
At least one per story
Acceptance Tools: Fit, FitNesse, RSpec, JBehave
Tests

36

Small in Number
Tests
Tests

Unit Tests

36

Small in Number
Tests
Tests

At least one per class or module
Unit Tests Tools: xUnit, TestNG

36

Acceptance Tests: A Critical Piece of Agile

Criteria for DONE

Every story must have at least one Acceptance Test
A story is not DONE until it passes it’s Acceptance Tests

37


Manual Acceptance Tests

38


Manual Acceptance Tests

Manual
Acceptance Tests

38

Why Acceptance Tests?

Criteria for Completion
Great Collaboration tool
Source of Feedback
Real data to measure progress

39

Data From Acceptance Tests
Total ATs Failing ATs Passing ATs

90

72

54

36

18

0
1 2 3 4 5 6 7 8 9 10

40

Acceptance Tests Are
Automated

41


The Button

How often would you press
it?
When would you press it?
Who would press it?
Testers, Developers,
Managers, Customers,
Spectators, etc.

42

43

Criteria for DONE

43

Criteria for DONE
+

43

Criteria for DONE
+ Automated

43

Criteria for DONE
+ Automated

Executable Speciﬁcation

43


Executable Speciﬁcation

A new paradigm for testing
Puts quality ﬁrst
Removes ambiguity from requirements

44

Who Writes
Acceptance Tests?

45

Who Writes Acceptance Tests?

The Customer

The Customer Role
Stake holder
Business Analyst
Quality Assurance
Product Owner
Developer

46


Tests Get Technical

The “Customer” may need technical help to write tests
Developers and QAs are technical
Pair test authoring

47


Business Rules Get Fuzzy

Sometimes developers need help understanding tests
Customers know business rules
Pair test implementation

48

Exercise #1

49

Exercise #1

The Login Test

Write a test plan, in plain text, for the business
rules of logging in.
Web application
User credentials are stored in relational
database
Successful login redirects to “Welcome” page

50

Writing Good
Acceptance Tests

51

Writing Good Acceptance Tests

Login Test Possibilities

1. Direct browser to URL for login page

52



1. Enter the username ‘wallace’

52



1. Enter the username ‘wallace’

Build a Testable Environment First

52



1. Add some users to the system

53



3. Enter a value into the username ﬁeld

53



3. Enter a value into the username ﬁeld

Be Speciﬁc

53


Tests are Examples

Use concrete examples
Specify concrete behavior
No ambiguity allowed

54



1. Insert into User table values (’wallace’,
‘ilikecheeze’)

55



‘ilikecheeze’)
2. Open a browser to the URL http://
localhost/myapp

55



‘ilikecheeze’)
2. Open a browser to the URL http://
localhost/myapp

Avoid Implementation Details

55

Good Acceptable Criteria and Tests

56


S

- SPECIFIC - Explicitly deﬁned and deﬁnite

56


S

M

- MEASURABLE - Possible to observe and quantify

56


S

M

A

- ACHIEVABLE - Capable of existing or taking place

56


S

M

A

R

- RELEVANT - Having a connection with the story

56


S

M

A

R

- RELEVANT - Having a connection with the story
T

- TIME-BOUND – When will the outcome be observed
56


Avoid Implementation Details
Acceptance Tests View

UI
Model and Presenter

Business Tier

Data Store
57


Login Test: Possible Solution

Add user to system: (’wallace’, ‘ilikecheeze’)
Process login with username ‘wallace’ and password ‘blah’
Check login failed
Process login with username ‘wallace’ and password ‘ilikecheeze’
Check login succeeded

58

Tools

59

Tools

Commercial Tools

WinRunner TestPartner EggPlant
Silk QTP TestComplete
RFT
Squish WindowTester

Are not suitable for Acceptance Testing in
an Agile environment

60

Tools

Open Source Options

FIT Sahi Frankenstein
FitNesse Watir Cucumber
Selenium Abbot RSpec/JBehave

Among the few tools that support Test
Driven Development

61

Wiki

62

Wiki

What is it?

A collaborative web site
Editable by any
Created by Ward Cunningham
Every project should have one
http://c2.com/wiki
http://en.wikipedia.com

63

Wiki

Creating Tests
Use Wiki syntax to create a page with test tables
Label the page as a Test Page
Use a page name of the form Test…
Turn on the Test property
Make sure your Fixtures are in the classpath
Use !path widget
Mechanics
!path values are concatenated
Java command to start FitServer is executed
Testable HTML is passed to FitServer
FitServer runs the tests
Results are passed back to FitNesse

64

Wiki

Creating Suites
There are 2 ways to make Suites
Set the Suite property
Create a page with the Suite property
Created test pages inside this page
When the suite is executed, all child test pages will
be included in the suite execution
Use the !see widget
!see <name of test page>
All “included” tests pages will be included in the
suite execution
Run a Suite by clicking the Suite button

65

Hands-on Session

Conference Proposal Submission Portal
Some sample Stories
Should be able to submit new proposal
Should be able to list all submitted proposal
Submitting proposal with same title should display appropriate error message
Should be able to delete submitted proposal based on the title
Should be able to delete submitted proposal based on the title
Should be able to search proposals by title
Should be able to search proposals by ID
Should be able to find all proposal by an author's name

66

Break

67

Patterns

68

Patterns

Organizing Tests

Allowing customers to add new tests without breaking the build

69

Patterns

Version Control

Keeping the acceptance test in version control with the code.

70

Patterns

Cross-Functional Pairing

Using FitNesse based acceptance tests for collaboration between cross-
functional team members.

71

Patterns

ATDD


72

Patterns

CSTT

Cleanup, Setup, Test, Teardown

73

Patterns

Independent Tests

Tests shouldn’t depend on each other.
Tests leave the system in the same state it started in.

74

Patterns

Dynamic Stubbing

Avoiding complications of external systems.

75

Patterns

Non-Production Setup/Teardown

Using non-production light weigh code for setup and teardown.
Helps test only what you want to test.

76

Patterns

Suite Levels

Creating different levels of suites depending on the depth/level of
feedback desired.
Smoke, Current Iteration/Sprint, Regression

77

Patterns

DRY

Using !include to avoid repeating yourself.

78

Patterns

Make it Real

Write ATs as close as possible to the real environment.

79

Patterns

Fixture Evolution

Allow Fixture implementation to evolve over time.
Treat ﬁxtures as ﬁrst class citizens.

80

Patterns

At Least One Test/Story

Every story should have at least one acceptance test
Avoid long/multipurpose tests.

81

Anti-Patterns

82

Anti-Patterns

Developer ATs

Developers writing acceptance tests by themselves, for themselves.

83

Anti-Patterns

Unit Testing

Don’t write ATs at the unit testing level
Unit tests are implementation speciﬁc
ATs are NOT implementation speciﬁc

84

Anti-Patterns

QA Testing Tool

Hard to write tests up front.
Perhaps only on large projects.

85

Anti-Patterns

Silver Bullet

Trying to use FitNesse for all types of Acceptance Tests
UI testing
XML testing

86

Anti-Patterns

Test After

Writing tests after the code is already written.

87

Anti-Patterns

Hidden Test Data

Hiding test data in the ﬁxtures.

88

Anti-Patterns
Implementation Dependent ATs

Making test pages (tables) dependent on implementation details and
data structures.

89

Anti-Patterns

Logging in Your Fixtures

Putting log statements or print statements in the ﬁxture code.
Fixtures are probably too complicated.

90

Reference

Portions of this presentation is adopted from Micah Martin’s
Introduction to Automated Acceptance Tests Presentation
Kent Beck, Test Driven Development By Example.
"Agile Testing Directions" - Brian Marick

http://www.opensourcetesting.org/

91

The End

Questions?

naresh@agilefaqs.com
http://blogs.agilefaqs.com

92

ATDD - Acceptance Test Driven Development

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (20)

Similar a ATDD - Acceptance Test Driven Development

Similar a ATDD - Acceptance Test Driven Development (20)

Más de Naresh Jain

Más de Naresh Jain (20)

Último

Último (20)

ATDD - Acceptance Test Driven Development