This document discusses acceptance test-driven development (ATDD) and provides a case study of implementing ATDD for a website acceptance testing project. It describes the goals of ATDD as crystallizing acceptance criteria and early regression detection. It then outlines the steps taken in the case study, including using Jenkins for continuous integration, Selenium for browser automation, and the Robot Framework with a Given-When-Then approach for tests. Potential issues and benefits are discussed, such as enabling liberal refactoring but requiring a different mindset and skillset to address intermittency.

1. Website
acceptance
testing
–
a
case
study
Eemeli
Kantola
10.11.2012

2. › Eemeli
Kantola
› Web
dev,
consulting,
About
me
lean/agile
enthusiasm
› Customer
projects
in
Futurice

3. I
like
to
create
awesome
web
apps
for
(human)
end
users

4. ATDD

5. ATDD:
“The
practice
of
expressing
functional
story
requirements
as
concrete
examples
or
expectations
prior
to
story
development”
–
Jennitta
Andrea,
http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectId=16315

6. › Crystallize
acceptance
criteria:
shared
understanding
of
reqs
and
def.
of
done
[1]
Goals
› Win
bug
battle
with
complicated
systems
[2]:
early
regression
detection
[1]
E.
Hendrickson,
http://testobsessed.com/blog/2008/12/08/acceptance-­‐test-­‐driven-­‐development-­‐atdd-­‐an-­‐overview
[2]
J.
E.
Boal,
http://testdrivendeveloper.com/2011/12/27/AutomatedTestsAreTheKeyToScalingAgile.aspx

7. Steps
Implement:
hammer
the
Design
how
to
fulfill
keyboard
until
all
tests
the
requirement
pass
(incl.
other
tests)
Write
Story
Design
Code
tests
Find
out
most
important
Formulate
test
cases,
thing
to
do,
write
a
user
story
most
probable
path
first
10.11.2012
Futurice
34

8. Case
study

9. Steps
in
real
life

10. Jenkins
CI
» Open-­‐source
continuous
integration
server
» Java
based,
lotsa
plugins
and
stuff
» Distributed
test
running
» Getting
started:
download
.war
and
fire
up
10.11.2012
Futurice
37

11. Selenium
» Automation
of
browsers
› E.g.
for
testing
purposes,
or
for
repetitive
admin
tasks
» Selenium
IDE
for
recording
scripts
(=playing
ariound)
» Scaling
and
distributing
test
running
» Java-­‐based,
with
wide
support
for
other
langs
&
testing
frameworks
10.11.2012
Futurice
38

12. Robot
Framework
» Generic
test
automation
framework
for
various
domains
› Web
part
backed
by
Selenium,
but
also
Java
GUI,
SSH,
running
processes...
» Tabular
test
data
syntax
» Extensible
with
Python/Java
10.11.2012
Futurice
39

13. Given-­‐When-­‐Then
(GWT)
Example
of
a
popular
approach:
» Given
the
shopping
cart
is
visible
and
has
three
items
in
it,
» When
the
user
clicks
an
item’s
trashcan
icon,
» Then
the
item
should
disappear
from
the
list
and
a
notification
should
be
shown
with
the
item’s
name
and
an
“Undo”
link.
10.11.2012
Futurice
40

14. Demo

15. Is
this
worth
all
› Yes!
No
time
not
to
this
hassle?
automate
testing

16. › Fairly
liberal
refactoring
› Finding
regression
bugs
long
before
going
to
prod
› TDD
=>
knowing
early
what
Enables:
you
are
supposed
to
code
› And
writing
less
useless
code
› Taking
in
new
devs
easily
› Continuous
deployment?
› …but
not
quite
yet

17. › Different
mindset
required
› Intermittent,
hard
to
reproduce
test
failures
Issues?
Sure:
› Asynchronity,
other
weirdness
› Requires
lotsa
mad
skillz
› …and
then
some

18. Thanks!
@ekantola
github.com/ekantola