This document discusses using specification by example (SBE) to test rich client GUI applications. It describes using a tool called TextTest that allows writing tests using a domain language and automatically records the GUI interactions and assertions. Tests in TextTest have two parts - a use case section describing actions in domain language terms, and an automatically generated GUI log section capturing screen contents. This allows testing applications by their specifications before code is written and preserves requirements as living documentation through automated regression tests.
7. Specification by Example
Gojko Adzic’s book
Collaborative workshops
Define Example Scenarios BEFORE
build code
7
8. Agile Requirements
User Story + Conversation + Examples
= Requirement
From “Agile Testing” by Crispin, Gregory
8
aka Scenarios
9. Scenario -> Regression test
BEFORE code is written, it’s a requirement
specification
AFTER code is written it’s a regression test
Important to preserve the language & format
9
Living Documentation
11. Recorded GUI Tests
Click around the GUI like a user
would
Record a script
Re-play the script to regression
test
11
Assumes the GUI is already built
21. Open Source
Development paid for by Jeppesen
supports Java Swing, Eclipse RCP, PyGTK, Tkinter, wxPython
http://texttest.org
21
22. TextTest tests have two parts
Use Case GUI Log
List of user actions:
Domain language
Recorded text:
Contents of GUI
22
23. TextTest - test fixtures
Create “use case” part of text by Record session in UI
Name your actions: snippets of domain language
UI Map - domain language mapped to actions
23
24. Compose a Domain Language
24
Group several user actions and name them
Match the language of the specification workshop
Re-usable terms for many tests
25. Record the Test Fixture/Step definitions
Use Case
(one per test)
“UI Map” + “shortcuts”
(shared by whole suite)
System Under Test
25
Recorded
Configured
26. StoryText tests
Use Case GUI Log
List of user actions:
Domain language
Recorded text:
Contents of GUI
26
27. The GUI log
“Assert” part of the test
Auto-generated by
StoryText
ASCII art represents UI
state
27
28. StoryText tests
Use Case GUI Log
Plain Text: search, store, manipulate, diff, version...
28
30. Sample Use-Case :
Undo Service Type changes
30
open default plan and show rosters
change service type of activity JPO 2537 for
Brogan to M
undo
wait for completion of undo
close and discard changes
32. Assert Everything by default
Log of absolutely everything that changes in the GUI
Each test defines filters to focus on interesting parts
Often catch errors the test wasn’t designed to find
32
33. Limitations of the approach
Only supports certain GUI libraries
Maintainability of tests depends on good test
design
◦ filter GUI log carefully
33
34. Conclusions
Specification by Example - automating Domain
Language scenarios needs fixture code
◦ TextTest “records” the fixtures
GUI intensive applications - need to check
carefully what’s on the screen
◦ TextTest takes “screen shots” as ASCII art
34