fixture-free automated acceptance test

1. Fixture-Free Automated
Acceptance Tests
Mike Stockdale
©2010 Syterra Software Inc.
Much Ado About Agile V

4. http://commons.wikimedia.org/wiki/File:Start_of_a_dugout_canoe_03.jpg

5. http://layered.typepad.com/antidote_to_burnout/2006/08/they_still_buil.html

7. Requirement
Detailed
Design
Unit Test
Code
System
Design
System Test
Acceptance
Test
SDLC V Model

8. http://www.natpryce.com/articles/000780.html

9. Feedback
• Check behavior
• Positive design pressure

10. http://twitter.com/#!/s_bergmann/status/29008066042

11. Requirement
Detailed
Design
Unit Test
Code
System
Design
System Test
Acceptance
Test
SDLC V Model

12. Agenda
• Agile testing beyond TDD
• Reality check: what are you doing?
• Automating user stories
• Demo
• Questions and Comments

13. A Small Slice
•Micro-test: 20 – 30 minutes
• User story: 2 -3 days

14. A picture example is worth a 1000 words

15. Some examples are better than others
http://www.flickr.com/photos/rampix/3258014594/

16. Example
Requirement
Test
elaborates
confirms
executes
AA-FTT 2008 Open Space

17. Micro (unit) test
Story test
Exploratory test
End-to-end test
Smoke test
Performance test
Security test
Regression test

18. Tests beyond TDD
• What tests does your team perform?
• Are they automated?
• Who writes, runs, maintains them?
• When are they written, run, maintained?
• What are your good and bad experiences?

19. One Approach
• Micro tests + exploratory tests
• Add regression tests as required
• All automation in one framework

20. Story test automation
• Change the cost/benefit

21. High risk
but
hard to automate?
http://www.flickr.com/photos/koalie/1699195578/

22. to reliably test manage a system, we
must be able to…
• determine what the system is doing.
• detect when it has stopped doing it.
• detect when the system has failed.
• explain why has it failed.
• restore to a known good state.
http://www.natpryce.com/articles/000780.html

23. Improved Visibility
http://www.flickr.com/photos/kevinsteele/28503911

24. Example
Requirement
Test
elaborates
confirms
executes
AA-FTT 2008 Open Space

25. Caveat: User story ≠ story test
• may change existing story tests
• may not be automated
• don’t force-fit into story test language

26. No single work group at any work step
should remove user stories to a
medium that, while more expedient to
their work, causes user stories to
become less accessible by others with
different skills and responsibilities.
http://blog.scottbellware.com/2010/10/user-stories-belong-to-everyone.html

27. http://twitter.com/#!/dhemery/status/22638188913

28. FixtureFramework
SUT
Host language

29. FixtureFramework
SUT
Host language
Story test

30. Fixture
Framework
SUT
Host language
Story test

31. Plus
Check 2 plus 3 is 5
PlusFixture:
return plus(x, x + y)
plus:
return y

32. Transparency
• Ubiquitous language
• Domain layer

33. • Ownership
• Cadence, flow
http://www.flickr.com/photos/kevinsteele/28503911

34. Caveat:
Story test ≠
General-purpose language

35. Scenario: User creates some sites and circuits, check connected sites list
Given a "site" exists with {"name"=>"Somewhere1", "identifier" => "TER1", "provider"=>"TER1 Provider"}
And a "site" exists with {"name"=>"Somewhere2", "identifier" => "TER2", "provider"=>"Some Provider"}
And a "site" exists with {"name"=>"Somewhere3", "identifier" => "TER3", "provider"=>"TER3 Provider"}
And a "circuit" exists with {"provider_name"=>"Another provider", "redacted_circuit_id"=>"ABC1",
"provider_circuit_id"=>"C1", "circuit_type"=>CircuitType.find_by_name("Peering"),
"service_type"=>CircuitServiceType.find_by_name("Dark Fiber"),
:capacity => CircuitCapacity.find_by_name("1 Gbps"),
"physical_wire_type"=>PhysicalWireType.find_by_name("Multi Mode Fiber"),
"status"=> CircuitStatus.find_by_name("Cancelled"), "a_end"=>Site.find_by_identifier("TER1"),
"b_end"=>Site.find_by_identifier("TER2")}
And a "circuit" exists with {"provider_name"=>"Switch and Data",
"redacted_circuit_id"=>"ABC2", "provider_circuit_id"=>"C2",
"circuit_type"=>CircuitType.find_by_name("Backbone"),
"service_type"=>CircuitServiceType.find_by_name("Dark Fiber"),
:capacity => CircuitCapacity.find_by_name("1 Gbps"),
"physical_wire_type"=>PhysicalWireType.find_by_name("Multi Mode Fiber"),
"status"=> CircuitStatus.find_by_name("Cancelled"),
"a_end"=>Site.find_by_identifier("TER1"), "b_end"=>Site.find_by_identifier("TER3")}
When I am on the "connected_sites" page for site "TER1"
Then the "connected-sites-list" should look like
| Site ID | Site Name | Site Provider | Provider Circuit ID | Provider Name | Circuit Status |
| TER2 | Somewhere2 | Some Provider | C1 | Another provider | Cancelled |
| TER3 | Somewhere3 | TER3 Provider | C2 | Switch and Data | Cancelled |
When I am on the "connected_sites" page for site "TER2" Then the "connected-sites-list" should look like
| Site ID | Site Name | Site Provider | Provider Circuit ID | Provider Name | Circuit Status |
| TER1 | Somewhere1 | TER1 Provider | C1 | Another provider | Cancelled |
When I am on the "connected_sites" page for site "TER3"
Then the "connected-sites-list" should look like
| Site ID | Site Name | Site Provider | Provider Circuit ID | Provider Name | Circuit Status |
| TER1 | Somewhere1 | TER1 Provider | C2 | Switch and Data | Cancelled |

36. Enabling fixture-free tests
• create object
• execute method
• check result
• system under test
• save and load objects
• encapsulation, reuse

37. Demo
• State test
• Function test
• Integration test
• Other features
• Domain adapter
• Parse methods
• Factories

38. Questions
Comments

39. Mike Stockdale
• jediwhale@syterra.com
• www.asoftwarecraft.com
• twitter.com/jediwhale
• fitsharp.github.com
Groups
• tech.groups.yahoo.com/group/fitnesse
• tech.groups.yahoo.com/group/aa-ftt