Do you understand Exploratory Testing? Are you familiar with the lingo and concepts. Things like Charters, Mind Maps, Heuristics? Feeling a bit awkward? This talk, The Awkward Teenager of Testing, is designed to reduce your awkwardness with those concepts by gaining experience by writing and executing charters, integrating a mind map using GML, and applying heuristics as resources.
2. Tested = Checked + Explored
Charters
Target Areas
Resources
Discovery
MindMaps
EcoSystem Heuristics
G o l d i l o c k sFollowtheData
Nouns and Verbs
Starvation
Reconnaissance Charter
A c t i v e R e a d i n g
AmbiguityAnalysis
Risk Heuristics
We think we understand exploratory testing just as we think we
understand teenagers, because “we have been there”.
6. What is Exploratory Testing?
• Tested = Checked + Explored
• Going off-script to find that undiscovered bug.
• What you do to promote code to the next testing level.
• High-value testing that relies on a tester’s intuition, skill, and
testing experience.
8. Resources
Target
Areas
To
Discover
Gives us a common, light-weight, solid foundation from which to venture forth. They are a
framework that aides us in gathering insights while we wander and help us avoid getting lost.
A Spot, not a
travel itinerary
You must plan
ahead what you
will need
You must think
You seek
treasures not
defects
Charters
9. Example Charter
Explore Sign In
With a state table and a valid
account
to discover if I am able to
access and view information
only when I am properly
signed in
10. Part 3: Mind Map
Adoption
Heuristics
Mind Map
Charter
Exploratory
Testing
TalkTalk
11. Galactic
Modeling
Language
EcoSystem
Information
Radiator
Mind Maps are a visual representation of your system. They are not detailed architectural
diagrams. They help orient your exploratory testing.
3 parts:
1) Boxes
2) Lines
3) Text
Keep it simple
Easy to update
Conversation
starters
As important as
a Scrum board
EcoSystem =
Mind Map +
Infrastructure
Mind Maps
12. Google.com
Fun Stuff
Images
Grid
Notification
Gmail
Icon Array
More Area
Settings
Terms
Privacy
About
Business
Ads
Stuffy Stuff
Search Area
Search Box
Search Btn
Lucky Btn
Other
Results Page
Doodle
Social Media
Example Mind Map
Sign In
Explore Sign In
With a state table and a
valid account
to discover if I am able
to access and view
information only when I
am properly signed in
14. Heuristic
Heuristic Definition
Nouns and Verbs List out and number all nouns and verbs in your system
Roll 1 dice for nouns and 1 dice for verbs; the result is what
you test
Never and Always Apply “Never” and “Always” conditions to the “ilities”
(Usability, Scalability, Reliability, etc.)
Follow the Data Apply “Create,” “Read,” “Update,” and “Delete” to your
data
State Table List States and Events in a table, explore possible odd
combinations
Goldilocks Try small (including zero), medium, large data entry
Format Explore fields with special formatting
Navigation Try keyboard, mouse, and browser buttons
15. Example Heuristics
Explore Sign In
With a state table and a
valid account
to discover if I am able
to access and view
information only when I
am properly signed in
Explore Search Area
With nouns and verbs
heuristic
to discover unexpected
behavior when I interact
in a random fashion.
Explore Google.com
With never and always
heuristic
to discover if I Shift-
TAB always moves me
back from where I tabbed
from
Explore Search Box
With the Goldilocks
heuristic and
randomtextgenerator.co
m
to discover effect on
search results with
various sizes of search
text
Explore Grid
With navigation heuristic
and the Safari and
Chrome browsers
to discover if I can
navigate successfully
using only the TAB, Shift-
TAB, and Enter Keys
17. Adoption Roadmap
Complexity
Practice Adoption
Charters Mind Map Heuristics
Foundational
Practices
Intermediate
Practices
Advanced
Practices
Doc Recons
1 charter per person
Paired testing
Expand the mind map
Updates are occuring
Plotting charters
Heuristics are used on some
charters
Team prioritizes use of
heuristics
Charters exists
Target areas tied to
mind map
Basis for metrics
1 page map exists
Mind map the heuristicsInclude hardware and
software architecture
N/A
N/A
18. Provide guidance, not step by step actions Read like a test script
Can be completed in one session Are not specific enough to know what has or has not been
tested in the charter execution
Are aligned with your stakeholder’s concerns Ignore high risk areas and do not have input from your
stakeholders
Use a common language or terms to describe the
target areas so that others know what has or has not
been explored
Are ambiguous as to where you have been or refer to
terms only your team uses
Are recorded and made visible so that future charters
can use the results from those charters as inputs into
new charters
Cut’n paste the same charters and look more like a
traditional smoke test
.
Demonstrate some thought about what you are trying
to discover or why you are going to a certain area
Use vague or broad terms about what you hope to
discover, e.g., “to discover bugs”
Complement and augment the other components of
your testing strategy
Repackage your test scripts and scenarios
Good Charters Bad Charters
19. Challenges and solutions
transitioning a traditional
test team into an
exploratory test team on a
large, heavily interfaced
system.
I’ve a feeling we’re not in Kansas anymore
Challenge: Changing a mindset
Solution: Framework, cadence, training, discussions, and metrics
Why do you want access to my system?
Challenge: You need “visas” to explore
Solution: You need allies, and strong ones
Yet another build!!!?
Challenge: Continuous Delivery means continuous deployments
Solution: Test in parallel with and after production deployments
What is truth?
Challenge: Acceptance criteria vs. Desired Outcome
Solution: Find someone who cares
Boldly going where no one has gone before
Challenge: Indigenous population may be indigent
Solution: “Marketing” brochure and chocolate chip cookies
Notes From the Trenches
20. References
3 Titans:
• Elisabeth Hendrickson
• James Bach
• Cem Kaner
Elisabeth Henderickson
Hendrickson, Elisabeth,
Explore It!: Reduce Risk and
Increase Confidence with
Exploratory Testing
James Bach
http://www.satisfice.com/blo
g/archives/category/explorat
ory-testing
Cem Kaner
http://www.kaner.com/pdfs/
QAIExploring.pdf
