These are the latest version of the slides for mine and Jonathan's talk on Testing Katas. They cover using katas to learn testing skills and then include an example kata that we have run in house.
CNIC Information System with Pakdata Cf In Pakistan
Testing katas - Try before you buy
1. Testing Katas: Try Before you Buy
Emma Armstrong
@EmmaATester
www.taooftesting.co.uk
Jonathan Watts
@whatie
2. Today’s Session
• What’s this session all about?
• What are katas and why use them for testing?
• Some background on using oracles for testing
• The kata
• Introduce the objective
• Testing the application
• Debrief session
• Just ask questions as we go through
3. What are katas?
• Repeated practical exercises to ingrain behaviours until they become
instinctual
• Allow you to cover various topics and situations in a safe learning
environment
• Encourage knowledge sharing and collaboration
4. Why use katas for testing?
• Deliberate practice
• Learning and development – Tacit skills
• Consistency
• Knowledge sharing
• Quality is everyone’s responsibility
5. Let’s try it then…
• Introduce the topic
• Get into groups
• Test the application as a group
• Report back on your findings
• Group discussion
6. Heuristics and Oracles
A heuristic is a mechanism to try and solve a
problem or learn something
Testing examples include:
• Brute force
• Testing at the boundaries
• Input methods
• Starvation
• Working backwards
• Common sense / Educated guess
• Rule of thumb
7. Heuristics and Oracles
“An oracle is a heuristic principle or mechanism by which someone recognizes a problem.”
Michael Bolton, 2010
Examples:
• Requirements • Marketing claims
• Laws and standards • Scientific models
• Previous versions • Maths
• Comparable products • Experts
• Your experience
When do you use this?
• Bug evaluation
• Reporting
• Test design
They are fallible
8. The kata - The consistency heuristic
“We expect the system to be consistent with systems that are in some
way comparable. This includes other products in the same product line;
competitive products, services, or systems; or products that are not in
the same category but which process the same data; or alternative
processes or algorithms.”
Michael Bolton
Examine the calculator web application from a black box perspective
using the consistency heuristic and document your findings.
For example, compare the behaviour our calculator to another
equivalent application and see where the inconsistencies are.
http://bit.ly/1v4Nw6R (http://emmaarmstrong.github.io/CalculatorJavaScript/Calc.html)
9. Debrief
• What did you find?
• How did you go about finding them?
10. Where do we go from here?
• There is also a development
opportunity for those people
running the katas
• We are aligning them with other
educational activities at Red Gate
• We are sharing this talk and the
katas we have run with anyone
who wants them
• Come and join in our katas
11. Questions?
The calculator
https://github.com/EmmaArmstrong/CalculatorJavaScript
Code katas
http://codekata.com/
http://codingkata.net/
http://dev.red-gate.com/category/blog/code-katas/
Test katas
http://www.soapui.org/Testing-Katas/what-are-testing-katas.html
http://testing-challenges.org/tiki-index.php
http://weekendtesting.com/
Test heuristics
http://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf
Thank you for your time
Notas del editor
About
Emma Armstrong
Jonathan Watts
Red Gate
What are katas? Kata is a Japanese word describing exercises that were done repeatedly until they became instinctual. Dave Thomas, the co author of The pragmatic programmer first coined the term in respect of code katas and testing katas are no different. We are talking about exercises done in groups, pairs or even individually that allow you to practice something and then learn from others experiences through discussion. We are not suggesting the same exercise done again but different exercises or the same exercise with different constraints.
We believe that the forms of katas can be used to help learn any skill but especially testing, where testing should be happening at all stages in the project and it should be instinctual, testers should know what to do, and adapt their approach based on the context they are in.
Why do katas? As we know, complete testing is rarely possible especially if you have over about 2 lines of code. This can mean the testing tasks at hand can seem infinite and so katas provide a way of enabling deliberate practice.
Maintaining product quality across multiple teams. We work in cross functional teams, which means some of the knowledge transfer of skills in testing that might have occurred due to being immersed in a testing team does not, so to try and develop expertise we need to facilitate this (instructional expertise) in a different way.
If you google Agile and quality, the phrase “Quality is everyone’s responsibility” will come up a few times. We believe that to achieve this you need to expose people of other disciplines to the activities that testers perform. The testing katas that we run are open to the whole company.
Format – At Red Gate we have deliberately chosen to keep the exercises down to only a 20-30minute exercise and then a 30minute -1hr meeting afterwards and only do 1 kata a month. We have also run 1hr workshops.
In today’s kata we are not going to give you any specifications we want you to apply the consistency oracle, specifically looking at the consistency with a comparable product heuristic. So we are focussing on using the behaviour of a comparable product to help evaluate the behaviour of the product under test.
There are several oracles that you will already be using daily even if you are not calling them out as such. As Cem Kaner says, an oracle is a mechanism for determining whether a program has passed or failed a test. The oracle that you use depends on several things, the testing that you are doing, the environment you are in, the application you are testing, the context of the testing for example it could be a specification, it can be laws/statutes, other products, you, etc..
Specs being wrong
Consistency with ‘something’ is an oracle we often rely on.
Specifications
Laws
Previous versions
Comparable products
Marketing claims
There are several oracles that you will already be using daily even if you are not calling them out as such. As Cem Kaner says, an oracle is a mechanism for determining whether a program has passed or failed a test. The oracle that you use depends on several things, the testing that you are doing, the environment you are in, the application you are testing, the context of the testing for example it could be a specification, it can be laws/statutes, other products, you, etc..
Specs being wrong
Consistency with ‘something’ is an oracle we often rely on.
Don’t do a kata on something you haven’t tried out yourself a few times
Try to keep the examples simple
Keep a note of examples
Need a backlog
Emma and Jonathan have gained some teaching experience
Improved communication
2 new testers
Improved aware of testing activities and methodologies
We have learnt new things
Continue to do our own deliberate practice?
Would you like to use our previous katas?
Coding katas talk