Growing Manual Testers into Automators
•
1 recomendación•558 vistas
Manual testing can't keep up with modern software development. Tests generated by Capture/Playback tools don't work either as these tests become brittle and break. Instead working with business and development testers need to create acceptance criteria that drives product development. To become automators testers need new tools. Testers need new ways of working. Testers need new skills. And the organization needs to support your testers growth. Here is how I and others have made it work.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (12)
Similar a Growing Manual Testers into Automators
Similar a Growing Manual Testers into Automators (20)
Más de Camille Bell
Último
Último (20)
Growing Manual Testers into Automators
- 1. Growing Manual Testers into Automators Camille Bell cbell@CamilleBellConsul0ng.com Twi5er @agilecamille Agile DC 2015 October 26, 2015
- 2. Obvious Limits of Manual Tes6ng cbell@CamilleBellConsul0ng.com 2 Itera6ons Time Time Needed Time Available New Features Regression Simplifica)on: e.g. Doesn't include interac)on of new & exis)ng features
- 3. Delay in Tes6ng (both manual & capture/playback) cbell@CamilleBellConsul0ng.com 3 Dev. Months Test Dev. Test Dev. Test Dev. Test Days Weeks Even in the best case tes)ng is a@er development coding.
- 4. • Keystroke, mouse capture • Test briLle (trivial UI changes break tests) • Hard coded test data • Lots of duplica6on • Test modifica6on near impossible (spagheP code when accessible) • No control mechanisms (for looping, etc.) • Doesn't scale • Over 6me degrades so much as to become throw away • Etc. • Tests wriLen aSer implementa6on • Not in business language • Doesn't facilitate collabora6on • Doesn't drive developer progress (not executable un6l aSer development) • Encourages finger poin6ng • Not an executable specifica6on • Etc. cbell@CamilleBellConsul0ng.com 4 Capture/Playback Tes6ng Fails Because:
- 5. These Are All The Same Thing • Acceptance Test Driven Development (ATDD) • Behavior Driven Development (BDD) • Story Driven Development • Specifica6on by Example – Minor subtle difference don't maLer for most purposes – Collaborate to define detailed acceptance criteria before developing applica6on code, turn those into executable acceptance tests (before developing applica6on code), watch them fail (no app) and later pass, when app func6onality implemented cbell@CamilleBellConsul0ng.com 5
- 6. Collabora6vely (Spec it/define tests) • Some6mes called the 3 Amigos (test, dev, business) • Conversa6on based • Focus on value • Specific examples • Concrete data • Outside in (business focus drives development) cbell@CamilleBellConsul0ng.com 6
- 7. Three Parts of Wri6ng any Test Cucumber 3 As Ruby 3 As Java Formal Testers What it is Given Assemble Arrange Precondi6on All the setup and dependencies for the test When Ac6vate Act Ac6on/Event Something that happens that will make the outcome different than if it hadn’t happened Then Assert Assert Expected Results or Postcondi6on What is expected to happen and any side effects cbell@CamilleBellConsul0ng.com 7
- 8. Test Automa6on Has 2 Sides Business Facing • Simple English • In business terms • Promotes collabora6on • Uses examples • Uses concrete data • Proves business needs met • Executable specifica6on • Modifiable & reusable Technology Facing • Exercises underlying SW • Connects to browsers, networks, databases, frameworks, etc, • Object Oriented • Extensive test libraries • Modifiable & reusable • Learnable programming language cbell@CamilleBellConsul0ng.com 8
- 9. Business Facing • User Stories, especially those created though story mapping, are a good place to start talking about acceptance criteria • That can lead to thinking about categories of test scenarios (test 6tles) • Given, When, Then comes next • Make sure there is concrete test data • Let's see how it works cbell@CamilleBellConsul0ng.com 9
- 10. Acceptance Criteria will become our specs when they have details Feature: Cash Withdrawal In order to get money when the bank is closed As a bank customer I want to withdraw cash at the ATM • Successful Withdrawal • Less than balance • Equal to balance • Withdrawal Failed Due to Insufficient Funds • Withdrawal Failed Because Cash Dispenser Doesn’t Dispense One Dollar Bills • Withdrawal Failed Because Account Closed First Cut Acceptance Criteria for several tests cbell@CamilleBellConsul0ng.com 10
- 11. • Successful Withdrawal (less than balance) Given my account has starting balance of $1000 When I withdraw $200 Then $200 should be dispensed And the ending balance of my account should be $800 Given, When, Then format – Business language with unambiguous detail Detailed Acceptance Criteria These detailed acceptance criteria are acceptance tests that are executable specs cbell@CamilleBellConsul0ng.com 11
- 12. • Though users only sees the UI, Acceptance Tests exercise behavior of the en6re applica6on stack (network, DB, etc.). • However since capture playback causes so many problems, we needed a beLer way to automate, including: – Modeling behavior, so it is easier to modify – Working in non-‐proprietary programming language testers can learn – Having good test architecture and design – Exercising good programming prac6ces (D.R.Y., meaningful variable names, etc.) – Following good code management (frequent check-‐ins, simple branch structure, frequent merging with trunk/main, etc.) cbell@CamilleBellConsul0ng.com 12 Technology Facing
- 13. Features Scenarios Automa6on Libraries (e.g. rspec, page-‐object, wa6r-‐webdriver, etc.) Support Code (e.g. env.rb, hooks.rb, Gemfile, your custom page objects) Business Facing Technology Facing cbell@CamilleBellConsul0ng.com 13 Step Defini6ons Cucumber/PageObject Technology Stack Gherkin (English-‐like business language) Built-‐in Ruby Libraries for Tes6ng Ruby (oSen modified templates) Steps wriLen in Ruby Steps
- 14. • Cucumber can be used with other programming languages • But the Ruby implementa6on comes with much more out of the box than any other implementa6on • For example Ruby libraries already exist to model – web pages PageObject – iPhone screens ScreenObject – 3270 green screens TE3270 – XML Nokogiri – etc. • In other languages testers have to write a lot of complicated code that's included with Ruby • And Ruby is easiest for testers to learn cbell@CamilleBellConsul0ng.com 14 Why Ruby?
- 15. • Yes, assuming we're tes6ng a web app • The stack includes wa6r-‐webdriver, which is a combina6on of Selenium and Wa6r – Which is fully programmable – Can be accessed though PageObject – And is good • NOT the Selenium IDE – Which is capture/playback – And is bad cbell@CamilleBellConsul0ng.com 15 Don't We Also Need a Web Interface?
- 16. Tester Behavior & System Access • Testers no longer just use a test tool -‐ they develop test code • Testers now will oSen behave like devs. Testers will – Write code – Run code – Lookup error messages and technical info on the web – Check-‐in code – Update gems and other SW – Install new browsers & other tools – Etc. • So they need the same rights and privileges as devs. For most shops this means: – Admin rights – Unrestricted web access – Working with a dev, sys admin or dev-‐op helps for beginning installs cbell@CamilleBellConsul0ng.com 16
- 17. Tools and Infrastructure • Installa6on details for Windows, Mac and Linux differ (documented in Cucumber and Cheese) • Testers need mul6ple browsers installed (Firefox and Chrome plus IE, Safari or whatever is standard) • Select and install an editor or IDE {consider Sublime Text (text editor) or RubyMine (IDE)} • Test code needs to be under source control, so connect to Git (modern choice) or other Source Control • For Ruby Gem security and other updates 'github.org' and 'rubygems.org' both testers and the CI server should update their Gemfile regularly • The CI needs full test stack. If OSs differ (e.g. Windows vs. Linux) some gems may differ. Jenkins is popular CI server. cbell@CamilleBellConsul0ng.com 17
- 18. • My style: – Ensure all infrastructure for every trainee is in place – First teach small bit of Ruby intro for non-‐ programmers – 3 day formal class on ATDD/BDD with Cucumber with lots of hands-‐on exercises – Start with teaching students how to test exis6ng web pages – Evolve to teaching how to define web pages that don't exist yet • Resources at end of slides • Don't just throw a book or web site at testers, teach them & guide them Training cbell@CamilleBellConsul0ng.com 18
- 19. Coaching & Daily Prac6ce • DON'T JUST TRAIN -‐ PRACTICE • Technical skills take 6me to learn • Requires reinforcement • Professional coaches really help • Pairing with devs may help ease technical learning • Later advanced testers could become internal coaches • Prac6ce 3 amigos • Mob Programming really helps cbell@CamilleBellConsul0ng.com 19 Two co-‐workers pair programming
- 20. Mob Programming • Whole team works together • On the same thing • At the same 6me • In the same space • Using the same computer • Driver rotates • Everyone par6cipates • Great for training & prac6ce • Great for collabora6on • Everyone up to speed faster • Some teams mob always • Others mob frequently cbell@CamilleBellConsul0ng.com 20
- 21. Demo
- 23. Resources – Free Ruby Info Ruby 101 – the basics: (Learn Ruby 2.x not Ruby 1.9) • On-‐line tutorials – my favorite is Ruby Monk hLps://rubymonk.com Learn to navigate the documenta0on: • Ruby's built-‐in classes and methods are very rich. • Testers should learn to check if the needed func6onality already exists • Ruby docs are found at hLp://ruby-‐doc.org/ cbell@CamilleBellConsul0ng.com 23 ALL SOME
- 24. Resources – Free PageObject Info Cheezy's (Jeff Morgan's) Blog • Other Cheezy blog posts are also worthwhile hLp://www.cheezyworld.com/ 2011/07/29/introducing-‐page-‐ object-‐gem/ Official documenta0on • PageObject is a gem. • Like most gems, it and its documenta6on can be found on GitHub (scroll down) hLps://github.com/cheezy/page-‐ object cbell@CamilleBellConsul0ng.com 24
- 25. Resources – Free Detailed Demo Cheezy's Demo Tes0ng Puppies Site • Uses RubyMine IDE, but other editors or IDEs work fine • Covers a later example from Cucumber and Cheese book resource • 1 hour 10 minute YouTube video hLps://www.youtube.com/watch? v=zpt15XIjpas cbell@CamilleBellConsul0ng.com 25
- 26. Resources – Free Cucumber Info Cucumber Wikis • Both wikis focus on headless Cucumber (not in browser with PageObject), but provide useful info on Cucumber and Gherkin • Older Wiki – easy to explore format, contains some info not in new wiki hLps://github.com/cucumber/cucumber/wiki • Newer Wiki – contains some nice blogs and plug for semi commercial product and support hLps://cucumber.io/ cbell@CamilleBellConsul0ng.com 26
- 27. Resources – Ruby Books If Ruby Monk is too challenging, try Learn to Program • For those who haven't programmed before (most manual testers) – Learn to Program -‐ Second Edi0on by Chris Pine • For those who know object-‐oriented programming in some other language (includes more Ruby than most testers will need) – Programming Ruby 1.9 & 2.0 by Dave Thomas • For a problem-‐based approach to wri6ng small pieces of Ruby that do useful things – Everyday Scrip0ng with Ruby by Brian Marick cbell@CamilleBellConsul0ng.com 27 Available from Amazon or the Pragma6c Bookstore hLps://pragprog.com/
- 28. Resources – Cheezy's Book Cucumber with PageObject • Covers Cucumber, Gherkin, PageObject, Wa6r, Selenium, etc. used in demo • Step by step tutorial with exercises • SoScopy only, work in progress, updated periodically – Cucumber and Cheese: A Testers Workshop by Jeff Morgan cbell@CamilleBellConsul0ng.com 28 Available only from Lean Pub hLps://leanpub.com/cucumber_and_cheese
- 29. Resources – Cucumber Books Ignore references to headless Rails tes0ng unless using Rails • Cucumber with Ruby – General – The Cucumber Book: Behavior-‐Driven Development for Testers and Developers by MaL Wynne & Aslak Hellesoy • Advanced Cucumber Topics – The Cucumber Book: Behavior-‐Driven Development for Testers and Developers by Ian Dees and MaL Wynne cbell@CamilleBellConsul0ng.com 29 Available from Amazon or the Pragma6c Bookstore hLps://pragprog.com/
- 30. Camille Bell Agile Technical & Process Coaching Retrospec6ves Agile Boot Camps Agile Training Updated Slides or just to chat about things agile cbell@CamilleBellConsul0ng.com Twi5er @agilecamille