Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Advanced automated visual testing with Selenium

Slides presented at Selenium Conference India 2016

Automated visual testing is a major emerging trend in the dev / test community. In this talk you will learn what visual testing is and why it should be automated. We will take a deep dive into some of the technological challenges involved with visual test automation and show how modern tools address them. We will review available Selenium-based open-source and commercial visual testing tools, demo cutting edge technologies that enable running cross browser and cross device visual tests at large scale, and show how visual test automation fits in the development / deployment lifecycle.

If you don’t know what visual testing is, if you think that Sikuli is a visual test automation tool, if you are already automating your visual tests and want to learn more on what else is out there, if you are on your way to implement Continuous Deployment or just interested in seeing how cool image processing algorithms can be, this talk is for you!

  • Inicia sesión para ver los comentarios

Advanced automated visual testing with Selenium

  1. 1. Adam Carmi (@carmiadam) Co-Founder and VP R&D at Applitools ADVANCED AUTOMATED VISUAL TESTING WITH SELENIUM
  2. 2. YOU CAN AND SHOULD AUTOMATE YOUR VISUAL TESTS!
  3. 3. AGENDA  Why automated visual testing?  Tools & Technology  Where does it fit?  Q & A
  4. 4. WHAT IS VISUAL TESTING? A quality assurance activity aimed to verify that a Graphical User Interface appears correctly to users
  5. 5. WHY AUTOMATE? The test matrix is too big to cover manually  Web browsers  Devices  Operating systems  Screen resolutions  Responsive design  L10n  3rd Party upgrades
  6. 6. WHY AUTOMATE? Mobile apps quality is critical  Harder to roll back changes  Can’t push daily  Updates take battery and data  Higher quality bar
  7. 7. WHY AUTOMATE? Release cycles keep getting shorter
  8. 8. AGENDA  Why automated visual testing?  Tools & Technology  Where does it fit?  Q & A
  9. 9. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  10. 10. DEMOWebDriverCSS
  11. 11. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  12. 12. AND MORE…  1 pixel offsets in element positioning  Dynamic content  Moving elements  Images of different size  Performance
  13. 13. IMAGE COMPARISON ENGINES
  14. 14. ImageMagick  A powerful command line tool for image processing.  Fuzzing is used to eliminate slight color differences  An error ratio is usually used to determine a match $ compare –metric AE –fuzz 5% img1.png img2.png diff.png 2246
  15. 15. THE JAVASCRIPT ENGINES  Resemble.js Pixel by pixel + error ratio + anti-aliasing http://huddle.github.io/Resemble.js  Blink-Diff Pixel by pixel + error ratio + anti-aliasing + perceptual color distance computation https://github.com/yahoo/blink-diff  Looks-Same Pixel by pixel + perceptual color distance computation + ignore Caret https://github.com/gemini-testing/looks-same
  16. 16. APPLITOOLS EYES  Handles anti-aliasing, pixel offsets, color similarity, and image scaling  Dynamic and moving content  Compare images of different sizes  No error ratio configuration required  Validates full UI pages  Fast!  Supports layout / structure matching
  17. 17. DEMOVISUAL LAYOUT MATCHING
  18. 18. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  19. 19. QUICK FEEDBACK TOOLS MOTIVATION  Get fast feedback on code changes  Run tests locally in the background SETUP  Render screenshots with a headless browser  PhatomJS, SlimerJS  Configuration file driven tests CONS  Partial coverage  Chrome, IE?  Old browser layout engine versions  Limited navigation
  20. 20. VISUAL COVERAGE TOOLS MOTIVATION  Verify that the app / site looks right in all real execution environments  Test as many UI states as possible  Execute many tests in parallel SETUP  Render screenshots using real browsers on a variety of operating systems and devices in parallel  WebDriver, Grid  Code driven tests  WebDriver, DSLCONS  Requires test infrastructure  Implement and maintain test code
  21. 21. SELENIUM BASED TOOLS CODE / SCRIPT ● Needle (Python WD) ● WebDriverCss (JS WebDriverIO) ● Gemini (JS DSL) ● Selenium Visual Diff (Java WD) ● VisualCeption (PHP CodeCeption) ● Pix-Diff (JS Protractor) ● Shoov (JS WebDriverIO) ● Vizregress (.NET WD) ● Rspec Page Regression (Capibara) ● Applitools Eyes (All WD + Appium) ● Fighting Layout Bugs (Java WebDriver) ● Huxley (Record Playback) CONFIGURATION ● Viff ● CSS Visual Test ● GreenOnion ● Wraith-Selenium
  22. 22. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  23. 23. REPORT DIFFERENCES As files on the file system WebDriverCSS
  24. 24. REPORT DIFFERENCES As a gallery Selenium Visual Diff
  25. 25. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  26. 26. UPDATE THE BASELINE Gemini
  27. 27. DEMOAUTOMATED MAINTENANCE
  28. 28. AGENDA  Why automated visual testing?  Tools & Technology  Where does it fit?  Q & A
  29. 29. WHERE DOES IT FIT? • Component s • Code review • Developers • Designers • QA Visual testing of frontend components by frontend developers
  30. 30. WHERE DOES IT FIT? • Pages • Page sections • Developers • Designers • QA • Others Full or partial validation of application screens A collaboration tool
  31. 31. WHERE DOES IT FIT? • Staging vs. Production • Ops • QA Validate your staging deployment using your production deployment as a baseline
  32. 32. WHERE DOES IT FIT? • Monitoring • Ops • QA No missing resources in production No corruption due to 3rd party data No corruption due to browser / OS upgrades
  33. 33. Adam Carmi (@carmiadam) Co-Founder and VP R&D at Applitools QUESTIONS?

×