Sauce Labs+Applitools - Automated Visual Testing in the Cloud
•Download as PPTX, PDF•
5 likes•9,541 views
- Reveal expert tips on how to successfully perform large-scale automated visual testing - Show how to leverage visual testing to increase coverage, while reducing maintenance efforts - Run a live cross-browser visual test with Sauce Labs and Applitools
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (14)
Similar to Sauce Labs+Applitools - Automated Visual Testing in the Cloud
Similar to Sauce Labs+Applitools - Automated Visual Testing in the Cloud (20)
More from Sauce Labs
More from Sauce Labs (20)
Recently uploaded
Recently uploaded (20)
Sauce Labs+Applitools - Automated Visual Testing in the Cloud
- 1. Adam Carmi Co-Founder and VP R&D at Applitools AUTOMATED VISUAL TESTING IN THE CLOUD Enhance Your Cross-browser Coverage Chris Broesamle Sales Engineer at Sauce Labs
- 2. AGENDA Why automatedvisual testing? Challengesin visualtestautomation Demo#1: Visualtestautomationwith ApplitoolsEyes Demo#2: Cross-browser testingin thecloud with SauceLabs Demo#3: Runninga cross-browser visualtestinthe cloud When toperformautomatedvisual testing? Q&A
- 3. WHAT IS VISUAL TESTING? A qualityassurance activity aimed to verify that a Graphical User Interface appears correctly to users
- 4. A VISUAL BUG
- 5. AND ANOTHER…
- 6. WHY SHOULD IT BE AUTOMATED? THE TEST MATRIX IS TOO BIG TO APPROXIMATE MANUALLY Webbrowsers Devices Operating systems Screen resolutions Responsive designs L10n
- 7. WHY SHOULD IT BE AUTOMATED?
- 8. WHY SHOULD IT BE AUTOMATED? NATIVE / HYBRID MOBILE APPS Harder to roll back changes Can’t pushdaily Updates take battery and data Higherquality bar
- 9. WHY SHOULD IT BE AUTOMATED? Many are already doing it… PhantomCSS Fighting Layout Bugs CSS Critc Wraith Needle Grunt PhotoBox dpdxt WebdriverCSS Eyes Huxley FBSnapshotTestCase GeminiSelenium Visual Diff VisualCeption Specter Snap And Compare
- 10. AGENDA Why automatedvisualtesting? Challenges invisualtestautomation Demo#1: Visualtestautomationwith ApplitoolsEyes Demo#2: Cross-browser testingin thecloud withSauceLabs Demo#3: Running a cross-browser visualtestin thecloud Whentoperformvisual testing? Q&A
- 11. THE WORKFLOW Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 12. THE CHALLENGES Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 13. SCALABILITY CHALLENGES Image comparison False positives Dynamic data Baseline maintenance Multiplebaselines Non-localityof changes
- 18. BRIGHTNESS 1/2
- 19. BRIGHTNESS 2/2
- 20. SUB PIXEL SHIFTS 1/2
- 21. SUB PIXEL SHIFTS 2/2
- 24. ANDMORE… 1 pixeloffsets in elementpositioning Dynamic content Movingelements Images ofdifferentsize Performance
- 25. AGENDA Why automatedvisual testing? Challengesin visual test automation Demo #1: Visualtest automationwith ApplitoolsEyes Demo #2: Cross-browser testingin thecloud with SauceLabs Demo #3: Runninga cross-browser visual test in thecloud Whento perform visual testing? Q&A
- 26. Chris Broesamle
- 27. Agenda • Sauce Labs Overview • Demo of Selenium on Sauce OnDemand
- 28. What is Sauce Labs?
- 29. We’ve built the worlds largest grid for executing Selenium & Appium tests. We’re open-source at heart and contribute to both projects.
- 30. Selenium Grid • A local test script is created using Remote WebDriver, which sends test commands to a grid • A grid consists of a hub server that directs requests and node servers that host OS/browser configurations for execution • Your local hub might be “gridhub.qa.mypage.com” • All you need to do is change that to “ondemand.saucelabs.com”
- 31. Local Selenium Grid Hub Receives test requests and relays them to Desired Node Nodes Execute commands on browsers as directed by the hub
- 34. Sauce Labs Infrastructure • Environments are kept up to date by our engineering team • Early access to browsers depends on browser developers • Browsers are retired as OS no longer supports it or WebDriver versions are deprecated • Selenium versions are chosen at run time based on compatibility with desired browser
- 35. Integrations • Plugins for Jenkins, Bamboo, TeamCity, and Travis CI • Manual testers can open defects in Jira from Sauce • Plugins handle authentication with Sauce, launching Sauce Connect, OS/browser selection in a build, and integration of results
- 36. Sauce Labs REST API • Authentication • Provisioning • Jobs (including test results) • Information • Usage • Tunnels • Storage
- 37. Selenium on Sauce • Demo
- 38. AGENDA Why automatedvisualtesting? Challengesin visualtestautomation Demo#1: Visualtestautomationwith ApplitoolsEyes Demo#2: Cross-browser testingin thecloud with SauceLabs Demo#3: Visualtestautomationin thecloud When toperform visualtesting? Q&A
- 39. WHERE DOES IT FIT? • Component s • Code review • Developers • Designers • QA
- 40. WHERE DOES IT FIT? • Pages • Page sections • Developers • Designers • QA • Others
- 41. WHERE DOES IT FIT? • Staging vs. Production • Ops • QA
- 42. WHERE DOES IT FIT? • Monitoring • Ops • QA
- 43. QUESTIONS?
Editor's Notes
- Most tools are for web apps except for FBSnapshotTestCase which is for IOS and Eyes which is platform agnostic.
- * Does not apply to fighting layout bugs.
- * Does not apply to fighting layout bugs.
- Like Boromir said…
- Open Source The Selenium project was started in 2004 by Jason Huggins (a Sauce Labs co-founder). It’s open source software released under the Apache 2.0 license. Huggins joking refereed to Selenium as the antidote to Mercury (no Hewlett Packard) poisoning a competitor to Selenium. We also have one of the creators of Appium which much like Selenium allows teams to create an automation scripts in any programing language then have those scripts automate Hybrid and Native mobile applications. Automation is faster and more accurate than manual Development cycles continually becoming faster and more agile. If organizations do not choose to automated eventually they will fall behind and no longer be able to compete with a high quality product. Supports all major programing languages With Selenium and Appium scripts can be created in an common programming language (Java, .NET, PHP, Pyhon, Ruby, Perl and JavaScript) Supports all major browsers This includes both mobile web and web. The de facto web automation tool Everyone uses it. 30% of fortune 500 use Selenium.
- .
- Why should customers care about out product? Sell the dream of a better future 1. Deliver a story or statement that arouses the audience’s interest Sauce Labs was founded 4 years ago by Jason Huggins (inventor of Selenium), Steve Hazel and John Dunham, with the goal of making testing awesome. 2. Pose a problem or question that has to be solved or answered Cross browser lab maintenance. How much time and money will it take you to setup and maintain on internal grid 3. Offer a solution to the problem you raised Sauce’s client cloud 4. Describe specific benefits for adopting the course of action set forth in your solution Time sand money savings 5. State a call to action
- Open Source The Selenium project was started in 2004 by Jason Huggins (a Sauce Labs co-founder). It’s open source software released under the Apache 2.0 license. Huggins joking refereed to Selenium as the antidote to Mercury (no Hewlett Packard) poisoning a competitor to Selenium. We also have one of the creators of Appium which much like Selenium allows teams to create an automation scripts in any programing language then have those scripts automate Hybrid and Native mobile applications. Automation is faster and more accurate than manual Development cycles continually becoming faster and more agile. If organizations do not choose to automated eventually they will fall behind and no longer be able to compete with a high quality product. Supports all major programing languages With Selenium and Appium scripts can be created in an common programming language (Java, .NET, PHP, Pyhon, Ruby, Perl and JavaScript) Supports all major browsers This includes both mobile web and web. The de facto web automation tool Everyone uses it. 30% of fortune 500 use Selenium.
- Open Source The Selenium project was started in 2004 by Jason Huggins (a Sauce Labs co-founder). It’s open source software released under the Apache 2.0 license. Huggins joking refereed to Selenium as the antidote to Mercury (no Hewlett Packard) poisoning a competitor to Selenium. We also have one of the creators of Appium which much like Selenium allows teams to create an automation scripts in any programing language then have those scripts automate Hybrid and Native mobile applications. Automation is faster and more accurate than manual Development cycles continually becoming faster and more agile. If organizations do not choose to automated eventually they will fall behind and no longer be able to compete with a high quality product. Supports all major programing languages With Selenium and Appium scripts can be created in an common programming language (Java, .NET, PHP, Pyhon, Ruby, Perl and JavaScript) Supports all major browsers This includes both mobile web and web. The de facto web automation tool Everyone uses it. 30% of fortune 500 use Selenium.
- Open Source The Selenium project was started in 2004 by Jason Huggins (a Sauce Labs co-founder). It’s open source software released under the Apache 2.0 license. Huggins joking refereed to Selenium as the antidote to Mercury (no Hewlett Packard) poisoning a competitor to Selenium. We also have one of the creators of Appium which much like Selenium allows teams to create an automation scripts in any programing language then have those scripts automate Hybrid and Native mobile applications. Automation is faster and more accurate than manual Development cycles continually becoming faster and more agile. If organizations do not choose to automated eventually they will fall behind and no longer be able to compete with a high quality product. Supports all major programing languages With Selenium and Appium scripts can be created in an common programming language (Java, .NET, PHP, Pyhon, Ruby, Perl and JavaScript) Supports all major browsers This includes both mobile web and web. The de facto web automation tool Everyone uses it. 30% of fortune 500 use Selenium.
- We currently support 270 OS and browser combinations. Anywhere from IE 11 running on Windows 8 to a dated version of Firefox running on Linux. Imagine the monumental effort involved in doing this yourself. There are 4 key components to making this happen. Hardware Software orchestration man hours
- Open Source The Selenium project was started in 2004 by Jason Huggins (a Sauce Labs co-founder). It’s open source software released under the Apache 2.0 license. Huggins joking refereed to Selenium as the antidote to Mercury (no Hewlett Packard) poisoning a competitor to Selenium. We also have one of the creators of Appium which much like Selenium allows teams to create an automation scripts in any programing language then have those scripts automate Hybrid and Native mobile applications. Automation is faster and more accurate than manual Development cycles continually becoming faster and more agile. If organizations do not choose to automated eventually they will fall behind and no longer be able to compete with a high quality product. Supports all major programing languages With Selenium and Appium scripts can be created in an common programming language (Java, .NET, PHP, Pyhon, Ruby, Perl and JavaScript) Supports all major browsers This includes both mobile web and web. The de facto web automation tool Everyone uses it. 30% of fortune 500 use Selenium.
- Open Source The Selenium project was started in 2004 by Jason Huggins (a Sauce Labs co-founder). It’s open source software released under the Apache 2.0 license. Huggins joking refereed to Selenium as the antidote to Mercury (no Hewlett Packard) poisoning a competitor to Selenium. We also have one of the creators of Appium which much like Selenium allows teams to create an automation scripts in any programing language then have those scripts automate Hybrid and Native mobile applications. Automation is faster and more accurate than manual Development cycles continually becoming faster and more agile. If organizations do not choose to automated eventually they will fall behind and no longer be able to compete with a high quality product. Supports all major programing languages With Selenium and Appium scripts can be created in an common programming language (Java, .NET, PHP, Pyhon, Ruby, Perl and JavaScript) Supports all major browsers This includes both mobile web and web. The de facto web automation tool Everyone uses it. 30% of fortune 500 use Selenium.
- Open Source The Selenium project was started in 2004 by Jason Huggins (a Sauce Labs co-founder). It’s open source software released under the Apache 2.0 license. Huggins joking refereed to Selenium as the antidote to Mercury (no Hewlett Packard) poisoning a competitor to Selenium. We also have one of the creators of Appium which much like Selenium allows teams to create an automation scripts in any programing language then have those scripts automate Hybrid and Native mobile applications. Automation is faster and more accurate than manual Development cycles continually becoming faster and more agile. If organizations do not choose to automated eventually they will fall behind and no longer be able to compete with a high quality product. Supports all major programing languages With Selenium and Appium scripts can be created in an common programming language (Java, .NET, PHP, Pyhon, Ruby, Perl and JavaScript) Supports all major browsers This includes both mobile web and web. The de facto web automation tool Everyone uses it. 30% of fortune 500 use Selenium.
- CSS files and HTML falling out of sync CSS changes with unexpected global implications. Frontend refactoring.
- Project / Product managers (collaboration)
- Data from external sources Not enough time to test pre-production Missing components in deployment.