Software test automation continues to be a huge challenge to achieving continuous testing in DevOps. Test flakiness and code maintenance — together with the complexity of automating advanced scenarios in shrunken timelines — has created a need for a next-generation test automation solution.
3. 3 | How to Embed Codeless Automation into DevOps perfecto.io
Today’s Speakers:
Eran Kinsbruner
• Chief Evangelist and author at Perfecto
• Blogger and speaker
• 19+ years in development & testing
• Author of
“The Digital Quality Handbook” and
“Continuous Testing for DevOps
Professionals”
• Twitter: @ek121268
Johnny Lam
• Senior Sales Engineer at Perfecto
• Software engineer with over 10 years in development & testing
• Expert in implementing and advising on test strategies for Fortune
500 enterprises and leading tech companies
• LinkedIn: https://www.linkedin.com/in/johnnykclam/
4. 4 | How to Embed Codeless Automation into DevOps perfecto.io
Today’s Agenda
1
2
3
Defining the main personas involved in continuous testing.
Introduction to codeless testing and its benefits.
The main differences and considerations to be aware of between
codeless and code-based testing.
4 Live demo: Perfecto Codeless!
5 Q&A
5. 5 | How to Embed Codeless Automation into DevOps perfecto.io
What is Continuous Testing?
Continuous testing is the process of executing automated
tests as part of the software delivery pipeline in order to
obtain feedback as rapidly as possible.
6. 6 | How to Embed Codeless Automation into DevOps perfecto.io
The Path to Continuous Testing
1 2 3 4 5
Stable automation Daily cycle Increase coverage Reach 95% Continuous testing
W H A T Y O U ’ L L G E T
W H A T Y O U ’ L L N E E D
• 99.9% availability lab
• Evidence collection
• Process integration
• CI
• Defect tracking
• Execution control
• Dashboarding
• Create scripts
• Maintain scripts
• Understand what’s
wrong
• Skillset matched
tool
• False negative
detection in reports
• Run scripts daily
• Get results in
minutes
• Valuable coverage
• Meaningful daily
feedback
• Advance validations
• Visual automation
• Basic orchestration
• Up-to-date lab
• Scaled lab
• Analysis grouping
• Automate all that is
possible
• Advanced
orchestration
• Elasticity
• Sharding
• Environment
control
• Scaled reporting
• Nightly
Continuously
• Cloud execution
• Very high scale
D E V O P S F R I E N D L Y Z O N E
7. 7 | How to Embed Codeless Automation into DevOps perfecto.io
Key Personas in Continuous Testing
8. 8 | How to Embed Codeless Automation into DevOps perfecto.io
Continuous Testing: The Three Personas
B U S I N E S S T E S T E R
S D E T ( S O F T W A R E D E V E L O P E R I N T E S T I N G )
S O F T W A R E D E V E L O P E R
Performs exploratory and structured manual testing.
Creates functional and nonfunctional code-based test
automation scripts (via Selenium, Appium, etc.).
Creates unit and build acceptance tests.
9. 9 | How to Embed Codeless Automation into DevOps perfecto.io
Business Tester:
Manual? Exploratory?
BDD? Codeless?
Persona-Based Challenges in Continuous Testing
P L A N C O D E B U I L D T E S T D E P L O Y R U N
Test Platform
Test Framework
Trigger Feedback
Time
90% Emulator/Local Device
10% Cloud Device
Unit
API
Espresso / XCUITest
Exploratory
Pre-commit
Manually – On-demand
Few minutes
70% Emulator
30% Cloud Devices
API
Espresso / XCUITest
Commit / Few # day
Automatic
< 5-10 Min
100% Cloud Devices
Espresso/XCUITest
Appium
Nightly
Automatic
< 6 Hours
100% Real Devices (S)
Espresso/XCUITest
24/7
Automatic
< 5 Min
SDET
Functional/NF
Developer
Unit/BAT
?
D E V O P S C O N T I N U O U S F E E D B A C K U N O P T I M I Z E D M O D E LU N O P T I M I Z E D M O D E L
10. 10 | How to Embed Codeless Automation into DevOps perfecto.io
I N T E R A C T I V E
T E S T S
UI/UX manual tests
Balancing Test Creation for the Three Different
Personas With the Right Tools
O P T I M I Z E D M O D E L
Developers & SDETs
(Code-Based)
Business Testers
Ownership (Codeless)
Business Testers
Ownership
11. 11 | How to Embed Codeless Automation into DevOps perfecto.io
Introduction to Codeless Testing
12. 12 | How to Embed Codeless Automation into DevOps perfecto.io
Introduction to Codeless Testing
R E C O R D
& P L AY B A C K
S E L F H E A L I N G
( M L / A I )
S TA N D A R D
F U N C T I O N A L I T I E S
C O N N E C T T O
C I / C L O U D
C O M P L E M E N T S
C O D E - B A S E D T E S T I N G
Codeless testing –
process of creating
automated tests
without writing a single
piece of code.
13. 13 | How to Embed Codeless Automation into DevOps perfecto.io
Main Differences Between Code-Based
and Codeless Testing
14. 14 | How to Embed Codeless Automation into DevOps perfecto.io
• Define manual test scenarios, BDD, and user
stories style scenarios.
• Create scripts in Java/JS from within IDEs
(IntelliJ, Eclipse).
• Define Objects using Object Spy tools, DOM
viewers, etc.
• Page-based test step creation.
• Insert visual validations and assertions.
• Typically takes longer to develop, more complex
(~6 hours per test).
• Medium-high code development skills required.
• Test changes are required proactively. Tests are more
error-prone to changes in production, objects.
• Tests are managed and maintained in an SCM tool
(GIT, Perforce).
• Tests are recorded with no coding in most
cases.
• Codeless UI tools used for creation.
• Objects are “learned” and generated on the fly.
• Test scenarios are often less structured and
more exploratory-based (flow-based).
• Time to author is shorter and can be ~1 hour
per test.
• Test reusability is easy.
• Business tester, little to no coding skills are
required.
• Tools support self-healing with auto test correction.
• Local or cloud test versioning is used. No GIT
integration.
SDET/DEVELOPERS – Code-Based BUSINESS TESTERS, DEVELOPERS - Codeless
Main Differences Between the Approaches
TEST AUTHORING
WORKFLOW &
SKILLSET REQUIRED
TEST
MAINTENANCE
15. 15 | How to Embed Codeless Automation into DevOps perfecto.io
SDET/DEVELOPERS – Code-Based BUSINESS TESTERS, DEVELOPERS- Codeless
Main Differences Between the Approaches
TEST EXECUTION
TESTING & APP
TYPES SUPPORTED
• Highly mature, includes samples, best
practices, documentation.
• Integrations exist for ALM tools, defect
management, etc.
• Functional, API, load, etc.
• Mobile native (Appium) and desktop web
(Selenium).
• Emerging technology, less mature, no well-
defined guidelines and practices (guide to
shift from standard to ML).
• Web is more mature than mobile codeless,
basic integrations only.
• Mostly functional (E2E) and basic API.
• Most support for web, mobile is lagging behind.
• Configured environment using tools like
TestNG Data Provider.
• Execution done locally, via CI, cloud-
based.
• Execution management is built into the
codeless tools.
• Execution done locally, via CI, cloud-
based.
TOOLS MATURITY
16. 16 | How to Embed Codeless Automation into DevOps perfecto.io
Code-Based AutomationCodeless Automation
17. 17 | How to Embed Codeless Automation into DevOps perfecto.io
5 Recommended Practices for
Embedding Codeless Testing in CT
1
2
3
Identify manual and/or flaky test steps.
Mark these as “codeless” to disconnect them from
the code-based automation suite (avoid duplications).
Record the tests in a “SMART” manner:
• Flow-based and exploratory recording.
• Reusable components.
• Robust object locators.
• Tagging for reporting and tracking.
B E S T P R A C T I C E
4 Connect codeless to CI and pipeline for continuous testing.
5 Identify maintenance and examination points for both suites.
18. 18 | How to Embed Codeless Automation into DevOps perfecto.io
Introducing Perfecto Codeless
19. 19 | How to Embed Codeless Automation into DevOps perfecto.io
Codeless Testing Legacy Tools Were Painful
L E G A C Y T O O L S M A N U A L T E S T I N G
R E C O R D E R S S C R I P T I N G
• Complex
• Time consuming
• Inefficient use of time
• Bottleneck in SDLC
• Limited capabilities
• Break easy
• Coding skills
• Demanding to maintain
20. 20 | How to Embed Codeless Automation into DevOps perfecto.io
Poll Question
Little to no test maintenance.
Record and playback without code.
Integration with CI and test
frameworks.
Advanced reporting capabilities.
All of the above.
What are you seeking most in codeless test automation?
21. 21 | How to Embed Codeless Automation into DevOps perfecto.io
Poll Results
22. 22 | How to Embed Codeless Automation into DevOps perfecto.io
Perfecto Codeless Testing Benefits
Recorders
• Full featured capabilities for additional
complex flows
• Robust AI-driven bindings that takes
care of 97.4% of app changes
Scripting
• No coding background necessary
to create automation scripts
• Reduce maintain cost of
automation
Setup
• No set up required
• Easy to learn
Process Fitness
• Efficiently reuse automation
scripts without the need to code
• Accelerates the testing process
and provides developers and
QA fast feedback
P E R F E C T O
C O D E L E S S
A U T O M AT I O N
23. 23 | How to Embed Codeless Automation into DevOps perfecto.io
A.I. Object Locator
Powerful Language Simplified & Smart Validations
Just Selenium
• No installations
• No modifications
• No plug-ins
• No extensions
• All platforms (Win, Mac, Linux)
• All browsers
• All versions
• Always updated, always ready
Multi-attribute
uniqueness matching
Over-time neural
network adaption
Framework &
app awarenessRun-time self-healing
P R O J E C T - L E V E L S H A R E D A D A P T I V E R E P O S I T O R Y
• Full browser control
• Object type awareness
• Data driven
• Functions
• Rich text validations
• Appearance & style validations
• Data validations
• Time validations
• Variables & data sets
• Branching
• JavaScript extensions
No installation,
no setup
Record on any
browser
Contextual point
& click recording
Object reusing
Interactive
debugger
T E S T S U I T E S
A S S E T S M A N A G E M E N T
P L AT F O R M S E T S
Integrated CI-connectors
S C H E D U L I N G
P R O J E C T M A N A G E M E N T
Perfecto Codeless
Web-Based Interactive Smart Recorder / Debugger Test Management
24. 24 | How to Embed Codeless Automation into DevOps perfecto.io
PERFECTO CODELESS SELENIUM
• Powerful web-
based IDE
• No coding skills
required
25. 25 | How to Embed Codeless Automation into DevOps perfecto.io
PERFECTO CODELESS SELENIUM
• Script debugging &
low maintenance
• Full integrations
26. 26 | How to Embed Codeless Automation into DevOps perfecto.io
PERFECTO CODELESS SELENIUM DEMO
1
2
3
4
Navigate to Amazon
Search in video games section for a game
Select a game
Add to cart
A M A Z O N S E A R C H & A D D V I D E O G A M E T O C A R T
F A C E B O O K C R E A T E A N E W U S E R
1 Create a new user
2 Data drive and run in parallel
27. 27 | How to Embed Codeless Automation into DevOps perfecto.io
Perfecto’s Smart Automation for Continuous Testing
SMART Test Execution
Fast and parallel test execution with
multi-team orchestration abilities and
management.
SMART ML-Driven Test
Reporting and Analysis
"Single pane of glass" provides
visibility and scales to support
millions of test results.
SMART Test Creation
Automated creation that matches your
team’s skillset (Appium, Espresso,
XCUITest, Quantum BDD).
SMART CLOUD Lab
Always on and stable. Always up
to date. Supports all mobile OS
and platforms.
The Perfecto
human factor
increases your
chance to succeed.
Black Belt
Testing Experts
Training
Dedicated
Success Manager
24/7 VIP
Support
28. 28 | How to Embed Codeless Automation into DevOps perfecto.io
About Perfecto
29. 29 | How to Embed Codeless Automation into DevOps perfecto.io
Perfecto Is a Perforce Company
A G I L E
P L A N N I N G
C O D E M G M T. &
C O L L A B O R AT I O N
E M B E D D E D
C O M P O N E N T S
O P E R AT I O N S
T O O L S
A U T O M AT E D
T E S T I N G
30. 30 | How to Embed Codeless Automation into DevOps perfecto.io
31. 31 | How to Embed Codeless Automation into DevOps perfecto.io
Thank You