Intelligent Testing with mabl provides adaptive tests and automated visual insights. Mabl uses machine learning to drive robust testing and regression analysis by simulating user journeys within applications. It trains on a user's interactions like elements clicked and pages viewed, then replicates the journey across environments to identify issues. Mabl also provides visual modeling of application states to detect differences between runs, helping evaluate test outcomes and evolve testing as applications change.

1. Intelligent Testing with mabl:
Adaptive tests and automated
visual insights
with Sergei and John

2. What is mabl?
Hi, I'm mabl
■ Founded in early 2017
■ SaaS test automation -
initial focus on web UI testing
■ Makes it easier to find and fix
bugs quickly
■ Machine learning to drive
robust testing and regression
analysis (performance,
errors, visual diffs)

3. ADAPTIVE USER-CENTRIC TESTS

4. Why adaptive testing
■ Scripting – brittle tests, high overhead
▲ Tightly coupled to changing UI elements
▲ Time writing scripts to minimize brittleness
▲ Time debugging brittle test failures
▲ Limited information for diagnosis
■ mabl approach
▲ Simulate user and test (end-to-end) journey
▲ Verification and insight into behavior of app
▲ Leverage machine intelligence and other machine strengths

5. Training a user journey
■ Capturing a user journey
▲ Domain-specific language to describe journey
▲ Evolving knowledge of how journey is achieved in app
■ What user interacts with
▲ Descriptive (element attributes)
▲ Context (ancestor element attributes, xpaths)
▲ Visual (image, bounding box, styling)
■ Expected behavior and states of app
▲ Descriptive (URL, title, page source, assets)
▲ Visual (screenshot, element bounding boxes)
▲ Timing (loading time, latencies)

6. Simulating the journey
■ Configuring and parameterizing
▲ Across environments, user types, app configurations
■ Identifying the right UI element
▲ Canonical identification
▲ Ranking partial/uncertain matches
■ Experimentation (multiple attempts)
▲ No need to craft best xpath/selector
▲ Robust to large UI revisions
▲ Broader coverage

7. Evaluating outcomes
■ Strong signals about action equivalence
▲ Assertions
▲ State transitions
▲ Comparison across runs and
environments
■ State identification
▲ Visual appearance
▲ Descriptive properties
(URL, title, assets loaded)
▲ Available actions

8. Evolving testing with the app
■ Results of adaptation
▲ Automatic updates
▲ Push-button (verify) update
▲ Retraining as a last resort
■ Continuous learning
▲ Incremental change
▲ A/B testing
▲ Transfer (across journeys, environments, users)

9. VISUAL MODELING OF APP STATES

10. can you
SPOT THE DIFFERENCE?

14. benign changes

19. interesting changes

23. Roadmap: Near future: Learn dynamic areas
■ Learn areas of dynamic or periodic content
■ Model degree and types of expected change in dynamic areas
■ User feedback and control
▲ Should be added to the baseline (new feature, changed style)
▲ Static -> dynamic
▲ Dynamic -> static

24. Demo Time!