Unit testing is mostly a solved problem, but how do you write tests for the visual side of your app—the part that your users actually see and interact with? How do you stop visual bugs from reaching your users?
We will dive deep into visual regression testing, a fast-growing technique for testing apps pixel-by-pixel. We will integrate perceptual diffs in Rails feature specs, and learn how to visually test even complex UI states. We will show tools and techniques for continuous visual integration on every commit, and learn how to introduce team visual reviews right alongside code reviews.
20. QA is necessary,
but complicated, slow, and manual.
And impossible to catch everything...
Medium sized app
60+ models
Hundreds of flows
Thousands of possible page states and permutations
Constant feature churn
24. describe 'posting a project', type: :feature do
it 'creates a new project' do
visit '/'
fill_in 'Title', with: 'My project'
click_button 'Post Project'
expect(page).to have_content('Project created!')
expect(page).to have_content('My project')
end
end
25. describe 'posting a project', type: :feature do
it 'creates a new project' do
visit '/'
fill_in 'Title', with: 'My project'
click_button 'Post Project'
expect(page).to have_content('Project created!')
expect(page).to have_content('My project')
end
end
Doesn’t fail! The button still technically “works”.
26. describe 'posting a project', type: :feature do
it 'creates a new project' do
visit '/'
fill_in 'Title', with: 'My project'
click_button 'Post Project'
# ???
expect(page).to have_content('Project created!')
expect(page).to have_content('My project')
end
end
27. describe 'posting a project', type: :feature do
it 'creates a new project' do
visit '/'
fill_in 'Title', with: 'My project'
click_button 'Post Project'
# Assert that…CSS computed style…color of…
# is not equal… to… ???
expect(page).to have_content('Project created!')
expect(page).to have_content('My project')
end
end
68. Back to our goal:
What if we could see
every pixel changed
in any UI state
in every PR?
69. That would require:
- Fast. Really fast. As fast as your test suite.
- Handle complex UI states
- Continuously integrated with your
workflow on any commit.
71. describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
end
end
72. describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
end
end
73. describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page, name: 'homepage')
end
end
74. describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
Percy
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
75. describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end Images??
Percy
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
76. describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end Noooooo...
Percy
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
77. describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end DOM / HTML
snapshots!
Percy
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
78. DOM / HTML
snapshots!
(SHA256
fingerprinted)
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
Percy
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
79. percy-worker a
mitmproxy
percy-worker a
mitmproxy
percy-worker a
mitmproxy
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end
describe 'app features', type: :feature do
it 'shows the dropdown menu when clicked' do
visit '/'
first('.dropdown-toggle').click
Percy::Capybara.snapshot(page)
end
end DOM / HTML
snapshots
percy-hub
percy-api
percy-worker a
mitmproxy
82. Visual testing is:
- Possible.
- A new way to think about testing.
- Helps give you deployment confidence.
83. Visual testing is:
- Possible.
- A new way to think about testing.
- Helps give you deployment confidence.
Much more work to be done to make it a
mainstream engineering practice!