This document summarizes Spotify's approach to continuous delivery and building quality into software development. It discusses that quality involves meeting stakeholder expectations, and the importance of quality to reduce costs from bugs. Spotify emphasizes automated testing from unit to integration levels. They implement a delivery pipeline to continuously integrate and deploy code. Other practices include code reviews, monitoring, and allowing employees time for experiments.
2. 2
Spotify brings you the right music for
every moment!
Started in 2006 (in Sweden)
Now 1500+ employees, 500+ engineers
Over 30 million songs available
Over 20,000 songs added every day
5 development centres across the globe
11. 11
Step 1 - Prototyping phase
Technical spike investigation
Hacking different Back-End solutions
Ad-hoc design discussions
Stub implementation of Front End
Getting early user’s feedback on Front End
13. 13
Step 2 - Setting delivery pipeline
Add more unit tests
Add more integration tests
Add more functional tests
Automate deployment configuration
Add dashboards and system monitoring
Clarifying End User Needs
23. Testable micro-services architecture
23
Unit Tests
Component Tests
Functional
Tests
System/End-2-End Tests
Integration Tests
Unit Tests
Component Tests
Functional
Tests
JS Unit Tests
JS Component Tests
UI
Functional
Tests
24. 24
Unit Tests
Check logic of minimal code snippet
with no or mocked dependencies
https://github.com/mockito/mockito
https://github.com/junit-team/junit
26. 26
Client App
Fake Back-End Server
Spotify Back End
Fake back-end for client apps
http://jsonstub.com
https://github.com/azagniotov/stubby4j
https://github.com/dreamhead/moco
27. 27
Integration Tests
Check an integration between components
You do not need whole system to check the integration
https://github.com/spotify/helios/blob/master/docs/testing_framework.md
34. 34
Was not covered during this talk …
Load Testing
Gradual Rollouts
A/B Testing
Feature Toggles
Monitoring/Alerting
Information Radiators
… but stay tuned
35. 35
Technical Test Engineer Test Engineer
! = Manual Tester! = Test Automator
~ Software Engineer in Test
Test Engineering Roles and Responsibilities
~ Context Driven Tester
36. 36
Technical Test Engineer
Work as a software developer
Advocate testability of the product
Argue on software design with software engineers
Help with building new/injecting existed development tools
38. 38
Knows business domain
Focused on exploring the product
Free to use any programming language to test specific use case
Free to use any programming language to automate his work
Test Engineer
39. 39
Mind Map as a Tool
https://www.mindmup.com
Product tree
Scenarios
Playbooks
Checklists
Session notes
42. 42
test ideas during healthy
discussions
test requirements via end
users collaboration
discuss system design and
conner cases earlier in the
planing/design meetings
define definition of done
Break Uncertainty
43. 43
But have responsible person to figure out what consensus means in each case
Use Google Docs collaboration for finding group consensus
51. 51
Ready for a journey?
Make Quality explicit
Find Quality promoters
Define your way of Quality
improvements
Set right Quality constraints
Share results as early as possible
Keep looking further quality improvements
Probably you will never end :)