Databases create a real challenge for automation and dealing with database deployments is a complex process. Databases contain our most valuable information, business data, which must be preserved and protected at all costs and yet the automation processes for database deployment are not widely adopted.

1. Continuous Delivery for Databases
SQL Relay – Cardiff - 2014
Presented by: James Smith
Co-Founder, DevOpsGuys

2. Who is this guy?
ABOUT ME
Co-founder @ DevOpsGuys
Two decades building & delivering enterprise web systems
Helped many, many companies implement Continuous Delivery practices
Found High Quality Belgian Beer in late nineties!

3. ABOUT DEVOPSGUYS
We provide
web application management
for customers, giving them access to the expertise necessary for
building, launching, maintaining & optimising applications
allowing them to
accelerate time-to-market
and to
focus on adding value to their business
Who are these guys?

4. Why is this important?
WHY?

5. Why is this important?
WHY?

6. BUSINESS HAS CHANGED

7. SPEED MATTERS

8. 8

9. DEPLOYMENT RATES
9
300 Deployments / Year
10+ Deployments / Day
50-60 Deployments / Day
Every 11.6 seconds

10. HIGH PERFORMERS
HIGH PERFORMING IT
ORGANISATIONS DEPLOY 30 TIMES
MORE FREQUENTLY, WITH 50%
FEWER FAILURES WITH 8000X
FASTER LEAD TIMES THAN THEIR
PEERS.
10

11. HIGH PERFORMERS
THEY ARE ALSO TWO TIMES MORE LIKELY TO
EXCEED PROFITABILITY, MARKET SHARE &
PRODUCTIVITY GOALS
THEY EXPERIENCE 50% HIGHER MARKET
CAPITALIZATION GROWTH OVER 3 YEARS
11

12. How have things changed?
HOW?
12

13. 2010
2012
DEV HAS CHANGED
2001 (1998)
2001 (and earlier)

14. WE KNOW AGILE WORKS

15. CONTINUOUS X?
Plan Code Build Test Release Deploy Operate
15
Continuous Integration
Continuous Delivery
Continuous Deployment

16. What is it?
CONTINUOUS DELIVERY

17. Matthew Skelton
REGULAR, RAPID, RELIABLE AND
CONTROLLED DELIVERY OF
WORKING SOFTWARE SYSTEMS
INTO PRODUCTION

18. Matthew Skelton
REGULAR, RAPID, RELIABLE AND
CONTROLLED DELIVERY OF
WORKING SOFTWARE SYSTEMS INTO
PRODUCTION

19. Nhan Ngo, a QA engineer at Spotify

20. CONTINUOUS DELIVERY
1. Build Management & CI
2. Environments & Deployment
3. Release Management & Compliance
4. Testing
5. Data Management
20

22. BUT DATABASE CHANGE IS
SCARY
22
Photo Credit: http://cache.lego.com/r/www/r/movie/-
/media/franchises/the%20lego%20movie/explore/downloads/wallpapers/lego_wps_1600_emmet.
jpg?l.r=-1410608028

23. NO, THIS IS SCARY
Photo Credit: ?

24. SO WE MINIMISE RISK
Photo Credit: https://ideas.lego.com/projects/62456

25. WE DEFEND THE THREAT
Photo Credit: "Rumrunner" via Compfight cc

26. WE STOP CHANGE
26
Photo Credit: s.kosoris via Compfight cc

27. CUSTOMERS GET SAD
27
Photo Credit: Kalexanderson via Compfight cc

28. THE BUSINESS GETS ANGRY
28
Photo Credit: powerpig builds via Compfight cc

29. AND IT GET THE BLAME!
29
Photo Credit: bobsfever via Compfight cc

30. YES BUT….
Dev are shipping application changes – frequently
Ops are creating servers - automatically
30

31. Perfectly normal question
WHY IS THERE SO MUCH LEGO
IN YOUR SLIDE DECK?

32. LEGO = DEVOPS
Rapid proto-typing & experimentation
Building blocks – no right or wrong way
Promotes collaboration
Strong cultural appeal
Small batch sizes
Visibly measureable
We’ve even automated it!
 Manufacturing
 Zenon
 Mindstorms
Danish phrase leg godt, which means "play well".

33. Why is database automation so hard?
WHY?

34. != CTRL C, CTRL V
Database deployment is not copying and replacing.
It is the transformation from a previous version to the next version while
preserving data integrity.
Deploying database change is hard
Deploying database change frequently is even harder

35. What are the common challenges?
WHAT?

36. #1: VERSION CONTROL
Photo Credit: ntr23 via Compfight cc

37. VERSION CONTROL
 Database changes not under VCS
 Worse – Changes not “always” committed to VCS
Communication of change
Living in “the sea of ‘branch/merge’ filth”

38. #2: AGAINST THE FLOW
38
Photo Credit: slang589 via Compfight cc
Photo Credit: slang589 via Compfight cc

39. AGAINST THE FLOW
DEV PRODUCTION
39
DEV PRODUCTION
CODE
DATA

40. #3: COMPLEXITY
40
Photo Credit: L@go via Compfight cc

41. COMPLEXITY
Dependence
Centralisation

42. #4: DATA VOLUME
Photo Credit: http://www.calgaryherald.com/news/calgary/Gallery+LEGO+KidsFest/9848406/story.html

43. DATA VOLUME
Transit time
Historical data – no archiving
One of the biggest impacts on Cycle time

44. #5: CONWAY'S LAW
Photo Credit: kirk_arts via Compfight cc

45. CONWAY'S LAW
"Any organization that designs a system (defined more broadly here than
just information systems) will inevitably produce a design whose structure is
a copy of the organization's communication structure.“
- Conway, 1968
45

46. CONWAY'S LAW
Is (was) a database really needed? 3 Tiers anyone?
Centralised vs Decentralised?
Formalised [change] control

47. So what are some of the solutions?
HOW?

48. #1: GET UNDER CONTROL
Photo Credit: RHiNO NEAL via Compfight cc

49. VERSION CONTROL
It’s code – it should be in VCS!
It’s code – it should be in VCS!
It’s code – it should be in VCS!
Schema & Static/Reference Data
Reverse engineer existing schema & reference data

50. #2: INTEGRATE
CONTINUOUSLY
Photo Credit: ....Tim via Compfight cc 50

51. CONTINUOUS INTEGRATION
 Automate your build steps (Full vs Incremental)
Build a library of manual tests
Automate your tests
51

52. #3: AUTOMATE TESTING
52
Photo Credit: s3aphotography via Compfight cc

53. TEST, TEST, TEST
Select the right tests for each stage;
Unit testing
 Integration Testing
Deployment Validation
Behaviour Validation
Determine the right data for testing
Do you need it all?
53

54. #4: AUTOMATION
Photo Credit: pasukaru76 via Compfight cc 54

55. AUTOMATE ALL THE THINGS
Testing
Deployment
Back-up and more importantly restore
Archiving
Rollback
55

56. #5: MICROSERVICES
56

57. AS A SERVICE
Treat your database as a service
Contracts – that you do not break (lightly)
57

58. RE-ARCHITECT
Split data along sensible partitions
Polyglot persistence
58

59. POLYGLOT PERSISTENCE
59

60. #6: CHANGE YOUR WAYS
Photo Credit: Stéfan via Compfight cc 60

61. IT’S CULTURE
Technology is only half of the story
Dev’s must work with DBA’s (no silo’s)
Management must think of operations as part of development
Deployment is part of development
Data retention is part of development
Fail faster, but fail safely
61

62. Can tools help?
TOOLING
62

63. THE PIPELINE
Source
Control
Integration Test Deploy

64. IN REALITY
64
Development
Operations
Plan Code Build Test Release Deploy Operate

65. SOURCE CONTROL
Source control is the foundation of Continuous Integration
Plus many, many more

66. CONTINUOUS INTEGRATION
Continuous Integration is the common orchestration point
66

67. DEPLOYMENT
Repeatable and consistent deployments every time.
67

68. MONITORING
Continuous delivery elevates the need for monitoring in production

69. THE TRUTH
Continuous Delivery relies on having 2 basic things;
1. Version Control
2. Automation

70. AND FINALLY…
Customers see results and new features more quickly.
Shorter feedback cycles increases our ability to learn.
Improve the whole system.
Reduce firefighting.
Everyone wins!
70

71. EVERYBODY WINS!
Photo Credit: Kalexanderson via Compfight cc 71

72. THAT’S ALL FOLKS
Photo Credit: Walter Benson via Compfight cc 72