Enterprises' transition to Agile development has led to a dramatic increase in the number of releases and development cycles. It has also intensified the challenges of controlling, tracking, and auditing changes to the application and database. This has led to critical errors, data loss, and costly downtime.

1. How Financial
Institutions Must Enforce
DevOps Organizations
Policy

2. Agenda
▪ Continuous Delivery at enterprise scale
▪ The reasons why the database is often left behind
▪ Best practices of Continuous Delivery for modern applications
▪ The way to achieve the optimal delivery pipeline – including the database
▪ Q&A

3. Presenter
Yaniv Yehuda
Co-Founder & CTO at DBmaestro
Spent the last years raising awareness about the
challenges around database development and
deployment, and how to support database
Continuous Delivery.

4. This is me I like going fast, but I do need to manage risks…

5. About DBmaestro
▪ DevSecOps for databases:
▪ Database version control
▪ Release management
▪ Security management

6. DevOps!
Everyone is talking
about it!

7. Financial Services Business Demands
Providing best customer experience in a highly competitive market:
Gaining customer’s loyalty in the mobile and hyper-connected world
In one of the most regulated sectors
Customers are looking for:
▪ Intuitive service over any application
▪ Device agnostic connectivity
▪ Reliability - you cannot be too careful about their money
Implications on Development, QA and Delivery:
release applications fast, safe and NEVER make errors!

8. ARA on the Rise, Database Lagging Behind
Manual work:
can’t scale, can’t match CD frequency, not repeatable, prone to error
* Dzone DevOps survey 2016

9. Fail fast fail often – continuous
improvement
Code is easy...
Building and releasing code
▪ Code can be replicated, branched and merged
▪ Best practices suggest a single build process
▪ Release process is done by copying compiled code &
executables

10. Fail fast? Pay the price…
Database is not that simple
Building and releasing databases
▪ A database is not a collection of files on the file system
▪ Each database environment contains a copy of the configuration
▪ Inconsistencies between environments
▪ Configuration drifts / out of process changes…
▪ Release process is incremental
▪ Cannot copy a DB from QA to Prod…
▪ Database upgrade and rollback scripts are separate from DB configuration

11. Pay the price
▪ Big Bank correlates 60% of release
issues to the database
− Managing environments, automating
▪ Banking app failing for a day! - Index
removed from production
− Choosing the right tools to automate with
▪ Disastrous down time - 5gb Table
dropped from production…
− Controlling release process

12. X
Int QA Stage Prod
Dev
Dev
Dev
Model
‘Break Glass’
Out of Process
Change
X
X
X
X
X
X
Configuration drift…
So how can we automate? This is exactly why we are doing things
manually!
The risk of over simplifying
automation

13. The old days...
▪ Waterfall : Consolidating changes to one big release
− Every 3-6-9 months?
▪ Being BIG
− Big dev cycles
− Big QA efforts
− ‘Getting ready’ for the Big release well in advance
▪ Spending time evaluating changes
− Developer/ ADBA introducing changes, building db upgrade scripts
− DBA evaluates, suggests improvements or approves
− DBA pushes changes forward / runs on higher environments

14. The old days... Safe but slow
▪ The Dev team is responsible
− For creating logical changes to the app/db
▪ The DBA is responsible
− For db changes code reviews (especially around high risk areas)
− For handling rollout and rollout risks
− For the health and continuous operation of the db
▪ Problem? Slow process…

15. Modern days – going faster
▪ Agile : small focused
− Every 2-3-4 weeks?
− Continuously?
▪ CD/CD
− Small/atomic changes
− Quick feedback loops (unit tests, automated tests…)
− Small changes being quickly pushed all the way to (pre)
production
▪ Evaluate changes?
− Dev team often says DBA team is a bottle neck…
(numbers game…)

16. Going faster – managing risk
Who is responsible??
− Dev teams wants to be agile and self-sufficiency
• Be responsible for pushing their changes forward
− DBAs are responsible for the health and continuous operation
of the db
− Who is responsible for handling rollout and
rollout risks?
• Evaluating changes?
• Managing risk?
• Blamestorming is inevitable…

17. Tension & Risk
▪ Dev team: “we accepts responsibility”
▪ DBA team: “we will most likely be one
you turn to in trouble…” {…you
shouldn't have gotten into in first place}
▪ Enter: DevOps!

18. DevOps: going forward fast, yet safely
DevOps for databases
• Cross responsibilities - Dev team, DBA, Release manager, DevOps expert…
• Creating a repeatable processes
– Release Automation
– Handling exceptions
• Setting ground rules – what goes where? when?
• Setting process controls – handling risk: policies & roles
• Managing Risk

19. Int QA Stage Prod
Dev
Dev
Dev
Model
‘Break Glass’
Out of Process
Change
Validate
OR
Validate
Validate
Dev Goal: effective and
productive
Ops Goal: safe, predictable, scalable &
controlled
Any
Source
Of
change
Balancing Dev & Ops

20. Database Release Automation challenges
Releasing
▪ Different sources of change (IDEs, scripts, App Setup)
▪ Simplistic automation potentially breaking environments
▪ Requires skill, diligent and invest a lot of efforts
▪ Dealing with configuration drifts
▪ Out of process changes & break glass scenarios
▪ How are we validating Success?
▪ And control Policies?
▪ Garbage in, garbage out?
▪ Achieving Compliance and Auditing

21. QA Stage Prod
‘Break Glass’
Out of
Process
Change
Validate
Validate
1. Comply to Policy rules
1. Enforce security roles
2. Prevent non-policy updates
2. Validate pre-release configuration
1. Stop automation
2. Notify Drift
3. Suggest resolution
3. Execute upgrade
1. Audit changes
4. Validate post-release results
1. Activate tests
2. Enable rollback if required
5. Alert or prevent out of process
changes
Safe, Fast, Stable, Repeatable, Scalable
and Secure Database Automation
Validated & Safe Automation

22.  Need to answer: Who? What? Where? When? Why?
 Security roles – Control who can do what and where
 Audit - Who did what when and why
Security, Compliance & Audit

23. Policy Management
▪ Define the policy rules to
govern changes on a per-
environment basis
− Incoming changes, whether in scripts or
interactive changes are measured against
policy rules
− What? Where? When?
− Prevent execution of risky or non-policy
code (prevent dropping of a column,
prevent removal of an index in prod etc)

24.  Jenkins, Bamboo, TeamCity, CircleCi, CA-ARA, XebiaLabs, Automic, Jira
Heterogeneous CD Pipeline

26. Q&A

27. Connect with us

28. Thank
you