46. 1. Choose few technologies
2. Invest in system
3. Cattle not pets
4. Centralized Logging
5. ABQ
6. Engage product during prod issues
7. HQ Dashboards
8. Use percentiles
9. Codify your SOPs
10.Develop a learning org
2 major product lines…
Analytics - See how your apps are doing
Marketing - setup engagement capaigns and measure the effectiveness or LIFT
Process TB of data daily
We store PB of data
This diagram represents 30% of our architecture
This diagram represents 30% of our architecture
This diagram represents 30% of our architecture
We have no dedicated OPS and QA functions or gatekeepers.
We have a dedicated tools team, but developers deploy their own code to production
We held the line. There has been many times we have hired both roles. The urge typically follows a production issues.
There are two major points of this presentation.
Your total operational costs is the sum of all technologies
New technologies have inherent known-unknowns or unknowns-unknowns
Invest, Invest, Invest in systems to make developer productive and to do the right thing
Invest in getting code to production fast safely
Invest in Fast Tests
Invest in tooling around making everyone a force multiplier
Invest in getting code to production fast safely
Invest in Fast Tests
Invest in tooling around making everyone a force multiplier
Invest in getting code to production fast safely
Invest in Fast Tests
Invest in tooling around making everyone a force multiplier
When we moved to our microservices strategy, we invested substantially in to help us deploy
Pets = You name them. They are all special snowflakes. They hold state.
Cattle = you care about the total milk production, not about an individual cow
Pets = You name them. They are all special snowflakes. They hold state.
Cattle = you care about the total milk production, not about an individual cow
When you start off, it is okay to have a couple Pets.
When you start off, it is okay to have a couple Pets.
Do not under estimate the cost of logging and monitoring
Communicate via the queues everywhere
Increase costs and increased development time
Better guarantees
Allows flexibility for downtime
Communicate via the queues everywhere
Increase costs and increased development time
Better guarantees
Allows flexibility for downtime
Proactively address production issues that might languish if another team was responsible for maintaining their services in production
Help them understand the pain of poor quality and technical debt
This culture of ownership yields certain incentives and priorities for the team.