Development, Deployment & Collaboration at Etsy

•

5 recomendaciones•1,633 vistas

Daniel Schauenberg

Tecnología

http://www.ﬂickr.com/photos/brianglanz/1095706242

How comfortable
are you deploying
a change right
now?

@mrtazzhttp://www.ﬂickr.com/photos/renaissancechambara/2349811492
small change

“If this is your first
day at Etsy, you
deploy the site”

@mrtazz
Developer VMs
• KVM
• Every engineer has one
• Fully Chef’d with the Etsy Stack
• Different sizes and Chef roles

@mrtazz
Continuous Integration
• Run set of tests before each deploy
• Full QA suite
• Princess/Production smoker tests
• Try (yup, there is one)

http://www.ﬂickr.com/photos/egfocus/6962179321

@mrtazz
The Bobs
• LXC virtualized hosts
• 14/physical hosts
• Spread over 3 SSDs
• Most of them attached to try

@mrtazz
Deployinator
• 2 Buttons, no ambiguity
• Overview of current state of deploy
• Links to Logwatcher and Dashboards
• Easy to add stacks for new tools to deploy

http://www.ﬂickr.com/photos/jbgeronimi/6363087361

@mrtazz
Monitoring
• Devs do their feature monitoring
• Everybody can access all the graphs
• Dashboard All The Things!
• Stream All The Logs!

If you are writing
code, you are
on-call

@mrtazz
On-Call Schedules
• ops on-call
• dev on-call
• payments on-call
• support on-call

@mrtazz
Dev On-Call
• On-call for 3 days
• All developers who are not in another
rotation
• L1 and L2 escalations
• L1 if it’s your first time

@mrtazz
Incident Response
• “This graph looks funny”
• “Hey I just got paged for elevated error rate
after deploys”
• “Supergrep is going crazy!!”

@mrtazz
#warroom
• only outage related conversations
• coordinate investigation, communication,
countermeasures and monitoring
• good place to lurk for new engineers

@mrtazz
Summary
• These are things that work for *us*
• Culture is an on-going effort
• Share everything
• Encourage learning/teaching

@mrtazz
Summary
• Lunch ’n learns
• DC visits
• On-call for a day
• Bootcamps/Senior rotations

codeascraft.com

etsy.com/codeascraft/talks

etsy.github.com

etsy.com/careers

Development,
Deployment and
Collaboration at Etsy
Daniel Schauenberg
dschauenberg@etsy.com

Más contenido relacionado

La actualidad más candente

Concourse and DatabaseGwenn Etourneau

Practical Continuous Deployment - Atlassian - London AUG 18 Feb 2014Matthew Cobby

VersaPay's Tools for Happyfficient DevelopersPhilippe Creux

Monitoring in the cloud with PuppetKris Buytaert

My Top 5 Favorite GemsJimmy Ngu

PuppetConf 2017: Using Puppet Enterprise APIs with PowerShell- Jason Johnson,...Puppet

Data + Visualization Toronto Meetup 4: The Google Maps APImattholtom

Railsconf 2014 - Deploying Rails is Easier Thank It Lookstalkingquickly

Update Smart: Use a Staging Site!Meagan Hanes

PuppetConf 2017: No Server Left Behind - Miguel Di Ciurcio Filho, InstructPuppet

Dev-Friendly OpsJosh Schramm

What's New in JHipsterLand - DevNexus 2017Matt Raible

The Ruby workflowClemens Helm

Stripe con 2021 UI stackMaximeRainville1

From Chef to Saltstack on Cloud Providers - Incontro DevOps 2015Corley S.r.l.

Modern Front End Tools & WorkflowAdir Amsalem

Background processes and tasks in an async worldparticlebanana

Anton Boyko "The future of serverless computing"Fwdays

React for .net developersmacsdickinson

Anton Boyko, "The evolution of microservices platform or marketing gibberish"Sigma Software

La actualidad más candente (20)

Concourse and Database

Practical Continuous Deployment - Atlassian - London AUG 18 Feb 2014

VersaPay's Tools for Happyfficient Developers

Monitoring in the cloud with Puppet

My Top 5 Favorite Gems

PuppetConf 2017: Using Puppet Enterprise APIs with PowerShell- Jason Johnson,...

Data + Visualization Toronto Meetup 4: The Google Maps API

Railsconf 2014 - Deploying Rails is Easier Thank It Looks

Update Smart: Use a Staging Site!

PuppetConf 2017: No Server Left Behind - Miguel Di Ciurcio Filho, Instruct

Dev-Friendly Ops

What's New in JHipsterLand - DevNexus 2017

The Ruby workflow

Stripe con 2021 UI stack

From Chef to Saltstack on Cloud Providers - Incontro DevOps 2015

Modern Front End Tools & Workflow

Background processes and tasks in an async world

Anton Boyko "The future of serverless computing"

React for .net developers

Anton Boyko, "The evolution of microservices platform or marketing gibberish"

Similar a Development, Deployment & Collaboration at Etsy

Continuous Integration, the minimum viable productJulian Simpson

Continuous TestingCrevise Technologies

Continuous DeploymentBrian Moon

Pragmatic Continuous Delivery - ReaktorDevDay 2012Neeme Praks

Untangling spring week10Derek Jacoby

Devops For DrupalKris Buytaert

Maintaining Retainers as a WordPress DeveloperDevinVinson

Great Tools Heavily Used In Japan, You Don't Know.Junichi Ishida

Production Ready WordPress #WPLDNEdmund Turbin

Devops Devops Devops, at FrosconKris Buytaert

Take control of your Jenkins jobs via job DSL.Łukasz Proszek

Continuous Delivery of Puppet ManifestsKris Buytaert

Frontend Engineer ToolboxYnon Perek

Essential Plone development tools - Plone conf 2012Kim Chee Leong

Vagrant for Effective DevOps CultureVaidik Kapoor

stackconf 2022: Infrastructure Automation (anti) patternsNETWAYS

Infrastructure as Code PatternsKris Buytaert

Task runners + theming automating your workflowJoshua Gilmer

Ship It ! with Ruby/ Rails EcosystemYi-Ting Cheng

Lessons Learnt in 2009pratiknaik

Similar a Development, Deployment & Collaboration at Etsy (20)

Continuous Integration, the minimum viable product

Continuous Testing

Continuous Deployment

Pragmatic Continuous Delivery - ReaktorDevDay 2012

Untangling spring week10

Devops For Drupal

Maintaining Retainers as a WordPress Developer

Great Tools Heavily Used In Japan, You Don't Know.

Production Ready WordPress #WPLDN

Devops Devops Devops, at Froscon

Take control of your Jenkins jobs via job DSL.

Continuous Delivery of Puppet Manifests

Frontend Engineer Toolbox

Essential Plone development tools - Plone conf 2012

Vagrant for Effective DevOps Culture

stackconf 2022: Infrastructure Automation (anti) patterns

Infrastructure as Code Patterns

Task runners + theming automating your workflow

Ship It ! with Ruby/ Rails Ecosystem

Lessons Learnt in 2009

Más de Daniel Schauenberg

Human Factors and PostMortemsDaniel Schauenberg

Deploy, Collaborate and ListenDaniel Schauenberg

Scaling Deployment at EtsyDaniel Schauenberg

Feature Flagging your Infrastructure for Fun and ProfitDaniel Schauenberg

Scaling Deployment at EtsyDaniel Schauenberg

StatsD Workshop Monitorama 2013Daniel Schauenberg

Etsy chef-workflowDaniel Schauenberg

Más de Daniel Schauenberg (7)

Human Factors and PostMortems

Deploy, Collaborate and Listen

Scaling Deployment at Etsy

Feature Flagging your Infrastructure for Fun and Profit

Scaling Deployment at Etsy

StatsD Workshop Monitorama 2013

Etsy chef-workflow

Último

MINDCTI Revenue Release Quarter One 2024MIND CTI

Six Myths about Ontologies: The Basics of Formal Ontologyjohnbeverley2021

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...apidays

Elevate Developer Efficiency & build GenAI Application with Amazon QBhuvaneswari Subramani

TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc

WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2

Understanding the FAA Part 107 License ..Christopher Logan Kennedy

Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelDeepika Singh

Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1

Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software

Finding Java's Hidden Performance Traps @ DevoxxUK 2024Victor Rentea

[BuildWithAI] Introduction to Gemini.pdfSandro Moreira

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...apidays

CNIC Information System with Pakdata Cf In Pakistandanishmna97

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra

FWD Group - Insurer Innovation Award 2024The Digital Insurer

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93

Why Teams call analytics are critical to your entire businesspanagenda

Development, Deployment & Collaboration at Etsy

1. Development, Deployment and Collaboration at Etsy Daniel Schauenberg dschauenberg@etsy.com @mrtazz

3. @mrtazz Etsy Stats

4. @mrtazz Etsy Stats

5. @mrtazzItem by TheBackPackShoppe

6. http://www.ﬂickr.com/photos/brianglanz/1095706242

7. avg 50 deploys/ day

8. avg n > m deploys/ day

9. How comfortable are you deploying a change right now?

10. @mrtazzhttp://www.ﬂickr.com/photos/renaissancechambara/2349811492 small change

11. Config Flags Item by RocajoStudio

12.

13. “If this is your first day at Etsy, you deploy the site”

14. Developer VMs

15. @mrtazz Developer VMs • KVM • Every engineer has one • Fully Chef’d with the Etsy Stack • Different sizes and Chef roles

16.

17. Continuous Integration

18.

19. @mrtazz Continuous Integration • Run set of tests before each deploy • Full QA suite • Princess/Production smoker tests • Try (yup, there is one)

20. http://www.ﬂickr.com/photos/egfocus/6962179321

21. @mrtazz The Bobs • LXC virtualized hosts • 14/physical hosts • Spread over 3 SSDs • Most of them attached to try

22.

23. Item by decomodwalls

24. Deployinator

25. @mrtazz Deployinator • 2 Buttons, no ambiguity • Overview of current state of deploy • Links to Logwatcher and Dashboards • Easy to add stacks for new tools to deploy

26. http://www.ﬂickr.com/photos/jbgeronimi/6363087361

27.

28. Monitoring

29. @mrtazz shouldigraphit.com

30. @mrtazz Monitoring • Devs do their feature monitoring • Everybody can access all the graphs • Dashboard All The Things! • Stream All The Logs!

31.

32.

33.

34. On Call

35. If you are writing code, you are on-call

36. @mrtazz On-Call Schedules • ops on-call • dev on-call • payments on-call • support on-call

37.

38. @mrtazz Dev On-Call • On-call for 3 days • All developers who are not in another rotation • L1 and L2 escalations • L1 if it’s your first time

39. Incident Response

40. @mrtazz Incident Response • “This graph looks funny” • “Hey I just got paged for elevated error rate after deploys” • “Supergrep is going crazy!!”

41. Is the site down?

42.

43. #warroom

44. @mrtazz #warroom • only outage related conversations • coordinate investigation, communication, countermeasures and monitoring • good place to lurk for new engineers

45. Post Mortems

46. blameless

47. Everybody’s invited

48. Learning Opportunity

49. Summary

50. @mrtazz Summary • These are things that work for *us* • Culture is an on-going effort • Share everything • Encourage learning/teaching

51. @mrtazz Summary • Lunch ’n learns • DC visits • On-call for a day • Bootcamps/Senior rotations

52. codeascraft.com etsy.com/codeascraft/talks etsy.github.com etsy.com/careers

53. Questions?

54. Development, Deployment and Collaboration at Etsy Daniel Schauenberg dschauenberg@etsy.com