SlideShare a Scribd company logo
1 of 21
Download to read offline
Chrome Release Cycle




 Author: Anthony Laforge (laforge@chromium.org)
Our Philosophy

 Think of any major website, even the fancy Web 2.HTML5
 ones... do they have version numbers?
 We took the same approach to our client software as an
 online web service.
 That is... we treat releases as a means of getting features
 out to users and not goals in and of themselves.
 It's about flow.
How Our Users Work

 Users sign up for Chrome release channels based on the
 level of stability they want.
 Updates happen automatically in the background.
 New features and functionality just flow in without any effort
 on the user's part.
How the Chrome Team Works


 Developers work almost exclusively on a single
 central trunk (possible due to our try bots and continuous
 build infrastructure)
 Our branches stem from points on the trunk
 We stabilize branches by pulling in changes from trunk (i.e.
 everything lands on trunk first)
Where we started (v1-4) - On Paper
In practice, for that final beta...we'd

  Cut a branch (when we thought we had everything)
  Merge about ~500 patches (since we didn't have
  everything)
  Spend weeks stabilizing and re-stabilizing issues.
  Ended up working 1-3 months to get a release out the door,
  always certainly missing our 13 week plan.
  And at the end finally shipping something we were happy
  with... but that left us pretty drained (i.e. the bad flow)
The chain of events

  We'd hold releases, sometimes our branch points, until
  features were done, which meant...
  Lots of changes got queued up on the trunk, which
  caused...
  Engineers merging lots of changes to the branch, which led
  to...
  Instability on the branch that we had to deal with and that
  meant...
That chain of events...

  Long lived branches, which...
  Made supporting the branch (i.e. merging changes) more
  difficult as we drifted further from the trunk...
  All of which led to...
       Unpredictable release cycles, with date targets we could
       never hit, and....
       Engineers who were always in a rush to get their features
       into "this" release since we couldn't make any promises
       about the schedule
Which led me to think...

  I need a long vacation.
  There has to be a better way.
  What's causing things not to flow smoothly?




                             ...Yes...In that order.
Primary Goals
 Shorten the release cycle and reduce the life span of a
 branch (make merges easier)
 Make releases more predictable and easier to scope
 Reduce the strain on the entire team




  Good                             Bad
Goal to Plan - Getting down the cycle

  Originally set out a plan to simplify the 12 week cycle, 6
  weeks on dev, 6 weeks on beta, and then to stable.
  Once diagrammed, using blocks to represent the weeks,
  realized that with 3 channels you could have two
  overlapping releases running at once, which could get us a
  stable release ~ every 6 weeks.
Block diagrams
It was a start, but it wasn't the full answer


   Sure, turning the wheel faster would make some things, like
   merges, easier.
   But without addressing the scope problems, the flow
   wouldn't be any better and would continue
   to interrupt releases.
   Things wouldn't be any more predictable and the whole
   cycle would just fall apart (more puddles, less of a stream).
Goal to Plan - Controlling Scope




 The pace of the schedule sets the boundaries for the amount
 of work that can be completed.
 It's important to have specific points in the schedule to
 review features and cut scope.
 Establish clear expectations (and engineering practice) to
 developers that any features not ready to ship at branch will
 be disabled (i.e. we only cut post branch, never add).
Plan to Pitch
  Reached out to the various cross functional groups on the
  Chrome team, who would all be impacted, before
  approaching our executives.
       Engineering, QA, Product, Marketing, Support,
       Localization, Security, etc...
  There were a lot of concerns to address, but the exercise
  forced a lot of thought on the implications of the schedule
  change.
  It took time, but it made the pitch easier to present to the
  leadership and the rest of the team.
Concerns
 Would large feature development still be possible?

 "Yes, engineers would have to work behind flags, however
 they can work for as many releases as they need to and can
 remove the flag when they are done."
 Can the engineers keep up?

 "Their pace won't need to change, since features can be
 disabled there should be no milestone pressure, things ship
 when they are ready."
 What would a world look like where we didn't base our
 marketing on releases?

 "We market features, not releases."
And so we implemented it

       11 week overlapping schedules.
Our General Rules

 The branch point is the end of our development cycle
 Features only get disabled post branch point
 Features should be engineered so that they can be disabled
 easily (1 patch)
 Only stability, security, and critical regressions can ever
 block a release
Things we struggled with
  Overcoming team inertia (particularly when we cut scope)
  Saying no consistently to the team
  Fighting the urge to return to date driven management
  decisions
  We initially didn't solve the trunk->dev->branch time to
  patch problem (daily channel helped).
  Setting the right burn down point (i.e. branch point).
  Disabling features.
Key Lessons


 Having clear and concise rules is important.
 Predictable schedules cut down on communication costs
 and team confusion.
 It takes work and fore-thought to disable features.
 Diligent feature tracking is important.
Conclusion
 Speed alone isn't always the right answer, it's about keeping
 things running smoothly.
 For us, scope was getting in the way of that goal.




We basically wanted to operate more like trains leaving Grand Central Station
(regularly scheduled and always on time), and less like taxis leaving the Bronx
                         (ad hoc and unpredictable).

More Related Content

Similar to Chrome发布周期

Chrome release cycle
Chrome release cycleChrome release cycle
Chrome release cycleJolicloud
 
Doing agile with an ISO-20000 Telco (AgilePT 2015)
Doing agile with an ISO-20000 Telco (AgilePT 2015)Doing agile with an ISO-20000 Telco (AgilePT 2015)
Doing agile with an ISO-20000 Telco (AgilePT 2015)Manuel Padilha
 
White Paper: Release This! - Tools for a Smooth Release Cycle
White Paper: Release This! - Tools for a Smooth Release CycleWhite Paper: Release This! - Tools for a Smooth Release Cycle
White Paper: Release This! - Tools for a Smooth Release CyclePerforce
 
The Architect's Two Hats
The Architect's Two HatsThe Architect's Two Hats
The Architect's Two HatsBen Stopford
 
How do we drive tech changes
How do we drive tech changesHow do we drive tech changes
How do we drive tech changesJaewoo Ahn
 
2012 - A Release Odyssey
2012 - A Release Odyssey2012 - A Release Odyssey
2012 - A Release OdysseyErnest Mueller
 
From Monolith to Microservices - What Could Go Wrong?
From Monolith to Microservices - What Could Go Wrong?From Monolith to Microservices - What Could Go Wrong?
From Monolith to Microservices - What Could Go Wrong?Phuong Mai Nguyen
 
DevOPs Transformation Workshop
DevOPs Transformation WorkshopDevOPs Transformation Workshop
DevOPs Transformation WorkshopJules Pierre-Louis
 
Questions Log: Dynamic Cubes – Set to Retire Transformer?
Questions Log: Dynamic Cubes – Set to Retire Transformer?Questions Log: Dynamic Cubes – Set to Retire Transformer?
Questions Log: Dynamic Cubes – Set to Retire Transformer?Senturus
 
A Tale of Two Workflows - ChefConf 2014
A Tale of Two Workflows - ChefConf 2014A Tale of Two Workflows - ChefConf 2014
A Tale of Two Workflows - ChefConf 2014Pete Cheslock
 
Scaling Up Lookout
Scaling Up LookoutScaling Up Lookout
Scaling Up LookoutLookout
 
Working with Agile technologies and SCRUM
Working with Agile technologies and SCRUMWorking with Agile technologies and SCRUM
Working with Agile technologies and SCRUMAndrea Tino
 
Agile Development Brown Bag Lunches Slides
Agile Development Brown Bag Lunches SlidesAgile Development Brown Bag Lunches Slides
Agile Development Brown Bag Lunches Slidesguesta1c5d7
 
Agile Delivery Methods And Leadership
Agile Delivery Methods And LeadershipAgile Delivery Methods And Leadership
Agile Delivery Methods And LeadershipRanjith Varghese
 

Similar to Chrome发布周期 (20)

Chrome release cycle
Chrome release cycleChrome release cycle
Chrome release cycle
 
Doing agile with an ISO-20000 Telco (AgilePT 2015)
Doing agile with an ISO-20000 Telco (AgilePT 2015)Doing agile with an ISO-20000 Telco (AgilePT 2015)
Doing agile with an ISO-20000 Telco (AgilePT 2015)
 
White Paper: Release This! - Tools for a Smooth Release Cycle
White Paper: Release This! - Tools for a Smooth Release CycleWhite Paper: Release This! - Tools for a Smooth Release Cycle
White Paper: Release This! - Tools for a Smooth Release Cycle
 
The Architect's Two Hats
The Architect's Two HatsThe Architect's Two Hats
The Architect's Two Hats
 
How do we drive tech changes
How do we drive tech changesHow do we drive tech changes
How do we drive tech changes
 
Kanban Primer
Kanban PrimerKanban Primer
Kanban Primer
 
2012 - A Release Odyssey
2012 - A Release Odyssey2012 - A Release Odyssey
2012 - A Release Odyssey
 
Egov Projects For Fun Profit
Egov Projects For Fun Profit Egov Projects For Fun Profit
Egov Projects For Fun Profit
 
ETCA_8
ETCA_8ETCA_8
ETCA_8
 
From Monolith to Microservices - What Could Go Wrong?
From Monolith to Microservices - What Could Go Wrong?From Monolith to Microservices - What Could Go Wrong?
From Monolith to Microservices - What Could Go Wrong?
 
DevOPs Transformation Workshop
DevOPs Transformation WorkshopDevOPs Transformation Workshop
DevOPs Transformation Workshop
 
Scrum overview
Scrum overviewScrum overview
Scrum overview
 
Questions Log: Dynamic Cubes – Set to Retire Transformer?
Questions Log: Dynamic Cubes – Set to Retire Transformer?Questions Log: Dynamic Cubes – Set to Retire Transformer?
Questions Log: Dynamic Cubes – Set to Retire Transformer?
 
A Tale of Two Workflows - ChefConf 2014
A Tale of Two Workflows - ChefConf 2014A Tale of Two Workflows - ChefConf 2014
A Tale of Two Workflows - ChefConf 2014
 
Intro to Gitflow
Intro to GitflowIntro to Gitflow
Intro to Gitflow
 
Scaling Up Lookout
Scaling Up LookoutScaling Up Lookout
Scaling Up Lookout
 
Working with Agile technologies and SCRUM
Working with Agile technologies and SCRUMWorking with Agile technologies and SCRUM
Working with Agile technologies and SCRUM
 
Agile Development Brown Bag Lunches Slides
Agile Development Brown Bag Lunches SlidesAgile Development Brown Bag Lunches Slides
Agile Development Brown Bag Lunches Slides
 
Agile Delivery Methods And Leadership
Agile Delivery Methods And LeadershipAgile Delivery Methods And Leadership
Agile Delivery Methods And Leadership
 
Kanban Methodology
Kanban MethodologyKanban Methodology
Kanban Methodology
 

More from 36Kr.com

《氪周刊》(第85期)
《氪周刊》(第85期)《氪周刊》(第85期)
《氪周刊》(第85期)36Kr.com
 
《氪周刊》(第85期)
《氪周刊》(第85期)《氪周刊》(第85期)
《氪周刊》(第85期)36Kr.com
 
《氪周刊》(第85期)
《氪周刊》(第85期)《氪周刊》(第85期)
《氪周刊》(第85期)36Kr.com
 
《氪周刊:互联网创业必读》(第84期)
《氪周刊:互联网创业必读》(第84期)《氪周刊:互联网创业必读》(第84期)
《氪周刊:互联网创业必读》(第84期)36Kr.com
 
《氪周刊:互联网创业必读》(第84期)
《氪周刊:互联网创业必读》(第84期)《氪周刊:互联网创业必读》(第84期)
《氪周刊:互联网创业必读》(第84期)36Kr.com
 
《氪月报》六月刊
《氪月报》六月刊《氪月报》六月刊
《氪月报》六月刊36Kr.com
 
微网介绍36
微网介绍36微网介绍36
微网介绍3636Kr.com
 
图吧新产品介绍 随行 20120613
图吧新产品介绍 随行 20120613图吧新产品介绍 随行 20120613
图吧新产品介绍 随行 2012061336Kr.com
 
益趣课程 Study.33iq.com demo final
益趣课程 Study.33iq.com demo final益趣课程 Study.33iq.com demo final
益趣课程 Study.33iq.com demo final36Kr.com
 
优伴户外旅行(0706)
优伴户外旅行(0706)优伴户外旅行(0706)
优伴户外旅行(0706)36Kr.com
 
1折店简介 --to36氪7.7
1折店简介 --to36氪7.71折店简介 --to36氪7.7
1折店简介 --to36氪7.736Kr.com
 
1折店简介 --to36氪7.7
1折店简介 --to36氪7.71折店简介 --to36氪7.7
1折店简介 --to36氪7.736Kr.com
 
36kr jing fm-0701
36kr jing fm-070136kr jing fm-0701
36kr jing fm-070136Kr.com
 
《氪周刊:互联网创业必读》(第83期)
《氪周刊:互联网创业必读》(第83期)《氪周刊:互联网创业必读》(第83期)
《氪周刊:互联网创业必读》(第83期)36Kr.com
 
4 七维网-chengdu
4 七维网-chengdu4 七维网-chengdu
4 七维网-chengdu36Kr.com
 
09心愿fm-chengdu
09心愿fm-chengdu09心愿fm-chengdu
09心愿fm-chengdu36Kr.com
 
01 扑扑网
01 扑扑网01 扑扑网
01 扑扑网36Kr.com
 
3 咕咚运动--chengdu
3 咕咚运动--chengdu3 咕咚运动--chengdu
3 咕咚运动--chengdu36Kr.com
 

More from 36Kr.com (20)

《氪周刊》(第85期)
《氪周刊》(第85期)《氪周刊》(第85期)
《氪周刊》(第85期)
 
《氪周刊》(第85期)
《氪周刊》(第85期)《氪周刊》(第85期)
《氪周刊》(第85期)
 
《氪周刊》(第85期)
《氪周刊》(第85期)《氪周刊》(第85期)
《氪周刊》(第85期)
 
《氪周刊:互联网创业必读》(第84期)
《氪周刊:互联网创业必读》(第84期)《氪周刊:互联网创业必读》(第84期)
《氪周刊:互联网创业必读》(第84期)
 
《氪周刊:互联网创业必读》(第84期)
《氪周刊:互联网创业必读》(第84期)《氪周刊:互联网创业必读》(第84期)
《氪周刊:互联网创业必读》(第84期)
 
《氪月报》六月刊
《氪月报》六月刊《氪月报》六月刊
《氪月报》六月刊
 
微网介绍36
微网介绍36微网介绍36
微网介绍36
 
图吧新产品介绍 随行 20120613
图吧新产品介绍 随行 20120613图吧新产品介绍 随行 20120613
图吧新产品介绍 随行 20120613
 
益趣课程 Study.33iq.com demo final
益趣课程 Study.33iq.com demo final益趣课程 Study.33iq.com demo final
益趣课程 Study.33iq.com demo final
 
魔豆
魔豆魔豆
魔豆
 
优伴户外旅行(0706)
优伴户外旅行(0706)优伴户外旅行(0706)
优伴户外旅行(0706)
 
1折店简介 --to36氪7.7
1折店简介 --to36氪7.71折店简介 --to36氪7.7
1折店简介 --to36氪7.7
 
1折店简介 --to36氪7.7
1折店简介 --to36氪7.71折店简介 --to36氪7.7
1折店简介 --to36氪7.7
 
36kr jing fm-0701
36kr jing fm-070136kr jing fm-0701
36kr jing fm-0701
 
豆果
豆果豆果
豆果
 
《氪周刊:互联网创业必读》(第83期)
《氪周刊:互联网创业必读》(第83期)《氪周刊:互联网创业必读》(第83期)
《氪周刊:互联网创业必读》(第83期)
 
4 七维网-chengdu
4 七维网-chengdu4 七维网-chengdu
4 七维网-chengdu
 
09心愿fm-chengdu
09心愿fm-chengdu09心愿fm-chengdu
09心愿fm-chengdu
 
01 扑扑网
01 扑扑网01 扑扑网
01 扑扑网
 
3 咕咚运动--chengdu
3 咕咚运动--chengdu3 咕咚运动--chengdu
3 咕咚运动--chengdu
 

Recently uploaded

FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024The Digital Insurer
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024The Digital Insurer
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdfSandro Moreira
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native ApplicationsWSO2
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Angeliki Cooney
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Victor Rentea
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Orbitshub
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...apidays
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 

Recently uploaded (20)

FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 

Chrome发布周期

  • 1. Chrome Release Cycle Author: Anthony Laforge (laforge@chromium.org)
  • 2. Our Philosophy Think of any major website, even the fancy Web 2.HTML5 ones... do they have version numbers? We took the same approach to our client software as an online web service. That is... we treat releases as a means of getting features out to users and not goals in and of themselves. It's about flow.
  • 3. How Our Users Work Users sign up for Chrome release channels based on the level of stability they want. Updates happen automatically in the background. New features and functionality just flow in without any effort on the user's part.
  • 4. How the Chrome Team Works Developers work almost exclusively on a single central trunk (possible due to our try bots and continuous build infrastructure) Our branches stem from points on the trunk We stabilize branches by pulling in changes from trunk (i.e. everything lands on trunk first)
  • 5. Where we started (v1-4) - On Paper
  • 6. In practice, for that final beta...we'd Cut a branch (when we thought we had everything) Merge about ~500 patches (since we didn't have everything) Spend weeks stabilizing and re-stabilizing issues. Ended up working 1-3 months to get a release out the door, always certainly missing our 13 week plan. And at the end finally shipping something we were happy with... but that left us pretty drained (i.e. the bad flow)
  • 7. The chain of events We'd hold releases, sometimes our branch points, until features were done, which meant... Lots of changes got queued up on the trunk, which caused... Engineers merging lots of changes to the branch, which led to... Instability on the branch that we had to deal with and that meant...
  • 8. That chain of events... Long lived branches, which... Made supporting the branch (i.e. merging changes) more difficult as we drifted further from the trunk... All of which led to... Unpredictable release cycles, with date targets we could never hit, and.... Engineers who were always in a rush to get their features into "this" release since we couldn't make any promises about the schedule
  • 9. Which led me to think... I need a long vacation. There has to be a better way. What's causing things not to flow smoothly? ...Yes...In that order.
  • 10. Primary Goals Shorten the release cycle and reduce the life span of a branch (make merges easier) Make releases more predictable and easier to scope Reduce the strain on the entire team Good Bad
  • 11. Goal to Plan - Getting down the cycle Originally set out a plan to simplify the 12 week cycle, 6 weeks on dev, 6 weeks on beta, and then to stable. Once diagrammed, using blocks to represent the weeks, realized that with 3 channels you could have two overlapping releases running at once, which could get us a stable release ~ every 6 weeks.
  • 13. It was a start, but it wasn't the full answer Sure, turning the wheel faster would make some things, like merges, easier. But without addressing the scope problems, the flow wouldn't be any better and would continue to interrupt releases. Things wouldn't be any more predictable and the whole cycle would just fall apart (more puddles, less of a stream).
  • 14. Goal to Plan - Controlling Scope The pace of the schedule sets the boundaries for the amount of work that can be completed. It's important to have specific points in the schedule to review features and cut scope. Establish clear expectations (and engineering practice) to developers that any features not ready to ship at branch will be disabled (i.e. we only cut post branch, never add).
  • 15. Plan to Pitch Reached out to the various cross functional groups on the Chrome team, who would all be impacted, before approaching our executives. Engineering, QA, Product, Marketing, Support, Localization, Security, etc... There were a lot of concerns to address, but the exercise forced a lot of thought on the implications of the schedule change. It took time, but it made the pitch easier to present to the leadership and the rest of the team.
  • 16. Concerns Would large feature development still be possible? "Yes, engineers would have to work behind flags, however they can work for as many releases as they need to and can remove the flag when they are done." Can the engineers keep up? "Their pace won't need to change, since features can be disabled there should be no milestone pressure, things ship when they are ready." What would a world look like where we didn't base our marketing on releases? "We market features, not releases."
  • 17. And so we implemented it 11 week overlapping schedules.
  • 18. Our General Rules The branch point is the end of our development cycle Features only get disabled post branch point Features should be engineered so that they can be disabled easily (1 patch) Only stability, security, and critical regressions can ever block a release
  • 19. Things we struggled with Overcoming team inertia (particularly when we cut scope) Saying no consistently to the team Fighting the urge to return to date driven management decisions We initially didn't solve the trunk->dev->branch time to patch problem (daily channel helped). Setting the right burn down point (i.e. branch point). Disabling features.
  • 20. Key Lessons Having clear and concise rules is important. Predictable schedules cut down on communication costs and team confusion. It takes work and fore-thought to disable features. Diligent feature tracking is important.
  • 21. Conclusion Speed alone isn't always the right answer, it's about keeping things running smoothly. For us, scope was getting in the way of that goal. We basically wanted to operate more like trains leaving Grand Central Station (regularly scheduled and always on time), and less like taxis leaving the Bronx (ad hoc and unpredictable).