Behind the Scenes - How We Produce OpenStack, Thierry Carrez

•

0 recomendaciones•590 vistas

Cloud Native Day Tel Aviv

Tecnología Empresariales

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
Behind the scenes
How we produce OpenStack
Thierry Carrez
Release Manager, OpenStack

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
OpenStack is large & growing
130+ git repositories
1.8+ MLOC

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
OpenStack is painful
Difficult to deploy
Difficult to test
Difficult to keep up with the changes

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
OpenStack is open innovation
Anyone can propose a change
Lots of different companies
Lots of different countries/cultures
No traditional management structure

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
?

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
5 Principles

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
1Not tested Not working→
Not automatically tested Not tested→

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
2Code reviews
Nobody gets a free pass

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
3Gate-centric development
I, for one, welcome our new robotic overlords

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
4Time-based releases
How I stopped worrying and embraced the coordination benefits

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
5Automate all the things
...especially the boring tasks

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
Processes

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
6months
Design Summit, 4 days to discuss next release goals
PTLs define release goals
Milestones every 6 weeks
Last milestone, no more features (Feature Freeze)
RC1, creation of the release branch
Final release

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
Design
Write
Code
Code
review
Milestone
Final
release
Tests Gate
Design
review
Tests

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
Your git
branch
master git
branch
github
git.o.o
tarballs.o.o
review.o.o
developer
reviewers
infrastructure
PEP8
Hacking
Unit tests
Integration tests

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
Challenges

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
Numbers
On a typical week day

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
1000
Patchsets proposed

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
7500
Gerrit comments and votes

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
16000
Test runs, most spinning complete clouds

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
250
Changes merged into OpenStack code

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
Thank you !

Coordination and
Leadership challenges
in producing OpenStack
Thierry Carrez (@tcarrez)
Release management PTL
thierry@openstack.org
http://fnords.wordpress.com
@tcarrez on
ttx on
picture credits
Chaos by Daniela Hartman (https://www.flickr.com/photos/29487767@N02/4255026872)
Coordination by Doug88888 (https://www.flickr.com/photos/doug88888/4687906267)
Boring tasks by Alex Eylar (https://www.flickr.com/photos/hoyvinmayvin/5166095952)
Congestion by Joiseyshowaa (https://www.flickr.com/photos/joiseyshowaa/7454479488)
Rare issues by University of Liverpool HLS (https://www.flickr.com/photos/liverpoolhls/10740309163)
Community focus by Thierry Carrez
Watercooler by Eric Domont (https://www.flickr.com/photos/edomond/5348007434)
Tension by J.D. Hancock (https://www.flickr.com/photos/jdhancock/7186117659)

Más contenido relacionado

Similar a Behind the Scenes - How We Produce OpenStack, Thierry Carrez

Skutil brings the best of both worlds to H2O and sklearn, delivering an easy transition into the world of distributed computing that H2O offers, while providing the same, familiar interface that sklearn users have come to know and love. - Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai - To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata

Skutil - H2O meets Sklearn - Taylor Smith

Sri Ambati

State of the Stack v4 - OpenStack in All It's Glory

Randy Bias

OpenStack is an IaaS provider software written in Python. As such, it provides a massive scalable operating system and services like: Image, Storage, Object, Compute, etc. This talks aims to give the audience an overview about OpenStack, its capabilities, its modules, coding styles, workflow and organization. As a successful community driven development case, it’s definitely a good reference for anyone willing to take that road or maybe joining existing projects.

OpenStack: A python based IaaS provider

Flavio Percoco Premoli

Gnunify13 130219113507-phpapp01

openstackindia

OpenStack: An introduction

Atul Jha

On the heals of Luis Lavena's RailsConf talk "Infiltrating Ruby Onto The Enterprise Death Star Using Guerilla Tactics" comes a local and frank talk about the current state of Open Source Software (OSS) participation from Windows developers. Learn what OSS is, what motivates its contributors, and how OSS can make you a stronger developer. Be prepared to fall in love with writing software again! We will start off with a 101 introduction to both the Ruby programming language and the Ruby on Rails web application framework. You will learn about ActiveRecord, a powerful ORM that maps rich objects to your databases, and the latest components to use it with SQL Server. As a Rails core contributor and author of the SQL Server stack, I will give you a modern insight into both that will allow you to leverage your legacy data with Ruby. Lastly, I will review the bleeding edge tools being actively created for Windows developers to ease the transition to Ruby, Rails and OSS from a POSIX driven world. Many things have changed. It is time to learn and perform some occupational maintenance.

Free The Enterprise With Ruby & Master Your Own Domain

Ken Collins

YouTube Recording: https://www.youtube.com/watch?v=cCdqOxD5G0M Whether you are a newbie to OpenStack looking at building your first cloud or an experienced operator with years of OpenStack success behind you, you've probably spent some time wondering what to expect from the OpenStack project over the next several releases. Will it finally support that new capability you've been waiting for? Should you plan for an upgrade in the next 6 months? While the development community is always working and planning new features, its takes a lot of time on IRC to get a complete view across the different projects. The OpenStack Product WG spent time this cycle working with the project teams and PTLs to understand their priorities for the next several OpenStack releases. Where we have always had an understanding of what's to come in the next release, we're hoping to present a long-term view of the future landscape of OpenStack. In this session, we'll present our findings across the different projects in an effort to give users a glimpse into the OpenStack roadmap

What's Next in OpenStack? A Glimpse At The Roadmap

ShamailXD

Driving development in PHP

Eduardo Gulias Davis

This 2nd major State of the Stack address is a complete refresh of the spring 2013 edition, broadcast live on BrightTALK from the OpenStack Summit in Hong Kong. (Replay: https://www.brighttalk.com/webcast/10353/92159) Randy Bias, CEO and Co-founder of Cloudscaling examines the progress from Grizzly to Havana and delves into new areas like refstack, tripleO, bare metal server provisioning, the move from "projects" to "programs", and public/hybrid cloud compatibility. Check out the updated statistics on project momentum and look more closely at big upgrades in Havana, including OpenStack Orchestrate (Heat), which has the opportunity to change the game for OpenStack in the greater private and hybrid cloud game. We also discuss the "what is 'core'" debate and examine the idea that OpenStack is a kernel, not a complete cloud OS.

State of the Stack v2

Randy Bias

Chaos is a ladder !

Haggai Philip Zagury

Terraform Code Reviews: Supercharged with Conftest

Jay Wallace

Open stack Austin meetup January 15, 2015

Tesora

OpenStack is an open source stack that can be deployed on raw computing resources to privately or publicly present Infrastructure as a Service. It now consists of more than 4.5 million lines of code, 85% of which is Python. In this talk, Thierry Carrez and Doug Hellmann, both Python Software Foundation fellows and OpenStack Technical Committee members, look at the symbiotic relationship between OpenStack and Python. We go back in history and explain why OpenStack originally picked Python as its main language 6 years ago, and explore what does Python bring to OpenStack. We dive into examples of OpenStack pushing Python libraries to their limits and exposing new bugs. We look into the massive cloud-based continuous integration system that OpenStack uses and explain how it exposes bugs in Python libraries in the minutes after they are published to PyPI. We look into Python libraries that were created by the OpenStack community and libraries that the OpenStack community took over. Finally we'll expose a few best practices that Python developers can follow to get the most of this symbiotic relationship.

How OpenStack Makes Python Better (and vice-versa)

doughellmann

The goal of today’s talk will be about becoming an RDO / OpenStack / TripleO active technical contributor, but what does that mean? It means that you are a developer or designer or support engineer or tester or project manager or writer or whatnot and you’d like to become involved in an open source project. It means that you want contribute to TripleO, which is a project within the umbrella space called OpenStack, and that you can join the RDO community for guidance, feedback, and support.

Become An OpenStack TripleO ATC - Easy As ABC

K Rain Leander

Scripting with Python to interact with Capella model Have you ever wanted to easily extract engineering data from your Capella model? Have you ever wanted to easily modify your Capella model and import information into it to update it? This webinar presents a prototype Capella Add-on that will address several example use cases - Read information from a Capella model and export to Excel, with queries - Update information in a Capella model - Add elements in a Capella model This new Capella add-on uses a common scripting language, not dedicated to Capella: Python. - It offers the capacity to use sample scripts addressing basic need and to build its own scripts, with libraries for common add-ons (Requirement, PVMT) - It’s easy to share, to use, has high customization capabilities support of Capella and Team for Capella, wide compatibility with Capella versions It is presented by : - Sophie Plazanet (Thales Group) - MBSE Specialist Master of Engineering & Master of Research in Advanced Systems & Robotics – Arts & Métiers ParisTech - Arnaud Dieumegard (Obeo) - Eclipse Modeling Consultant Ph.D. in Reliability for Systems and Software - INP Toulouse To illustrate the examples, you'll find the videos on this playlist: https://bit.ly/capella_webinar_211216_playlist

Scripting with Python to interact with Capella model

Obeo

Join us for a webinar on how to secure your CI/CD pipeline for Kubernetes with GitOps best practices and continuous runtime protection. As modern developers and DevOps teams are embarking on a quest for speed and reliability through automated CI/CD pipelines for Kubernetes, enterprises still need to ensure security and regulatory compliance. Together with Deepfence, the Weaveworks team will explain and demonstrate how GitOps continuous delivery pipelines, combined with continuous security observability, improves the overall security of your development workflow - from Git to production. In this webinar we will demonstrate: Deepfence container scanning Git-to-Kubernetes using FluxCD Deepfence continuous runtime security

Hardening Your CI/CD Pipelines with GitOps and Continuous Security

Weaveworks

Giving Back to Upstream | DockerCon 2019

Phil Estes

This presentation provides an introduction to the OpenStack DefCore Committee, which is working to create interoperability standards for OpenStack Powered clouds. You'll gain insight into the interoperability challenges of OpenStack clouds, and learn how DefCore creates it's Guidelines. Learn why the Technical Committee, Board of Directors, end users, and vendors have a seat at the table. You'll laugh, you'll cry, you'll immediately want to stop talking about cloud computing and go watch science fiction all night. This talk was originally presented at the Triangle OpenStack Meetup Group's September 21, 2015 meeting in Durham, NC. A recording can be found here (this talk starts at the 46:10 mark): https://vmware.webex.com/vmware/lsr.php?RCID=a51f9e6882f54ccab8b715c8c0162484 A new revision with updates was given at a meeting of the China Open Source Cloud League on May 20, 2016 in Beijing. The slides here on Slideshare represent that presentation.

DefCore: The Interoperability Standard for OpenStack

Mark Voelker

Hashitalks 2021 Infrastructure Drift & Driftctl

Stephane Jourdan

StackEngine Demo - Docker Austin

Boyd Hemphill

Similar a Behind the Scenes - How We Produce OpenStack, Thierry Carrez (20)

Skutil - H2O meets Sklearn - Taylor Smith

State of the Stack v4 - OpenStack in All It's Glory

OpenStack: A python based IaaS provider

Gnunify13 130219113507-phpapp01

OpenStack: An introduction

Free The Enterprise With Ruby & Master Your Own Domain

What's Next in OpenStack? A Glimpse At The Roadmap

Driving development in PHP

State of the Stack v2

Chaos is a ladder !

Terraform Code Reviews: Supercharged with Conftest

Open stack Austin meetup January 15, 2015

How OpenStack Makes Python Better (and vice-versa)

Become An OpenStack TripleO ATC - Easy As ABC

Scripting with Python to interact with Capella model

Hardening Your CI/CD Pipelines with GitOps and Continuous Security

Giving Back to Upstream | DockerCon 2019

DefCore: The Interoperability Standard for OpenStack

Hashitalks 2021 Infrastructure Drift & Driftctl

StackEngine Demo - Docker Austin

Más de Cloud Native Day Tel Aviv

Cloud Native is a Cultural Decision. By Reshef Mann

Cloud Native Day Tel Aviv

In any Cloud Native architecture there’s a seemingly endless stream of events that happen at each layer. These events can be used to detect abnormal activity and possible security incidents, as well as providing an audit trail of activity. In this talk we’ll cover how we extended Falco to ingest events beyond just host system calls, such as Kubernetes audit events or even application level events. We will also show how to create Falco rules to detect behaviors in these new event streams. We show how we implemented Kubernetes audit events in Falco, and how to configure the event stream.

Container Runtime Security with Falco, by Néstor Salceda

Cloud Native Day Tel Aviv

Kafka Mirror Tester: Go and Kubernetes Powered Test Suite for Kafka Replicati...

Cloud Native Day Tel Aviv

Running I/O intensive workloads on Kubernetes, by Nati Shalom

Cloud Native Day Tel Aviv

Service meshes are all the buzz in cloud-native world. How come only yesterday we didn't know such a thing existed and now everybody seems to want one? If you're already running a microservice-based system or only starting out with one, you may be asking yourself : "Do I also need a mesh?" In this session we'll try to answer what the mesh is good for, what problem it solves, what new questions it poses.

WTF Do We Need a Service Mesh? By Anton Weiss.

Cloud Native Day Tel Aviv

Update Strategies for the Edge, by Kat Cosgrove

Cloud Native Day Tel Aviv

Building a Cloud-Native SaaS Product The Hard Way. By Arthur Berezin

Cloud Native Day Tel Aviv

If you are interested in monitoring, and successfully set up a system (whether home-grown or custom-off-the-shelf) for your own use, there comes a moment when you go from monitoring only the systems you care about, to monitoring systems that other people care about. Monitoring for yourself is all about having the best data for the least effort. Monitoring for others? That's when your job becomes a game of "what just happened" whack-a-mole.

The Four Questions (Every Monitoring Engineer gets asked), by Leon Adato

Cloud Native Day Tel Aviv

K8s Pod Scheduling - Deep Dive. By Tsahi Duek.

Cloud Native Day Tel Aviv

Cloud Native: The Cattle, the Pets, and the Germs, by Avishai Ish-Shalom

Cloud Native Day Tel Aviv

MySQL shell is the MySQL client of the future. It will help you in your daily operations, whatever they are. It doesn't matter if you are a developer or an administrator, if you want to work with relational or non relational data, if you want to setup or monitor your cluster, if you want to work with SQL language or javascript or python. Discover how MySQL shell will help you, no matter what you want to do with MySQL!

MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.

Cloud Native Day Tel Aviv

Cloud native transformation patterns, by Pini Reznik

Cloud Native Day Tel Aviv

Cloud and Edge: price, performance and privacy considerations in IOT, by Tsvi...

Cloud Native Day Tel Aviv

Running Highly Available Large Scale Systems is a lot of work. For the past 2.5 years, we've been running 100% serverless on a full production environment, serving customers worldwide. No VMs, no containers, no Kubernetes. Just code. In this session I will present why we decided to go fully serverless at Torii, how it helped us move faster than our competitors, where did serverless computing worked best and where there's more work to be done.

Two Years, Zero servers: Lessons learned from running a startup 100% on Serve...

Cloud Native Day Tel Aviv

The “Twelve-Factor” application model has come to represent twelve best practices for building modern, cloud-native applications. With guidance on things like configuration, deployment, runtime, and multiple service communication, the Twelve-Factor model prescribes best practices that apply to everything from web applications to APIs to data processing applications. Although Serverless computing and AWS Lambda have changed how application development is done, the “Twelve-Factor” best practices remain relevant and applicable in a Serverless world. In this talk, we’ll apply the “Twelve-Factor” model to Serverless application development with AWS Lambda and Amazon API Gateway and show you how these services enable you to build scalable, low cost, and low administration applications.

12 Factor Serverless Applications - Mike Morain, AWS - Cloud Native Day Tel A...

Cloud Native Day Tel Aviv

When creating a new Microservice you typically need to add a lot of boilerplate to the code, such as logging, metrics, authentication, SSL, secrets/credentials, etc... All this ends up overshadowing the actual logic of the service itself and results in a lot of dependencies and code. Because of this, we at Soluto created a template for the boilerplate, which worked great... Until we had to upgrade one of the dependencies which resulted in huge effort of code upgrades and deployment across each and every service that uses the template. And that’s besides the fact we needed to make a separate template for each language we used (and over the years, the number of programming languages we use at production increased dramatically). We needed a way to simplify all this. We wanted to get rid of the boilerplate while maintaining the functions listed above. So we decided to delegate the entire responsibility to the Kubernetes infrastructure. In this lecture, I will do a live coding session and show you how to remove the boilerplate from your code and move it to the infrastructure, and discuss the benefits and limitations of this approach.

Not my problem! Delegating responsibilities to the infrastructure - Yshay Yaa...

Cloud Native Day Tel Aviv

Machine Learning is no doubt the hottest trend in IT nowadays. Deep Neural Network (DNN), a subfield of Machine Learning with mode of operation loosely inspired by the brain, allows us to solve complex problems such as image recognition that has been very difficult to solve using standard programming paradigms. DNN concepts are not new. However, and until recently, applying them in practice could not be realized due to their high computational demands. With the recent development in parallel computing, especially around GPU acceleration and high speed and efficient networking, DNN has become a reality in modern data centers. In this talk we will describe the system requirements to effectively run a machine learning cluster with popular frameworks such as TensorFlow. We will discuss how such a system can be deployed in an OpenStack-based cloud without compromises, enjoying high-performance DNN programming paradigm as well as the benefits of cloud and software-defined data centers.

Brain in the Cloud: Machine Learning on OpenStack & Kubernetes Done Right - E...

Cloud Native Day Tel Aviv

Cloud native applications are commonly thought as stateless, horizontally scalable workloads that you can scale-up and down on-demand. Kubernetes, as the commodity cloud native orchestrator, was originally designed for such workloads. A lot has evolved since Kubernetes’ inception, and nowadays many of the stateful applications are migrating to Kubernetes. While not everything is perfect, more and more features are added to support complex stateful use-cases. In this session Arthur will cover the following topics: - Breakdown of a stateful application - Planning a stateful application on Kubernetes - The state of Kubernetes StatefulSets, Persistent Volumes, DNS, Networking, operators and High Availability - A practical use-case and DEMO of a stateful application with Kubernetes

A stateful application walks into a Kubernetes bar - Arthur Berezin, JovianX ...

Cloud Native Day Tel Aviv

The story of how KubeMQ was born - Oz Golan, KubeMQ - Cloud Native Day Tel Av...

Cloud Native Day Tel Aviv

I want it all: go hybrid - Orit Yaron, Outbrain - Cloud Native Day Tel Aviv 2018

Cloud Native Day Tel Aviv

Más de Cloud Native Day Tel Aviv (20)