Meet the women and technology running container orchestration at platform.sh
•Descargar como PPTX, PDF•
0 recomendaciones•70 vistas
Solutions Architect Sawssen Bardaoui introduces the technology and women behind Platform.sh, the Idea-to-Cloud Application Platform that helps companies like Magento, Symfony, The Economist and Ironman build, run and scale.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Meet the women and technology running container orchestration at platform.sh
Similar a Meet the women and technology running container orchestration at platform.sh (20)
Último
Último (20)
Meet the women and technology running container orchestration at platform.sh
- 1. Meet the Women and Technology Running Container Orchestration at Platform.sh European Women in Tech 2018, Amsterdam
- 2. Sawssen Bardaoui Customer Solutions Architect sawssen@platform.sh
- 3. For every advancement, there’s a new set of challenges: CI tools; virtualization; multiple clouds; each with its own idiosyncrasies.
- 5. Applications are more than just code.
- 6. Containerization Packaged to run quickly, reliably, and easily portable from one environment to another
- 9. FAST.
- 10. Modular.
- 11. PORTABLE.
- 12. LXC Choose a container management system: ● Lightweight ● Gives you control ● Greater flexibility
- 13. Criteria for choosing a storage management system: ● Distributed ● Fault-tolerant ● Scalable to exabyte level ● Can use commodity hardware
- 14. Logical Volume Manager The secret sauce for creating copies of prod to troubleshoot bugs
- 15. And now to orchestrate it all.
- 16. Apache Zookeeper Designed to store coordination data: status information, configuration, location information, etc.
- 17. > git checkout -b “Big Sale Promo” > git checkout -b “NYC Launch” > git checkout -b “Bugfix SUV-333” > git checkout -b “Fixing dependency” > git checkout -b “Added mongodb” > git merge master Use Git for everything
- 19. Galera Cluster ● Multi-master, Active-Active Cluster: Read and write to any node at any time. ● Synchronous Replication: No slave lag, no data is lost at node crash. ● Same state in all nodes. ● Multi-threaded Slave: Better performance. ● Hot Standby: no downtime during failover (since there is no failover). Running 3 or more MySQL servers together with replicated data
- 20. What do Zookeeper and Galera have in common? ← Galera Consensus Zookeeper Consensus →
- 21. Container orchestration is not easy
- 22. We’re hiring!
- 23. "One of the most intriguing aspects of Platform.sh is our team. We come from all over the world, different career and tech backgrounds, wide variety of individual strengths and talents, yet we form a cohesive group where everyone is welcome and valued." Maria Antinkaapo. VP of Customer Success
- 24. Join us! Talk to Robert Douglass and myself to join us! https://platform.sh/company/careers
Notas del editor
- Hi I’m Sawssen Bardaoui, I’m customer solutions architect at platform.sh. We are an idea-to-cloud-application platform, which means we provide cloud hosting and continuous deployment platform. At Platform.sh, I help onboard and support companies and organisations such as Arsenal Football Club, and Mustela migrate their web applications to Platform.sh. Today I’ll be sharing the advantages of doing container orchestration and how to do it.
- Anyone doing web application coding? For years, we’ve been struggling with web application deployment best practices such as the Four Tier deployment model: Development, Testing, Staging and Production. If you wanted a new environment to test code, you had to buy it, build it, maintain it, and find a way to deploy to it. As a result, most development teams maintained the absolute minimum number of environments or servers necessary to meet their own workflow requirements. In a lot of cases, this was actually less than four, and sometimes as little as two (or one, if you did your development directly onto your production server which is very bad). Obviously, cutting back on environments makes it very hard to know if you are testing and deploying code safely and reliably. To fix that, we started building CI (Continuous Integration) tools. This lets you test your change in isolation, which is very important, but it also gives you the opportunity to test it against an identical operating environment. But There is no CI pipeline in the world which does not require some knowledge of how it operates, yet most CI pipelines push the complexities of configuration back onto the developer. We built an entire new disciple (“DevOps”) to manage the complexity of solutions required to deal with these problems. While some CI providers have managed to partly simplify this process, it is still a non-trivial challenge in most cases. Developers need to be up-skilled and invested in, and since there are so many different ways of doing it, only a portion of those skills are transferable. Many businesses employ people full-time just to build and manage their build pipelines.
- The world has become so complicated. Wasn’t automation supposed to save us from all this? So How can I just push code and not care about all the rest. There definitely should be another way. By container orchestration We’re now going to look at the different technologies to do it: Containers Storage (Orchestration) tools
- Applications are more than just code. More than the executable that we run. An application is one with its underlying infrastructure. It is nothing without its configuration, its data, and its place in a topology. And here is where containers come in.
- Containers allow an application to be packaged with a skeleton of its dependencies, giving it lightweight attributes, This makes the application run quickly and reliably, as well as make it easily portable from one computing environment to another. The benefits from all this, is that deployments gets safer (downtime is minimized, data is protected from loss), development workflows are improved, since you can easily and rapidly copy your production environment to a new development one in few minutes.
- Put simply, a container consists of an entire runtime environment: an application, plus all its dependencies, libraries and other binaries, and configuration files needed to run it, bundled into one package. By containerizing the application platform and its dependencies, differences in OS distributions and underlying infrastructure are abstracted away. It is actually an alternative to virtualization, but with much more benefits
- First, Containers are lightweight. A container may be only tens of megabytes in size, whereas a virtual machine with its own entire operating system may be several gigabytes in size. Because of this, a single server can host far more containers than virtual machines.
- Second major benefit is that containers can be created instantly. While virtual machines may take several minutes to boot up their operating systems and begin running the applications they host, containerized applications can be started almost instantly. That means containers can be created in a "just in time" fashion when they are needed and disappear when they are no longer required, which frees up resources on their hosts.
- A third benefit is that containerization allows for greater modularity. Rather than run an entire complex application inside a single container, the application can be split into modules (such as the database, the application front end, and so on). This is the so-called microservices approach. And this is what we do at Platform.sh At Platform.sh, we use micro-containers to host web applications: from e-commerce websites based on Magento or Drupal to a Wordpress blog. To be Microcontainers oriented, means that every service gets its own container (Mysql, Elasticsearch, Solr, RabbitMQ, or whatever service your application will need).
- Finally, Containers make applications easily portable, since it abstracts the application’s components and makes it run independently from any OS or Infrastructure.
- So we’ve seen the advantages of containers. today, the most popular or well known technologies for running containers are Docker and Kubernetes. How many of you use Docker, raise your hands (there were a nice master class about Docker this morning) ? What about on Kubernetes? At Platform.sh, we use LXC because we find it to be more lightweight and gives us more control to manage our own microcontainers. For example, storage cloning, snapshotting the storage of the container. This requires signalling the application within the container to flush data before we snapshot. Docker doesn’t provide this.
- Now that we have the solution to deploy applications services quickly and efficiently, we need to find a way to manage the storage on this distributed system built on top of containers. Why distributed storage: “Storing data on multitude of standard servers, which behave as one storage system although data is distributed between these servers.” When choosing a Storage management system, consider these 4 factors: Is it ...? Distributed: whether storage can lie on multiple machines Fault tolerant: data can still be available in event of failures, since it is replicated Scalable to exabyte level: have the ability to add storage up to 1 billion gigabytes. Can use commodity hardware: allow the usage of large numbers of already-available computing components for parallel computing, to get the greatest amount of useful computation at low cost We like using CEPH for because of its CRUSH algorithm (Controlled Replication Under Scalable Hashing). CRUSH is a pseudo-random data distribution algorithm that efficiently and robustly distributes object replicas across a heterogeneous, structured storage cluster and because it provides an infinite storage scaling Ceph Storage Cluster It is basically a reliable, easy to manage, distributed object storage cluster, that stores/handles data as objects. Ceph Object Gateway This is a powerful Amazon S3 and Swift compatible gateway that brings the power of Ceph Object Store to modern applications. If you want an application to communicate with the Ceph Object Store using objects, you do it via the Ceph Object Gateway. Ceph Block Device This is a distributed virtual block device that delivers high performance, cost effective storage for virtual machines and related applications. If you want a Virtual Machine/Disk to communicate with the Ceph Object Store, you do it using the Ceph Block Device gateway. Ceph File System This is a distributed, scale out file system with POSIX semantics that provides storage for legacy and modern applications. If you want files and directories to be able to communicate with the Ceph Object Store, you do it via the Ceph File System Gateway.
- One of the bugbears of troubleshooting bugs is not being able to reproduce it without affecting the live environment. If any of you are doing application support in here like me, then I guess you would die to have this kind of feature when investigating production bugs, don’t you? :D, because I did when I was doing support. I really did! Here is something cool you could do. When you combine CEPH and Logical Volume Manager or LVM, you can actually copy running websites and create development or testing environments in only a few minutes without any downtime! LVM or Logical Volume Manager replaces the disk partitioning, it can be considered as a thin software layer on top of the hard disks and partitions. But it offers much more than just that! It gives you dynamic disk resizes without any service disruption. You can manage large disk farms because it allows disks to be added and replaced without downtime or service disruption. It also allows hot swapping and consistent backups And the ability to encrypt multiple physical partitions, with one passwords. Thanks to containerisation, storage management with CEPH and LVM, you can quickly copy your running production application in a test environment without any downtime on your production environment. At Platform.sh, we do rely on the LVM to make snapshots and manage copy-on-write.
- Managing all this complex architecture and distributed systems will soon become a nightmare if we don’t orchestrate it all efficiently. Here are some tools we use.
- Apache developed a cool tool, called Zookeeper, to coordinate distributed systems. It is a centralized configuration store, letting you store all of your metadata on your cluster: the roles, the data models it’s running, things to that effect. This means it guarantees a consistent state of all your containers. ZooKeeper runs on a cluster of servers called an ensemble that share the state of your data. Whenever a change is made, it is not considered successful until it has been written to a quorum (at least half) of the servers in the ensemble. A leader is elected within that ensemble, and if two conflicting changes are made at the same time, the one that is processed by the leader first will succeed and the other will fail. ZooKeeper guarantees that writes from the same client will be processed in the order they were sent by that client. This guarantee, along with its replication feature, allow the system to be used to implement locks, queues, and other important primitives for distributed queueing. At Platform.sh, we like using Zookeeper to store the metadata for our multitenant clusters running our customers’ websites, because we find it: Reliable: it keeps working even if a node fails, When a failed server is brought back online it is synchronized with the rest of the ensemble and can resume service. Has Simple Architecture: The architecture of ZooKeeper is quite simple as there is a shared hierarchical namespace which helps coordinate the processes. Allows Fast Processing: ZooKeeper is especially fast in "read-dominant" workloads (i.e. workloads in which reads are much more common than writes). Is Scalable: The performance of ZooKeeper can be improved by adding nodes.
- Because with containers, infrastructure is considered as code, we found a way to use git server to create the needed environments.
- Deploying an app means simply pushing the source code to the git server. The git server that runs on Platform.sh is at the same time “just a normal git repository” and a smart piece of software. When you have pushed to it, it will try to understand what it needs to do. But you can also explicitly tell it precisely what requirements your application has. There are a couple of “dot files” in YAML format you can put at the root of your application that will describe its dependencies. You can say for example “my app needs mysql as a database”, or “my app needs this or that other php extension”, or you can say “my front web app depends on these three API apps”. If for example you need to build your app a specific Node.js, Python or Ruby library (for example Less or Jekyll) you simply specify those in the YAML file. There are many other aspects of the application you can control through this simple declarative style. You don’t write scripts, you just tell git what you need.
- Now, to make the distributed system more reliable, consider using Galera Cluster to manage your databases. That is why at Platform.sh we use it on our Entreprise grade Clusters, which are for applications requiring dedicated hardware. Galera Cluster is a synchronous multi-master database cluster, based on synchronous replication and Oracle’s MySQL/InnoDB. When Galera Cluster is in use, you can direct reads and writes to any node at any time, and you can lose any individual node without interruption in operations and without the need to handle complex failover procedures. In Galera, a transaction is executed on one machine, the transaction controller, it collects a set of rows that are being changed by the transaction, the write set. On commit, it sends the write set to all other participating cluster nodes, which try to apply it. Cluster nodes report back to the transaction controller. If all nodes succeed, the transaction commits on the transaction controller as well, otherwise it is rolled back. That way all the nodes will have the same state.
- Interesting fact about Zookeeper and Galera, is that they have something in common, combined they make a great tool to be used on distributed systems. Both have consensus algorithms, which improves the reliability and the consistency of the distributed system. Processes in a distributed system need to agree on a master, on the members of a group, on configuration, on the owner of a lock, and on when to cross a barrier.
- As a solutions architect who has helped a lot customers on this, I can definitely say that container orchestration is not easy. Even though working with these different types of technologies and solving customer challenges are what gets me up in the morning. But, if you do like this sort of thing, then …
- We’re hiring! You get keep up with the fast changes of the tech world, to help us stay flexible to meet different customers’ needs. We have minor upgrades and product improvements released almost every week, and big ones almost on a monthly basis. To give you an idea of what technologies we’ve been working and rolling out: October 2018: Adding Go full support Sept 2018: Adding Python 3.7 support July 2018: Improving Security and Compliance documentation to better support GDPR June 2018: Offering NodeJS 10.x version and MongoDB 3.6
- I’m proud to work amongst a really diverse company and also be part of Maria’s team in Customer Success. Maria came to the company as a Technical Account manager before being promoted to Director of Customer Care. Her empathy and skills in problem-solving quickly led to her most recent promotion as VP of Customer Success this year. Maria is not the only female talent recognised at Platform.sh, Allison Simmons who has been the brains behind the UX and UI of our product since day one was also made VP of Brand and Experience this year. Andrea Soper has been playing a vital role in improving our product as part of the Engineering Team. Ivana Kotur is part of the Security and Compliance team and works on privacy topics such as GDPR and PIPEDA. Maria Melanathy and Stephanie Dogherty help our customers on a daily basis, answering their questions and troubleshoot issues they may have when using our platform. Olga Linek will be joining us very shortly in December as our newest Customer Success Engineer.
- We are a distributed-first company so you can work wherever you like at Platform.sh. To find out the different roles that we’re looking for or if you want to learn more about our company culture, come speak to Robert Douglass and I afterwards.