Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)

1. Kubernetes One-Click Deployment: Hands-on Workshop @joerg_schad @dcos

2. Jörg Schad Technical Community Lead / Developer ● Core Mesos developer at Mesosphere ● Passions are deep learning, distributed data systems, and data analytics

3. Chris Gaun PMM at Mesosphere / CNCF Ambasador ● Previous to that Gartner analyst covering public IaaS ● @gaunetes

4. First, Some Questions (Show of Hands) • Are you a developer? • Are you in operations? • Are you using K8s (in production)? • Are you using DC/OS (in production)?

5. Goals Of Workshop

8. © 2018 Mesosphere, Inc. All Rights Reserved. noun | ˈmīkrō/ /ˈsərvəs/ : an approach to application development in which a large application is built as a suite of modular services. Each module supports a specific business goal and uses a simple, well-defined interface to communicate with other modules.* Microservices are designed to be flexible, resilient, efficient, robust, and individually scalable. *From whatis.com OVERVIEW

9. © 2018 Mesosphere, Inc. All Rights Reserved. Operating System Operating System Operating System ServiceApp ServiceServiceAppApp 9 MICROSERVICES - Polyglot - Single Responsibility - Smaller Teams - Utilization - Machine types/groups - Dependency hell Machine Infrastructure Machine Machine ServiceService ServiceServiceServiceService

10. © 2018 Mesosphere, Inc. All Rights Reserved. ServiceApp ServiceServiceAppApp OS 10 CONTAINERS - Rapid deployment - Dependency vendoring - Container image repositories - Spreadsheet scheduling OS OS Machine Infrastructure Machine Machine Container Runtime Container Runtime Container Runtime ServiceService ServiceServiceServiceService

12. © 2018 Mesosphere, Inc. All Rights Reserved. 12 CONTAINER SCHEDULING - Placement - Replication/Scaling - Resurrection - Rescheduling - Rolling Deployment - Upgrades - Downgrades - Collocation RESOURCE MANAGEMENT - Memory - CPU - GPU - Volumes - Ports - IPs - Images/Artifacts SERVICE MANAGEMENT - Labels - Groups/Namespaces - Dependencies - Load Balancing - Readiness Checking CONTAINER ORCHESTRATION

13. © 2018 Mesosphere, Inc. All Rights Reserved. Orchestration 13 Machine Infrastructure Web Apps & Services Scheduling Resource Management Container Runtime Machine & OS Service Management CONTAINER ORCHESTRATION Machine & OS Machine & OS Container Runtime Container Runtime

14. The premier platform to run data-driven, containerized applications on any infrastructure North American Banks Worldwide Telco Companies Highest-Valued Startups Automotive Technology Efforts Represents organizations using Mesosphere technologies 5 of Top 10 30% of 7 of Top 12 5 of Top 10 6 Leading

15. Kubernetes Concepts

16. Container Pods

17. Volume Data Loader Front End Pods

18. Volume Data Loader Front End Pods Set of Related Containers Tightly Coupled Ephemeral/Stateless Share Namespace Share Network Unit of Management

19. Pods Set of Related Containers Tightly Coupled Ephemeral/Stateless Share Namespace Share Network Unit of Management Pod

20. Pod Networking D B C A Routable (layer 3) No NAT (internode)10.0.0.1 10.0.0.2 10.0.0.3 10.0.0.4

21. Labels D B C A app = portal tier = frontend version = v1 app = portal tier = backend version = v1 app = portal tier = frontend version = v2 app = portal tier = backend version = v2

22. Label Selectors D B C A app = portal tier = frontend version = v1 app = portal tier = backend version = v1 app = portal tier = frontend version = v2 app = portal tier = backend version = v2 app = portal SELECTOR

23. Label Selectors D B C A app = portal tier = frontend version = v1 app = portal tier = backend version = v1 app = portal tier = frontend version = v2 app = portal tier = backend version = v2 app = portal version = v1 SELECTOR

24. Controllers Desired State Current State UPDATEANALYZE

25. Replica Set (Controller) NODE 1 A NODE 2 A NODE 3 A NODE 4 A Replicas = 4 DESIRED Replicas = 4 CURRENT

26. Replica Set (Controller) NODE 1 A NODE 2 A NODE 3 A NODE 4 A Replicas = 4 DESIRED Replicas = 3 CURRENT

27. Replica Set (Controller) NODE 1 A NODE 2 A NODE 3 A NODE 4 A Replicas = 4 DESIRED Replicas = 4 CURRENT A

28. (µ)Services Service A Stable Cluster-wide IP Cluster-wide DNS name Layer 3 Load Balancer Updates continuously app = portal tier = frontend version = v1 SELECTOR A A A

29. (µ)Services Service A New Pod added automatically based on its label selector app = portal tier = frontend version = v1 SELECTOR A A A A

30. Kubernetes concepts 30 kind: Deployment kind: ReplicaSet kind: Pod scales N instances of Pod Set of Containers Upgrades Deployees

31. Configuration File 31 apiVersion: apps/v1beta2 kind: Deployment metadata: name: nginx-deployment spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80

32. Configuration File 32 apiVersion: apps/v1beta2 kind: Deployment metadata: name: nginx-deployment spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80 ReplicaSet Pod

33. Service Spec and Labels kind: Service apiVersion: v1 metadata: name: my-service spec: selector: app: MyApp ports: - protocol: TCP port: 80 targetPort: 9376 Label

34. Kubernetes Architecture

35. ETCD 01 MASTER 01 etcd LoadBalancer WORKER 01 API Server Scheduler Controller Manager Kubectl Kubelet Kube Proxy Container Engine (Docker, CRI-O) Pod Pod Pod HTTPS HTTPS HTTPS HTTPS

36. ETCD 01 MASTER 01 LoadBalancer WORKER 01 API Server Scheduler Controller Manager Kubectl Kubelet Kube Proxy Container Engine (Docker, CRI-O) Pod Pod Pod HTTPS HTTPS HTTPS HTTPS • Etcd is a distributed, consistent key-value store • Uses the RAFT consensus algorithm for leader election • Supports revisions and event streams • Primary store for all K8s API objects (single source of truth) • The only storage backend currently supported by Kubernetes etcd

38. ETCD 01 MASTER 01 LoadBalancer WORKER 01 Scheduler Controller Manager Kubectl Kubelet Kube Proxy Container Engine (Docker, CRI-O) Pod Pod Pod HTTPS HTTPS HTTPS HTTPS etcd API Server • Entry point to the system • CRUD server for all the API objects in Kubernetes • Processes REST requests and updates the objects in etcd • Performs authentication / authorization MORE: https://goo.gl/KL8WfQ

39. ETCD 01 MASTER 01 LoadBalancer WORKER 01 Scheduler Kubectl Kubelet Kube Proxy Container Engine (Docker, CRI-O) Pod Pod Pod HTTPS HTTPS HTTPS HTTPS etcd API Server • Daemon process that implements the control loops built into Kubernetes • E.g. … see NEXT slide MORE: https://goo.gl/NJyRP3 Controller Manager

40. ETCD 01 MASTER 01 LoadBalancer WORKER 01 Scheduler Kubectl Kubelet Kube Proxy Container Engine (Docker, CRI-O) Pod Pod Pod HTTPS HTTPS HTTPS HTTPS etcd API Server Types of Controllers in Kubernetes: ● ReplicaSet - desired state of pods ● ReplicationController - deprecated ● Deployments - updates ● StatefulSets - data services ● DaemonSet ● GarbageCollection - kills orphans ● Jobs ● CronJobs Controller Manager

41. ETCD 01 MASTER 01 LoadBalancer WORKER 01 Kubectl Kubelet Kube Proxy Container Engine (Docker, CRI-O) Pod Pod Pod HTTPS HTTPS HTTPS HTTPS etcd API Server Controller Manager • Answers: Where should this pod run? • Based on policy, available resources, affinity, labels, Qos requirements, etc. • Binds unscheduled pods to nodes EXPERT MODE: ● Node Affinity/Anti-Affinity ● Taints and Tolerations ● Pod Affinity/Anti-Affinity https://goo.gl/nvLDE9Scheduler

43. ETCD 01 MASTER 01 LoadBalancer WORKER 01 Kubectl Kube Proxy Container Engine (Docker, CRI-O) Pod Pod Pod HTTPS HTTPS HTTPS HTTPS etcd API Server Controller Manager Scheduler • Agent that runs on each node • Recieves a set of PodSpecs and insures all pods are running and healthy • Interacts with container runtime (Docker, CRI-O) • Performs some node-level health checks MORE: https://goo.gl/FEKN43 Kubelet

44. ETCD 01 MASTER 01 LoadBalancer WORKER 01 Kubectl Container Engine (Docker, CRI-O) Pod Pod Pod HTTPS HTTPS HTTPS HTTPS etcd API Server Controller Manager Scheduler Kubelet • Network proxy and load balancer for Services • Used to be a user-space proxy • Currently uses iptables MORE: https://goo.gl/ph4sAs Kube Proxy

47. - RBAC + IAM - Network segmentation - OSS framework/ container control - Multiple isolated clusters - Vanilla / standard / OSS Kubernetes - Identical setup / components / no snowflakes - Multi/hybrid cloud - Multi data center - Easy updating / versioning - Scaling - Integrations - ELB, networking, Storage, Monitoring - Easy installation Security Interoperability Easy Management Delivering Kubernetes Challenges Where they run Kubernetes Multiple management hurdles 42% On Prem 57% DIY AWS ~40% 40+% Require more security for wider deployment CNCF poll

48. © 2018 Mesosphere, Inc. All Rights Reserved. 49 Kubernetes Machine Infrastructure Container Runtime Machine & OS Machine & OS Machine & OS Container Runtime Container RuntimeContainer Runtime Machine & OS Machine & OS Container Runtime Orchestration Scheduling Resource Management Service Management Web Apps & Services Machine Infrastructure OrchestrationScheduling Resource Management Service Management Web Apps & Services

49. © 2018 Mesosphere, Inc. All Rights Reserved. KUBERNETES ARCHITECTURE 50 ETCD 01 MASTER 01 etcd LoadBalancer WORKER 01 API Server Scheduler Controller Manager Kubectl Kubelet Kube Proxy Container Engine (Docker, CRI-O) Pod Pod Pod HTTP S HTTP S HTTP S HTTP S

50. © 2018 Mesosphere, Inc. All Rights Reserved. 51 Kubernetes HA Machine Infrastructure Container Runtime Machine & OS Machine & OS Machine & OS Container Runtime Container RuntimeContainer Runtime Machine & OS Machine & OS Container Runtime Machine Infrastructure Web Apps & Services etcd etcdetcd api-ser ver api-ser ver api-ser ver Kube- proxy Kube- proxy scheduler Kube- proxy scheduler schedulercontrollercontroller controller kubelet kubelet kubelet ...

51. © 2018 Mesosphere, Inc. All Rights Reserved. 52 Kubernetes HA Machine Infrastructure Container Runtime Machine & OS Machine & OS Machine & OS Container Runtime Container RuntimeContainer Runtime Machine & OS Machine & OS Container Runtime Machine Infrastructure Web Apps & Services etcd etcdetcd api-ser ver api-ser ver api-ser ver Kube- proxy Kube- proxy scheduler Kube- proxy scheduler schedulercontrollercontroller controller kubelet kubelet kubelet ...

55. © 2018 Mesosphere, Inc. All Rights Reserved. 56 Datacenter Typical Datacenter siloed, over-provisioned servers, low utilization Mesos/ DC/OS automated schedulers, workload multiplexing onto the same machines Tensorflow Jenkins Kafka Spark Tensorflow

56. PHYSICAL INFRASTRUCTURE MICROSERVICES, CONTAINERS, & DEV TOOLS VIRTUAL MACHINES PUBLIC CLOUDS DATA SERVICES, MACHINE LEARNING, & AI Security & Compliance Application-Aware Automation Multitenancy Hybrid Cloud Management 100+ MORE DatacenterEdge Datacenter and Cloud as a Single Computing Resource Powered by Apache Mesos 20+ MORE Unified hybrid cloud operations Securely manage cloud, datacenter, and edge infrastructures from a single control plane 4 Mesosphere DC/OS Intelligent resource pooling Optimize workload density for highest utilization with resource guarantees 3 Broad workload coverage Run today & tomorrow’s applications including traditional J2EE, containers, analytics & ML 1 Application-aware automation Automate workload-specific operating procedures to “as-a-Service” anything from Kubernetes to data services 2

57. Deploy Scale Configure Recover 3 AM ... Typical Datacenter siloed, over-provisioned servers, low utilization Kubernetes Jenkins Kafka Spark Cassandra

58. DC1 DC2 DC3 Deploy Scale Configure Recover 3 AM ...

59. © 2018 Mesosphere, Inc. All Rights Reserved. Two-level Scheduling 1. Agents advertise resources to Master 2. Master offers resources to Framework 3. Framework rejects / uses resources 4. Agent reports task status to Master 60 MESOS ARCHITECTURE Mesos Master Mesos Master Mesos Master Mesos AgentMesos Agent Service Cassandra Executor Cassandra Task Kubernetes Scheduler Spark Executor Spark Task Mesos AgentMesos Agent Service Docker Executor Docker Task K8s Executor Kubelet Task Marathon Scheduler Kafka Scheduler

60. © 2018 Mesosphere, Inc. All Rights Reserved. 61 DC/OS and Kubernetes Machine Infrastructure Container Runtime Machine & OS Machine & OS Machine & OS Container Runtime Container RuntimeContainer Runtime Machine & OS Machine & OS Container Runtime Orchestration Scheduling Resource Management Service Management Web Apps & Services Machine Infrastructure Mesosphere DC/OS OrchestrationScheduling Resource Management Service Management Web Apps & Services

61. © 2018 Mesosphere, Inc. All Rights Reserved. 62 DC/OS and Kubernetes Orchestration Machine Infrastructure Web Apps & Services Scheduling Resource Management Container Runtime Machine & OS Service Management Machine & OS Machine & OS Container Runtime Container RuntimeContainer Runtime Machine & OS Machine & OS Container Runtime Orchestration Scheduling Resource Management Service Management Web Apps & Services Machine Infrastructure Mesosphere DC/OS Big Data Services ● Spark ● Flink ● Kafka ● ….

62. © 2018 Mesosphere, Inc. All Rights Reserved. 63 Mesos and Kubernetes Machine Infrastructure Container Runtime Machine & OS Machine & OS Machine & OS Container Runtime Container RuntimeContainer Runtime Machine & OS Machine & OS Container Runtime Machine Infrastructure DC/OS / Apache Mesos Web Apps & Services etcd etcdetcd api-ser ver api-ser ver api-ser ver Kube- proxy Kube- proxy scheduler Kube- proxy scheduler schedulercontrollercontroller controller kubelet kubelet kubelet ...

63. © 2018 Mesosphere, Inc. All Rights Reserved. 64 Mesos and Kubernetes Machine Infrastructure Container Runtime Machine & OS Machine & OS Machine & OS Container Runtime Container RuntimeContainer Runtime Machine & OS Machine & OS Container Runtime Machine Infrastructure Mesosphere Apache Mesos Web Apps & Services etcd etcdetcd api-ser ver api-ser ver api-ser ver Kube- proxy Kube- proxy scheduler Kube- proxy scheduler schedulercontrollercontroller controller kubelet kubelet kubelet ...

64. © 2018 Mesosphere, Inc. All Rights Reserved. 65 Reality Machine Infrastructure Container Runtime Machine & OS Machine & OS Machine & OS Container Runtime Container RuntimeContainer Runtime Machine & OS Machine & OS Container Runtime Orchestration Scheduling Resource Management Service Management Web Apps & Services Machine Infrastructure DC/OS/Apache Mesos

73. Kubernetes with DC/OS 1.11 Features Simple HA Cluster Provisioning Robust API Server Auth Non-disruptive Upgrades Automated self-healing Scalable Kubernetes Transport Layer Security Cloud Native Services

74. Why Kubernetes on DC/OS: Automation Management Ability to easily create & scale pure play Kubernetes, data services, and OSS components anywhere Lifecycle Ability to install/upgrade/patch every component of the solution stack (data, K8s, CI/CDs and ML) as-a-Service Ability for each LOB to choose components and manage resources and lifecycle according to its will Security 30% Fortune 50 | CNI | TLS | Authentication

75. Simple HA Cluster Provisioning Kubernetes with DC/OS 1.11 Features Robust API Server Auth Non-disruptive Upgrades Automated self-healing Scalable Kubernetes Transport Layer Security Cloud Native Services

76. Dead Simple Secure & HA Provisioning 1. Prerequisites 2. Installing the Client Tools 3. Provisioning Compute Resources 4. Provisioning the CA and Generating TLS Certificates 5. Generating Kubernetes Configuration Files for Authentication 6. Generating the Data Encryption Config and Key 7. Bootstrapping the etcd Cluster… 3x for HA 10. Bootstrapping the Kubernetes Control Plane… 3x for HA 13. Bootstrapping the Kubernetes Worker Nodes 14. Configuring kubectl for Remote Access 15. Provisioning Pod Network Routes 16. Deploying the DNS Cluster Add-on… Deploying other Add-ons 20. Smoke Test 21. Cleaning Up We turn this (21 steps)*…. $ dcos package install kubernetes Into this…. * Kubernetes the Hard Way by Kelsey Hightower

77. What’s Kubernetes Done Right? ● Automated - Kubernetes operations should be automated ● Evergreen - You want to always run the latest version of Kubernetes ● Included - Kubernetes is a commoditized standard that is included in a solution ● Open source - Market standard is pure OSS Kubernetes ● Unified - Kubernetes is not an island, you’ll have other workloads

78. Enhanced Data Security (EE) ● Secure Authentication, Authorization and In-Transit Data Encryption ● Enable Authentication of Users, Apps (Client-Server) & Inter-Service Communication (Server to Server) ● One-Click Configuration of Transport Security (TLS) ● Significantly Reduce Operational Overhead Non-locked down DIY Kubernetes clusters target of hidden malware

79. Enabling Developer Agility: Kubernetes is one part of a holistic CI/CD pipeline Source Code Control Build and Test Release Deploy, Monitor and Log Automatically trigger CI/CD pipeline based on code check-in. Start automated build and test, including functional, security and performance tests. Update artifact repository with latest successful code artifacts and pull newest images Deploy applications to container orchestration and watch with monitoring and logging CONTINUOUS INTEGRATION (CI) CONTINUOUS DEPLOYMENT (CD) Marathon

80. © 2018 Mesosphere, Inc. All Rights Reserved. 81 ● CNI - Container Networking Interface ● Kube-DNS - Kubernetes DNS server ● Kube-proxy - East-west loadbalancer ● Ingress - North-South loadbalancer Networking in Kubernetes

93. © 2016 Mesosphere, Inc. All Rights Reserved. 94 1. Install Kubectl a. https://kubernetes.io/docs/tasks/tools/install-kubectl/ 2. Install cli (only if you installed from UI) a. dcos package install kubernetes --cli 3. Configure Kubectl a. dcos kubernetes kubeconfig b. Kubectl config view Setting up Kubectl

97. Zero Touch Self-Healing Robust API Server Auth Non-disruptive Upgrades Scalable Kubernetes Transport Layer Security Cloud Native Services Simple HA Cluster Provisioning Automated self-healing

98. ServerServer Server Application Aware Scheduling Automated Self Healing Server Server Server Server Server Server Server Server KubernetesZero Touch 20+ MORE Cloud Native Services

99. © 2017 Mesosphere, Inc. All Rights Reserved. 100 Kill kubelet process ● dcos task exec -it kube-node-0-kubelet bash ● ps aux | grep "./kubelet " root 62 2.3 0.6 741984 100580 ? Sl 15:16 3:25 ./kubelet --address=10.0.4.5 --hostname-override=kube-node-1-kubelet.kubernetes.mesos ● kill -9 62

103. One-Click Scalability and Upgrades Robust API Server Auth Transport Layer Security Cloud Native Services Simple HA Cluster Provisioning Automated self-healing Scalable Kubernetes Non-disruptive Upgrades

107. © 2018 Mesosphere, Inc. All Rights Reserved. 108 Update to next K8s Version `dcos package install kubernetes --cli --package-version=1.0.2-1.9.6` And `dcos kubernetes update --package-version=1.0.2-1.9.6`

109. © 2018 Mesosphere, Inc. All Rights Reserved. 110 SMACK Stack Generator Display 1. Financial data created by generator 2. Written to Kafka topics 3. Kafka Topics consumed by Spark or Flink 4. Results written back into Kafka stream (another topic) 7. Results displayed

110. © 2018 Mesosphere, Inc. All Rights Reserved. 111 SMACK Stack Generator Display 1. Financial data created by generator 2. Written to Kafka topics 4. Results written back into Kafka stream (another topic) 5. Results displayed 3. Kafka Topics consumed and analyzed by Flink Kubernetes Cluster (running on top of DC/OS)

114. © 2018 Mesosphere, Inc. All Rights Reserved. 115 Deploy Microservices # Deploy Display $ kubectl apply -f https://raw.githubusercontent.com/dcos-labs/dcos-kubernetes- workshop/master/flinkDemo/flink-demo-actor.yaml # Deploy Generator $ kubectl apply -f https://raw.githubusercontent.com/dcos-labs/dcos-kubernetes- workshop/master/flinkDemo/flink-demo-generator.yaml $ kubectl get pods $ kubectl logs flink-demo-actor-655890656-8d1ls

117. 11 Sign Up For Slack 1. Slack URL: https://chat.dcos.io/ 2. Join #kubernetes channel 3. OSS support / feedback

118. © 2018 Mesosphere, Inc. All Rights Reserved. 119 THANK YOU! ANY QUESTIONS? @dcos users@dcos.io /groups/8295652 /dcos /dcos/examples /dcos/demos chat.dcos.io https://github.com/mesosphere/dcos-kubernetes-quickstart https://mesosphere.com/blog/another-kubernetes-service/

119. CONFIDENTIAL Make it insanely easy to build and scale world-changing technology

120. © 2018 Mesosphere, Inc. All Rights Reserved. 121 Step 1 ssh in to your bootstrap node with the centos/FindBlissInIgnorance credentials Step 2 cd into the ~/tools/dcos-ansible directory: $ cd ~/tools/dcos-ansible Enter IP addresses Warning Make sure you use your respective 10.0.0.X IPs in the hosts file a Step 3 Execute the following command to start the installation of DC/OS 1.11: $ ansible-playbook -i hosts --private-key student_* --extra-vars "dcos_version=dcos-1.11" main.yaml It will take several minutes to run through the deployment. We will verify the installation was successful in Lab 1: Upgrading DC/OS shortly.

Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)

QAware GmbH

Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)