14. Tungsten: Dockerfile
FROM gcr.io/karix/golang:1.9
ENV GOBIN /go/bin
COPY . /go/src/github.com/karixtech/tungsten/
WORKDIR /go/src/github.com/karixtech/tungsten/
RUN dep ensure
EXPOSE 8009
EXPOSE 8010
RUN make build
14
CNCF survey - cloud native technologies - over 200% - December 2017 - 40% enterprises use k8s in production.
Karix.io leverages k8s; send and receive messages over channels; Build and orchestrate; focus deployment pipeline; ensure staging production clusters replica; boost dev productivity - improve system reliability.
K8s objects; build pipelines, packaging, deployment; throw light optimised CPU and memory by load testing; improved dev productivity - currently working on.
State machine; set desired state - current state matches desired state; Nodes - VMs & bare metal; master controls each node; kubelet daemon - containers are running in pod
Pod - basic building block; running process; never standalone pods - cannot have state; dont get rescheduled.
Manage pods by Deployment; collection similar pods; maintain certain number pods
Service - abstraction logical set of pods - policy to access them; Load balancers; expose external network - job of balancing traffic
Build tech platform; integrate existing services - presence web mobile; Responsibility - enabling - developer tooling; goal - setup testing environment; running on VM; integration testing - staging VM ansible; Problem - one environment - QA/acceptance testing; big hurdle - major bottleneck - developer productivity; Realised - could not - single QA - meet deadlines; Goal clear - couldn’t timeshare; on demand; containers got drilled; Karix.io; decided to run containers; architecture - what microservices - independently scalable components; 2 engineers; thought process - developer time - building service - not managing infrastructure; super simple onboarding; Any product - start build pipeline; core unit - all software releases - revolve.
This is build pipeline that we worked out for our platform karix.io. I will keep coming back to this slide as we progress through our pipeline stages.
Time gap - release tag - run in production - optimized; build fails - debug the build; notified early on; container building - packaging - pushing notified; Build immutable - release gets tagged, new changes cannot be pushed - cluster in the state; Chose Jenkins - versatile tool; Jenkins Pipeline - DSL - modeling delivery pipeline; Code shipped instructions; turn concept dev-ops into reality; Service will be built - run test cases - find config - how to package; enables developers - part of their code.
Golden rule - 3 files - folder charts.
So behind the scenes our developer has pushed their code to github.
Tungsten - golang; main layers - messaging architecture
Dockerfile - generic; host base image - high security
Makefile - compiling, building, test; standard targets - little change - build script;
Jenkinsfile - groovy; calls standard Makefile - logic upload builds; runs on k8s - spawned pod using the jenkins slave base image; triggered - spawn slave Pod - run instructions - terminate job finishes; kubernetes-plugin; semantic versioning; patch staging; minor major production; release tag as version.
At this point a release tag has been created and GitHub has notified Jenkins.
Stages - Jenkinfile; split different stages - debug better; first stage fetch golang
Build tungsten image; postgres image;
test cases pass - push to registry
So back to our pipeline progress, At this point image has been built and pushed to the the container repository.
Service - route traffic; pod selector - route traffic to; external port - accept incoming traffic; target port
HPA - auto scaling spec; top of deployment
Generated from values.yml; helm package tar; tar pushed chart repository; chartmuseum
At this point the deployment package has been created and pushed to the the chart repository.
Believed deploying software - breeze; click button - single command; building packaging - get back shell. Jenkins notify built, packaged, pushed to chart repo. Deployment manual
Pushing versioned chart - update local index; install command; helm fetches the tungsten chart ; deploys CM, SVC and deployment. Rolling updates/rollbacks benefits deploying containers; devops teams - zero downtime application upgrades; overview upgrade works
tungsten pods; helm terminate signal - start booting new pod. Service incorporate - start sending traffic - readiness checks. Upgrade one by one
tungsten pods; helm terminate signal - start booting new pod. Service incorporate - start sending traffic - readiness checks. Upgrade one by one
tungsten pods; helm terminate signal - start booting new pod. Service incorporate - start sending traffic - readiness checks. Upgrade one by one
tungsten pods; helm terminate signal - start booting new pod. Service incorporate - start sending traffic - readiness checks. Upgrade one by one
At this point the package has been installed on our staging cluster and is ready for load testing.
launched karix.io - New to k8s; k8s production; load test feasibility setup - determine resources - individual pods; business requirement to benchmark capacity; came up with average spend on infrastructure vs QPS.
k6 - seamless integration with pipeline; prometheus
At this point the load testing has happened.
resource usages hit threshold - wrt - requests fired on component; trial and error; .5 CPU started benchmarking from there; came up with data no of requests wrt given resources
And finally the package gets deployed on production with the right set of resources.
deployment process established - improved dev productivity - efficient workflow
Quick uniform deployments; feature - existing bug isolated, independent clusters; feature specific cluster - without affecting staging dev; Own environment - deploy builds; low resources - financially sustainable; replica - without compatibility; Test coverage 80% = no fault code; earlier bug the better; multiple envs - bugs reduce drastically; 2 engineers - hired first engineer; deploys to production - first week; Boost confidence - sense of achievement.
k8s - rapidly developing - incorporate best practices - deploying orchestrating applications - different aspects; known technical debts - ship/release faster; improving and fixing pipeline - specific topics.
Code changes - rigmarole; “my code is building” - high debugging cycles. Gitkube - runs git remote - push changes; has instructions; Operator - extend type of applications - need to maintain state; database/cache managed - didn’t want operational overhead - persistent storage; operators evolving - explore mechanism - staging/dev; Exploring integration with vault/k8s secrets - challenging part - app needs code changes
Pulling code - restarting daemons; Ansible scripts; clusterssh - parallel ssh - deployment script; k8s - not one-stop solution; rethink your philosophy; onus of uptime scaling k8s master - desired state always maintained.