Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

November 15 cloud bees clusterhq meetup fli, flockerhub, and jenkins

731 visualizaciones

Publicado el

Slides from Nov 15 Meetup in San Francisco at ClusterHQ with Cloudbees talking about Jenkin, Docker Pipeline, Fli and Flockerhub.

Publicado en: Software
  • Sé el primero en comentar

November 15 cloud bees clusterhq meetup fli, flockerhub, and jenkins

  1. 1. Let’s talk CI/CD and DevOps with Cloudbees & ClusterHQ Ryan Wallner Developer Evangelist, ClusterHQ
  2. 2. What’s in here? How to get CI and Staging to use realistic data for each branch and commit from every developer
  3. 3. Docker and Jenkins today ● Jenkins is great for automating builds … but it’s not so great at managing data, this is left to external systems ● Docker and Docker Compose are great for defining and running multi or single container environments … not so true for managing data. ● How do you pull point-in-time references of data to anywhere in your CI/CD pipeline? ○ Consider Dev, Test, Staging, QA and Production.
  4. 4. Example Jenkins Workflows
  5. 5. Jenkins Master Jenkins Slave + Docker Jenkins Slave + Docker Jenkins Slave + Docker Docker Staging $ git push origin branch Notify Build Build Success? Dev Test Staging *or maybe only when you merge into master or specific staging branches
  6. 6. But.. staging and testing without realistic data isn’t great ● Does it represent what a user would see? ● Is there “enough” data to see how the application works under load? ● Is the data isolated so I can test or see how staging looks with different types or amounts of data? ● Do I know where this data came from, who created it?
  7. 7. FlockerHub and Fli FlockerHub is GitHub for data a. Hub & Catalog for Docker data volumes b. Fine-grained access controls & user mgmt c. Available initially as ClusterHQ hosted service Fli is Git for data a. cli interface for FlockerHub b. Snapshot Docker volumes c. Push & pull from FlockerHub d. Available as Apache 2.0 download
  8. 8. User View Data Servers https://flockerhub.clusterhq.com https://data.flockerhub.clusterhq.com Github/Google Data operations Session Auth Token token.txt
  9. 9. Staging / test with realistic data is great - w/ FlockerHub ● Does it represent what a user would see? Yes, I can have prod ops give us obfuscated production snapshots so we can get a realistic view of data. ● Is there “enough” data to see how the application? Yes, we can get a fresh, up-to-date snapshot from Ops teams. ● Is the data isolated so I can see how staging looks with different types or amounts of data? Yes, we can get fresh clones of a snapshot we choose each and every commit. ● Do I know where this data came from, who created it? Yes, We can track who the snapshot was created by and who has access to it.
  10. 10. Example Jenkins Workflow with Fli and FlockerHub
  11. 11. Jenkins Master Jenkins Slave + Docker Jenkins Slave + Docker Jenkins Slave + Docker Docker Staging $ git push my-branch Notify Build Build Success? Dev Test Staging *Jenkinsfile has a reference of what Snapshot to use in staging. $ fli pull *every developer, every branch. Isolated snapshots push pull
  12. 12. Jenkins Master Jenkins Slave + Docker Docker Staging $ git push my-branch Notify Build test_123 With snapshot 321 Success? Dev Test Staging *Jenkinsfile has a reference to use snapshot 321 for test 123 $ fli pull 1. pull (only changes) 2. clone snapshot 3. give clone to docker 4. docker run && test 5. snapshot volume if failure 6. docker stop/kill
  13. 13. Manage snapshots, orgs and teams https://flockerhub.clusterhq.com
  14. 14. Demo Jenkins workflow with Fli and FlockerHub
  15. 15. Thank You Sign up https://clusterhq.com/register/flockerhub-beta/ @ryanwallner
  16. 16. Walkthrough ❏ Checkout new branch to work with, push and watch build in Jenkins ❏ Show Staging ❏ Change branch to use a different snapshot that has a bad phone number or vin number ❏ Show output from error in CI ❏ Use manifest produced by failure to reproduce locally. ❏ vi fli-docker-manifest.yml (in inventory-app) ❏ fli-docker run -f fli-docker-manifest.yml -c -t /root/fh.token ❏ docker ps to show locally running re-created env with snapshot ❏ run the test that failed locally. ❏ docker run --net=flicompose_net -e FRONTEND_HOST="frontend" -e DATABASE_HOST="db" -e FRONTEND_PORT=8000 --rm -v ${PWD}/frontend/:/frontend clusterhq/mochatest "cd /frontend && npm install && mocha --debug test/test_db_vehicle_vins.js" ❏ cleanup with fli-docker destroy -f fli-docker-manifest.yml -clean
  17. 17. Backup/Example Slides
  18. 18. Test failures within Jenkins
  19. 19. Test logs show failed test and what snapshot was taken after the failure
  20. 20. Jenkins Build/Test with Snapshots/Clones Jenkins first pulls the necessary snapshots and creates a volume for the Docker app to use. [parallel tests 9] Syncing Volumeset inventory-app [parallel tests 9] Getting ID of the Snapshot 750k-records-snap [parallel tests 9] ID of snapshot is 6234bb23-15c9 [parallel tests 9] Pulling inventory-app:6234bb23-15c9 [parallel tests 9] Creating volume from Snapshot [parallel tests 9] Volume /chq/e41408a1-9e41/bd6f5bc0-f4ad created [parallel tests 9] Loading the path into the application. The volume /chq/e41408a1-9e41/bd6f5bc0-f4ad/ will be mapped into the docker-compose.yml The application boots up with a point in time snapshot of data in staging for the particular branch * http://jenkinsdemo.clusterhq.com/job/inventory-pipeline-multi/job/master/
  21. 21. Staging is available per commit with its own isolated data.
  22. 22. fli-docker https://github.com/ClusterHQ/fli-docker
  23. 23. Stateful-Apps as Code? fli-docker The fli-docker utility is designed to simplify the deployment of stateful applications inside Docker containers using a Flocker Hub Stateful Application Manifest (SAM) used side by side with the Docker Compose file.
  24. 24. In a local development environment $ cat docker-compose.yml services: db: networks: - net image: rethinkdb volumes: - rethink-data:/data ports: - 28015 - 29015 - 8080 command: rethinkdb $ cat fli-manifest.yml docker_app: docker-compose.yml flocker_hub: endpoint: data.flockerhub.clusterhq.com volumes: - name: rethink-data snapshot: someSnapshot-of-DB volumeset: inventory-app

×