Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

CodeOne 2019: "Continuous Delivery with Docker and Java"

206 visualizaciones

Publicado el

Implementing a continuous delivery (CD) pipeline for Java applications is not trivial, and the introduction of container technology to the development stack can bring additional challenges and requirements. This session looks at key lessons learned when creating an effective pipeline for building and deploying Docker container-based Java applications. Key takeaways include:

• The impact on Java and continuous delivery of using Docker containers
• The benefits and challenges of packaging containerized Java applications
• Validating nonfunctional/operational requirement changes imposed by executing Java applications within a container
• Lessons learned the hard way (in production, at 3 a.m., with lots of coffee)

Publicado en: Software
  • Sé el primero en comentar

CodeOne 2019: "Continuous Delivery with Docker and Java"

  1. 1. Continuous Delivery with Docker Containers and Java The Good, the Bad, and the Ugly Daniel Bryant @danielbryantuk
  2. 2. Containers: Expectations versus reality 18/09/2019 @danielbryantuk “DevOps”
  3. 3. @danielbryantuk • Tech Consultant, Product Architect at Datawire,… • Ex-academic, software developer, DBA, ops, CTO, conference tourist • Java Champion and continuous delivery advocate • Leading change through technology and teams 18/09/2019 @danielbryantuk bit.ly/2jWDSF7 oreil.ly/2RgU3Pe
  4. 4. https://www.infoq.com/articles/api-gateway-service-mesh-app-modernisation/ https://www.youtube.com/watch?v=o1MJi54_R4o&list=PLj6h78yzYM2PpmMAnvpvsnR4c27wJePh3&index=179
  5. 5. Continuous Delivery && Docker 18/09/2019 @danielbryantuk
  6. 6. Velocity (with stability) is key to business success “Continuous delivery is achieved when stability and speed can satisfy business demand. Discontinuous delivery occurs when stability and speed are insufficient.” - Steve Smith (@SteveSmithCD) 18/09/2019 @danielbryantuk
  7. 7. Velocity (with stability) is key to business success “Continuous delivery is achieved when stability and speed can satisfy business demand. Discontinuous delivery occurs when stability and speed are insufficient.” - Steve Smith (@SteveSmithCD) 18/09/2019 @danielbryantuk
  8. 8. 18/09/2019 @danielbryantuk Feedback: - Was our initial hypothesis proven? - How can we improve business, architecture and ops?
  9. 9. The good (with Docker and Java) • Dev environment setup can Dockerized • Docker enables repeatable builds • Legacy tech (old frameworks etc) can be hermetically sealed 18/09/2019 @danielbryantuk
  10. 10. The bad (lessons learned for speed/stability) “Why is the container image 1GB? It’s a helloworld Java app!!!” “Dinosaurs must have completed their dev/test/deploy loop faster than me” “This Java app runs slow (or freezes) in Docker” 18/09/2019 @danielbryantuk
  11. 11. Impact of container tech on CD 18/09/2019 @danielbryantuk
  12. 12. 18/09/2019 @danielbryantuk
  13. 13. 18/09/2019 @danielbryantuk The container image is the single binary “Build Binaries Only Once (BBOO)”
  14. 14. Lessons learned 18/09/2019 @danielbryantuk
  15. 15. 18/09/2019 @danielbryantuk
  16. 16. Make your dev environment like production • Must build/test containers locally • Perform (at least) happy path tests • Use identical base images from production • With same configuration 18/09/2019 @danielbryantuk
  17. 17. Lesson learned: Dockerfile content is super important • OS choice (alpine or Distroless?) • OS configuration • Build artifacts • Ports, volumes etc • Oracle vs OpenJDK vs …? • JDK vs JRE (vs jlinked binary?) • Hotspot vs OpenJ9 vs SubstrateVM vs…? • AOT vs JIT && CDS and ACDS 18/09/2019 @danielbryantuk
  18. 18. Please talk to the sysadmin people: Their operational knowledge is invaluable 18/09/2019 @danielbryantuk
  19. 19. Start from good foundations: base image 18/09/2019 @danielbryantuk https://github.com/danielbryantuk/oreilly-docker-java- shopping
  20. 20. 18/09/2019 @danielbryantuk https://adoptopenjdk.net/
  21. 21. Start from good foundations: base image 18/09/2019 @danielbryantuk https://github.com/danielbryantuk/oreilly-docker-java- shopping
  22. 22. Getting smaller, post Java 9… • Use jlink to create custom JRE • Jlink binary contains only: • Your app modules • Dependencies (JARs, modules) • JRE modules needed • Portola Project / Azul Zulu 18/09/2019 @danielbryantuk
  23. 23. But, why is my image so big? 18/09/2019 @danielbryantuk
  24. 24. 18/09/2019 @danielbryantuk
  25. 25. 18/09/2019 @danielbryantuk
  26. 26. Take a “dive” into a container 18/09/2019 @danielbryantuk https://github.com/wagoodman/dive
  27. 27. 18/09/2019 @danielbryantuk
  28. 28. 18/09/2019 @danielbryantuk
  29. 29. Building in containers (multi-stage FTW) 18/09/2019 @danielbryantuk
  30. 30. BuildKit 18/09/2019 @danielbryantuk
  31. 31. The bad: different test and prod containers? • Create “test” version of container • Full OS (e.g. Ubuntu), JDK • Test tools and data • Create “prod” version of the container • Minimal OS • JRE only • Easy to see app/configuration drift 18/09/2019 @danielbryantuk
  32. 32. The bad: different test and prod containers? 18/09/2019 @danielbryantuk
  33. 33. Working remotely, locally… 18/09/2019 @danielbryantuk https://opencredo.com/working-locally-with-microservices/ https://www.telepresence.io/
  34. 34. Lots of tooling in DevEx space 18/09/2019 @danielbryantuk https://blog.hasura.io/draft-vs-gitkube-vs-helm-vs- ksonnet-vs-metaparticle-vs-skaffold-f5aa9561f948 https://www.slideshare.net/dbryant_uk/mucon-2019-creating-an- effective-developer-experience-for-cloudnative-apps-148402134
  35. 35. 18/09/2019 @danielbryantuk
  36. 36. Lesson learned: Metadata is valuable • Application metadata • Version / GIT SHA • Build metadata • Build date • Image name • Vendor • Quality metadata • QA control, signed binaries, ephemeral support • Security profiles (AppArmor), Security audited etc 18/09/2019 @danielbryantuk
  37. 37. Metadata - Adding Labels at build time • Microscaling Systems’ Makefile • Labelling automated builds on DockerHub (h/t Ross Fairbanks) • Create file ‘/hooks/build’ • label-schema.org • microbadger.com 18/09/2019 @danielbryantuk
  38. 38. External registry with metadata support 18/09/2019 @danielbryantuk
  39. 39. 18/09/2019 @danielbryantuk
  40. 40. Running tests with containers 18/09/2019 @danielbryantuk
  41. 41. Testing NFRs in the build pipeline • Performance and Load testing • Gatling / jmeter / Flood.io • Security testing • Findsecbugs / OWASP Dependency check • Bdd-security (OWASP ZAP) / Arachni • Gauntlt / Serverspec • Docker Bench for Security / CoreOS Clair 18/09/2019 @danielbryantuk
  42. 42. (Technical Speed): Docker and Java • Migrate to Java 11 LTS • Eclipse OpenJ9 • www.eclipse.org/openj9/oj9_performance.html • Ahead of Time (AOT) Compilation • OpenJ9 and SubstrateVM • Application Class Data Sharing (ACDS) • blog.codefx.org/java/application-class-data-sharing/ 18/09/2019 @danielbryantuk https://www.ibm.com/developerworks/library/j-rtj2/
  43. 43. Stability: Docker and Java • Watch for JVM cgroup/taskset awareness (with JDK <= 8u131) • getAvailableProcessors() may incorrectly report the number of cpus in Docker (JDK-8140793) • Runtime.availableProcessors() ignores Linux taskset command (JDK-6515172) • GC threads, default fork/join thread pool sizes (and others) is based from host CPU count • Set container memory appropriately • JVM requirements = Heap size (Xmx) + Metaspace + JVM overhead • Account for native thread requirements e.g. thread stack size (Xss) • Entropy • Host entropy can soon be exhausted by crypto operations and /dev/random blocks • -Djava.security.egd=file:/dev/./urandom (notes on this) 18/09/2019 @danielbryantuk 43
  44. 44. 18/09/2019 @danielbryantuk https://docs.google.com/presentation/d/11VjOwW8MjDqXX9uRx0BEGYrIQtGGcXJJWMxS2q- 02nA/edit#slide=id.g3c0528a66b_1_162
  45. 45. Security: Basic (Java) Code Scanning 18/09/2019 @danielbryantuk
  46. 46. Security: Dependency Scanning 18/09/2019 @danielbryantuk www.owasp.org/index.php/OWASP_Dependency_Check
  47. 47. Security: Container Images github.com/arminc/clair- scanner
  48. 48. Delaying NFRs to the ‘Last Responsible Moment’ Newsflash! Sometimes the last responsible moment is up-front Modern platforms/architectures don’t necessarily make this easier 18/09/2019 @danielbryantuk
  49. 49. Summary 18/09/2019 @danielbryantuk
  50. 50. In summary • Docker and Java are a great combination • But make sure you understand the technology and challenges • Continuous delivery is essential with modern architecture/tech • Container images must be the (single) source of truth within pipeline • Provenance (metadata) and validation (testing NFR) of builds is vital • Not all developers are operationally aware 18/09/2019 @danielbryantuk
  51. 51. Thanks for listening… Twitter: @danielbryantuk Email: daniel.bryant@tai-dev.co.uk Writing: https://www.infoq.com/profile/Daniel-Bryant Talks: https://www.youtube.com/playlist?list=PLoVYf_0qOYNeBmrpjuBOOAqJnQb3QAEtM 18/09/2019 @danielbryantuk bit.ly/2jWDSF7 oreil.ly/2RgU3Pe
  52. 52. Bedtime reading 18/09/2019 @danielbryantuk

×