Charles Korn- Build & Testing Environments as Code: Because Life's Too Short Not To (Evolution)
•Descargar como PPTX, PDF•
2 recomendaciones•746 vistas
The value of defining our production infrastructure as code is unquestionable, but in 2018 we are still struggling with our local build and testing environments. All too often, starting a new project is an incredibly painful process of working out which tools we need to install, which versions will happily coexist together, and configuring them to make them behave as we would like. As the application evolves and changes over time, maintaining and updating this environment across all developer machines and CI agents is difficult enough, let alone thinking about integration and journey testing, where we'd like to quickly and reliably spin up external dependencies like databases and other services in a known state to test against. Charles will share practical advice on running your builds and tests quickly, consistently, and completely automatically, on your machine, your colleagues' machines and on CI, as well as how to be up and running in just a few minutes without the overhead of virtual machines, thanks to the power of Docker.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (9)
Similar a Charles Korn- Build & Testing Environments as Code: Because Life's Too Short Not To (Evolution)
Similar a Charles Korn- Build & Testing Environments as Code: Because Life's Too Short Not To (Evolution) (20)
Más de Thoughtworks
Más de Thoughtworks (20)
Último
Último (20)
Charles Korn- Build & Testing Environments as Code: Because Life's Too Short Not To (Evolution)
- 1. CHARLES KORN Software Developer @charleskorn #EvolutionTW #ThoughtWorks
- 2. BUILD AND TESTING ENVIRONMENTS AS CODE …because life’s too short not to
- 3. Dollars and Sense, Inc.
- 6. Build tools Libraries RuntimesCode IDE / text editor Version control tool Build environment
- 7. Development environment Build environment Testing environment(s)
- 11. International transfers service Data store Exchange rate service Data store Kafka CAD CHF EUR GBP SEK USD AUD
- 12. WHAT’S WRONG WITH THE STATUS QUO?
- 14. (In)consistency
- 15. Onboarding time Average onboarding time: 2 hours to 2 weeks
- 16. Cost of change
- 17. No isolation
- 18. Overhead
- 19. Team autonomy (or lack thereof)
- 21. Flakiness 84% of new test failures at Google are due to flakiness
- 22. It’s painful
- 23. It’s painful …which leads to further pain
- 25. Inconsistent High onboarding time High cost of change No isolation High overhead Lack of team autonomy Flakiness It’s painful Debugging and cycle times
- 26. THE IDEA
- 28. Your computer or a CI agent Build container Build tools Libraries Runtimes Code directoryCode directory IDE / text editor Version control tool
- 29. Consistency
- 30. Lightweight
- 32. Quick onboarding time 30 minutes and 4 seconds from bare metal to running app
- 33. Quick setup time
- 34. Low cost of change
- 35. batect build and test environments as code tool
- 36. DEMO
- 37. Test environment
- 41. International transfers service Postgres database Exchange rate service Test driver
- 44. Integration and journey testing on CI
- 45. International transfers service Postgres database Exchange rate service Test driver
- 46. International transfers service Postgres database Exchange rate service Test driver
- 47. International transfers service Postgres database Exchange rate service Test driver
- 48. International transfers service Postgres database Exchange rate service Test driver
- 49. International transfers service Postgres database Exchange rate service Test driver
- 50. International transfers service Postgres database Exchange rate service Test driver
- 51. International transfers service Postgres database Exchange rate service Test driver
- 52. batect build and test environments as code tool
- 53. “What about Docker Compose?”
- 54. DEMO
- 55. (with apologies to the Beatles)
- 57. BUT…
- 58. IDE integration
- 60. Inconsistent High onboarding time High cost of change No isolation High overhead Lack of team autonomy Flakiness It’s painful Debugging and cycle times
- 61. Consistent Quick onboarding time Low cost of change Isolated Low overhead Enables team autonomy Reduces flakiness Quick and easy Easy debugging and short cycle times Inconsistent High onboarding time High cost of change No isolation High overhead Lack of team autonomy Flakiness It’s painful Debugging and cycle times
- 62. SO NOW WHAT?
- 63. Start small, work incrementally
- 65. International transfers service Postgres database Exchange rate service Test driver
Notas del editor
- CLICKER ON Thank you for coming Introduce myself A technique I’ve been using for nearly two years A side project of mine for last ~year Using containers for your development environment I’ve used this in the past - apps and libraries Scala backend, Android app, Ruby app, Rust app, Golang, Java, Lambda… Used it at my last two clients Technique makes developer experience much better SLOW DOWN
- Let’s pretend: Dollars and Sense Development team for international transfers system (explain what that is) Customers want to be able to transfer money from one country to another Might involve a currency conversion as well
- We’re responsible for everything in orange Service in Java Postgres database Consumers make calls to our service to make transfers Depend on exchange rate service in grey Communicate over HTTP Fairly common microservices style architecture Two questions I want to answer today: How do we take code and produce an executable artifact from it? How do we test that artifact before it goes into production? SLOW DOWN
- This is where the development environment comes in Two main parts Build environment Tools and configuration needed to build, unit test and package your application
- Build environment Compilers Build tools Testing tools ...whatever is needed to go from source code to a unit tested artifact potentially ready for deployment In our example: Java code to JAR Deliberately exclude IDEs etc - part of the development environment, not part of the build environment
- Testing environment External dependencies for integration tests etc. eg. databases, queues, caches, other services etc. Might use fakes for some things and actual code for other components Might have multiple configurations (eg. integration testing might have a different configuration compared to journey testing) In our example: service needs database and exchange rate service Our example is pretty simple, but this can quickly become quite complex
- How is this normally handled?
- Install all the things Most common way Install everything locally Our example: JVM, build tools, Postgres etc. Install dependencies locally Configure locally Usually done manually Ditto for colleagues Ditto for CI agents Ditto for test environments
- Development VM Ideally scripted, installs and configures everything automatically Vagrant is popular for this SLOW DOWN
- Shared test / integration environments (explain what that is - where we can run the application in a production-like environment for testing with external dependencies) (explain example - advance three times) Real running software All dependencies Representative data Usually built and used by multiple teams (each of the different colours)
- (audience participation) Who uses these techniques? (follow up) What are some of the issues you run into? What are the problems with these approaches?
- For build environments, there are a number of issues SLOW DOWN
- Inconsistency Very easy to get into an inconsistent or undesired state “Configuration drift” Nothing guarantees that your setup is the same as your colleagues’ Even if scripted: nothing stops you from changing stuff Ditto for you compared with CI Less of an issue for Vagrant, but can still drift Impact? Means you might have problems executing particular tasks - just doesn’t work Might have small issues that are difficult to identify and diagnose
- Onboarding time New developer joins our team at Dollars and Sense, how long does it take for them to be productive? (advance) Survey: anywhere between 2 hours and 2 weeks Not documented: cue days / weeks of trying to piece everything together Documented but not scripted: documentation probably not up-to-date or incomplete Even if it is scripted or perfectly documented, can take a while
- Cost of change Version updates Configuration changes New tools or components Especially hard if setup isn’t scripted - explain why (have to communicate, manually do stuff etc.) Even if scripted, nothing compels you to update your local configuration once it changes, you have to be disciplined, still have to communicate with team Even Vagrant won’t automatically update your environment SLOW DOWN
- No isolation All installed tools and configuration apply globally to your machine Some tools are better at this than others (eg. Ruby with RVM, Node / NVM) Even if things are meant to support side-by-side installation, nothing guarantees this Build up cruft over time Working on multiple projects with conflicting requirements becomes painful
- Overhead Not an issue for local installation VMs are very heavyweight Use lots of disk space, memory Therefore difficult to run multiple projects at once Because of this, most of the time VMs are not used on CI Go back to all the issues with the ‘install all the things’ approach And now have two ways of managing environments
- Team autonomy Development environment affects team autonomy Development environment influences tools, languages, components used Teams need to be able to pick the things that make sense for them and their situation One particular way this manifests itself (esp. in large organisations): CI Ideally you’d have team-specific CI instances, but that doesn’t always happen Often have shared infrastructure End up in one of two situations: All projects constrained to use the same thing No autonomy Projects have specialised CI agents within shared infrastructure Leading to an explosion of different configurations (can be difficult to manage) Or tension encouraging teams to not stray from the beaten path - reduces autonomy
- Similar issues for test environments Won’t repeat them all, just important ones
- Flakiness Fun fact: 84% of new test failures at Google are due to flakiness Leads to alarm fatigue, ignoring failing tests One of the biggest causes of flaky tests: environmental issues like inconsistency Survey: every response mentioned flakiness of some sort as a contributor to test failures - none mentioned logic issues Source: https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html
- It’s painful Setting up dependencies (eg. data stores, other services etc.) is a pain Have to do this on each developer machine Have to do this on CI agents potentially As system evolves, have to keep these in sync, update and maintain them As system grows, more dependencies = more pain Survey: some teams spent up to a person-week maintaining environments every month SLOW DOWN
- Because this is so painful, often share test environments other teams to reduce effort of maintaining them Which makes everything harder Have to coordinate across teams when making changes …or teams just change things + break stuff for other teams Have to find a setup that works for all teams, rather than being optimised for your team All these teams are responsible for the environment, which means no team is responsible for the environment, leads to it not being looked after
- Debugging in test environment This setup makes debugging things harder as well Generally really difficult to spin up the test environment locally Survey: over 50% couldn’t run integration / journey / functional tests locally but wanted to …so you have to debug issues in a shared environment Bad if this environment is shared by your team, even worse if it’s shared by multiple teams Your debugging impacts other people’s work and vice versa Also: cycle time is too long (code change to running in environment takes minutes to hours)
- Not a big fan Inconsistent High onboarding time High cost of change No isolation High overhead Lack of team autonomy Flakiness It’s painful Debugging and cycle times
- The idea How can we use containerisation to make this much better? SLOW DOWN
- Let’s start with the build environment
- The idea (explain what’s in the diagram) Picture will be familiar to people who’ve used Docker for build environments before Every time we run a command, we start a new container just for that command, then tear it down afterwards (link to following slides) What are the benefits of this particular approach?
- Consistency Can guarantee the same result everywhere - your machine, colleagues’ and CI No configuration drift Due to ephemeral nature Any changes you make are lost as soon as the container stops at the end of the command Because you get a new container every time, you use the currently checked out configuration every time Always have a repeatable, known, clean environment
- Lightweight Nature of containers means that it’s very lightweight No overhead of running a whole VM No whole new OS to run or store on disk Makes it very fast to start and run Makes it possible to work on multiple projects at once eg. library and application, backend and frontend No issues running on CI - no resource contention
- Isolated Nature of containers Everything is in its own isolated universe Not affected by things outside the container Things outside the container aren’t affected by it -> Can work on multiple projects in parallel safely and easily
- Quick to onboard new team members All you need is Docker and your version control tool (eg. Git) Everything else is automatic Talked about two hours to two weeks earlier… (advance) 30m 4s to go from bare metal to build and tests running (20m OS install and VM extensions, ~4m 30s for Docker and Git, ~5m 30s to run everything for the first time) Mauro anecdote - meaningful commit by lunch
- Doesn’t come at a cost of setup time Quick to set up for the first time - use existing Docker images or easily build your own
- Low cost of change When I say change: mean new tool, new version, new configuration Existing image or Dockerfile defines the build environment Tools Configuration So any time we want to make changes, we just change image or update the Dockerfile Because configuration lives alongside code, it’s versioned alongside the code As soon as we commit, everyone gets the change Next time someone runs a command, they pick up the new environment If we need to go back in history, environment changes too
- batect Technique I’ve been using for a while Managing all of this is non-trivial Tried a number of tools over a number of different projects No good tools out there None quite felt right Will talk about one tool in a bit Doing this in a performant way is non-trivial Dealing with proxies is non-trivial Dealing with file permissions issues with Docker on Linux is non-trivial Making sure everything is cleaned up afterwards is non-trivial Creating a good developer experience + good ease of use is tricky Built the tool I’d want to use
- That’s the build environment, let’s talk about test environments Technique’s and batect’s strength All of the benefits from before transfer across Consistency Isolation Quick onboarding Low cost of change No longer need a person-week a month
- How do we test our service when it’s running with its dependencies? (reiterate parts of the app)
- Generally have two kinds of tests: Different people have different terminology Integration tests with individual components (individual classes tested against fake downstream services + real DB) Journey tests that test all parts in end-to-end scenarios from the outside (spin up service against fake services + DB) If you have a UI, might call this functional testing
- If we wanted to run those kinds of tests, could we apply some of the principles we used for the build environment? Goal: frictionless development experience
- Yes - we can run our service and its dependencies as containers as well (advance animation) And we can run our tests from a container too In this example, we actually reuse our build container with Gradle and JUnit for this, but this could be anything you want If we were building something with a UI, could use headless Chrome + Selenium here Thanks to Docker, can start all of these things quickly, run the tests, then tear everything down (link to following slides) What does this give us? SLOW DOWN
- Reduced flakiness Everyone runs the same thing, so flakiness is reduced And if you run into a flaky test, it’s easier to debug, because you can run exactly the same thing locally batect helps you wait until components are ready before starting the tests eg. wait for database to start before starting service -> no flakiness there PENG anecdote: zero flaky tests Matt anecdote: running through this presentation, said ‘reduced’ wasn’t strong enough
- Local environment Can spin everything up locally Can reliably run integration and journey testing locally Some situations where that’s not necessarily something you need Can easily do exploratory testing locally Can easily debug things locally with a short cycle time
- Integration and journey testing on CI Can now really easily and reliably run integration and journey tests on CI
- We have a few containers to manage, and coordinating this gets tricky Let’s take journey testing as an example
- First we need to build our app before we start anything
- Some images have to be built, others have to be pulled
- We also want to make sure we start things in the right order
- First, downstream service and database
- Then, once they’re ready, start the service
- Then, once the service is ready, run the tests (advance) We can also use Docker’s networking features to run this in an isolated network No issues with port conflicts Easy way to address each container within the network
- …batect takes care of all of this for you Makes it really easy to define environments like this Really easy to define different configurations for different use cases Integration test Journey test Maybe also want an exploratory manual testing config with fake services …whatever you want Most importantly: can run the exact same thing locally, on your colleague’s machines or on CI, all using the same configuration
- What about Docker Compose? #1 question I get Used it in the past for this technique Biggest issue: not designed with this use case in mind With Docker Compose, need some non-trivial script to achieve what we want 170 lines Its strength: standing up a stack But we want: pull together different components in different combinations quickly and temporarily batect makes it easy and natural batect is also significantly faster due to parallelisation On average, 2-3 seconds faster than Docker Compose - 6-14% improvement on total execution time
- Spoke earlier about the issues with CI agents - either shared but constrained to same versions or specialised for each team’s requirements With this approach: just need Docker on every CI agent Easy (and safe) to share agents between teams: Agents are all the same Teams can use whatever tools they want, with whatever configuration they want Makes it easy to scale CI resources up and down as needed Just add more of the same when needed, remove them when they’re idle
- Path to production Focused on developers so far But all of this is for nothing if we don’t get it into prod and in front of users Can reuse Docker image we use to run the app locally - build it once, test it and push it Nice connection if you’re using Docker all the way through to prod Of course, this technique is more broadly applicable - can use with Lambda, Android etc.
- but… Not perfect, there are some gotchas
- IDE integration Bit of a pain - most tools expect things to be installed locally eg. IntelliJ expects you to be targeting a locally installed JVM Could use Docker integration in some IDEs (eg. RubyMine, PyCharm) End up having to install some things locally
- iOS and OS X apps Need Xcode Can’t run OS X in Docker container, so can’t run Xcode in a container Can still use this technique to create environment around the app eg. backend services an app communicates with Works well for Android apps though
- To summarise, we go from… (advance)
- …to this Inconsistent -> consistent High onboarding time -> quick onboarding: first meaningful commit by lunch High cost of change -> low cost of change No isolation -> isolated: work on multiple projects in parallel High overhead -> low overhead Lack of team autonomy -> enables it Flakiness -> less flakiness It’s painful -> quick and easy Debugging and cycle times -> easy debugging and short cycle times
- Now that I’ve teased you with what’s possible, how can you get started / adopt this for an existing project?
- My suggestion Start small, work incrementally Focus on low-hanging fruit Build environment usually makes sense as a first step Be able to run build, unit tests, linting etc.
- Then start to look at integration testing, one dependency at a time Focus on areas with biggest bang for buck - eg. flakiest dependency
- Then, once you have those components set up, it’s easy to move to journey testing, exploratory testing setup etc
- Will send slides around Code up on GitHub now Thank you for coming Questions (or come and chat afterwards)