Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Test Smart, not hard

163 visualizaciones

Publicado el

Deploy distributed systems faster and safer with contract tests

It’s 2018 and we still rely on integrated environments and large end-to-end test suites to release complex ecosystems, also called “software". In this talk, Matt breaks down the arguments for such nonsense and provides a better, faster, safer alternative.

From https://www.meetup.com/sfjava/events/255379906/

Publicado en: Tecnología
  • Sé el primero en comentar

Test Smart, not hard

  1. 1. PRESENTED BY Matt Fellows (@matthewfellows) Test S-M-R-T not HARD Deploy distributed systems faster with contract tests and safer! v
  2. 2. ABOUT ME 2 1. Principal Consultant at DIUS 2. Pact since 2014 3. Would work in sports/fitness if not IT @matthewfellows
  3. 3. A story
  4. 4. Soundify ® A STORY
  5. 5. Everyone is doing microservices APIs A STORY
  6. 6. A STORY
  7. 7. Metric Value No. teams 1 No. components 1 Test environments 1 Time in pipeline (commit to prod) 5 minutes Risk 2.5% Deployments per day 10 A STORY > CD HEALTH CHECK
  8. 8. A STORY
  9. 9. Metric Value No. teams 2 No. components 2 Test environments 4 Time in pipeline (commit to prod) 10 minutes Risk 5% Deployments per day 10 A STORY > CD HEALTH CHECK
  10. 10. A STORY
  11. 11. A STORY
  12. 12. Metric Value No. teams 3 No. components 6 Test environments 18 Time in pipeline (commit to prod) 30 minutes Risk 10% Deployments per day 1 A STORY > CD HEALTH CHECK
  13. 13. ACQUIRED A STORY
  14. 14. A STORY
  15. 15. Metric Value No. teams 10 No. components 20 Test environments 20 200 Time in pipeline (commit to test) 120+ minutes Risk 20% Deployments per month 1 A STORY > CD HEALTH CHECK
  16. 16. Lessons?
  17. 17. So what did we learn? A STORY > LESSONS
  18. 18. A STORY > LESSONS
  19. 19. Want: O(1) or O(n) Got: O(n2 ) or worse A STORY > LESSONS
  20. 20. Where did we go wrong? A STORY > LESSONS
  21. 21. Cohesive services, loosely coupled (want) vs Cohesive teams, tightly coupled by services (got) A STORY > LESSONS
  22. 22. Previous tooling and strategies were not good enough A STORY > LESSONS
  23. 23. “Integration tests are a scam” - JB Rainsberger A STORY > LESSONS > INTEGRATION TESTS
  24. 24. A STORY > LESSONS > INTEGRATION TESTS Scam, you say? Justify! Integrated tests are: ● Slow ● Fragile ● Hard to manage When they fail, you can’t point to the problem!
  25. 25. “But my integration tests run in Docker, why can’t I use them?” - People A STORY > LESSONS > INTEGRATION TESTS
  26. 26. “Because Maths” - Me A STORY > LESSONS > INTEGRATION TESTS
  27. 27. A STORY > LESSONS > INTEGRATION TESTS
  28. 28. A STORY > LESSONS > INTEGRATION TESTS
  29. 29. Branches per box vs test cases required 2 code branches = 128 tests 5 code branches = 78,125 tests 10 code branches = 10M tests A STORY > LESSONS > INTEGRATION TESTS
  30. 30. Good tests have the exact opposite properties A STORY > LESSONS > INTEGRATION TESTS
  31. 31. Mocks to the rescue? A STORY > LESSONS > INTEGRATION TESTS
  32. 32. A STORY > LESSONS > INTEGRATION TESTS
  33. 33. A STORY > LESSONS > INTEGRATION TESTS
  34. 34. Dictator Driven Contracts A STORY > LESSONS > API DESIGN
  35. 35. Dictator Driven Contracts A STORY > LESSONS > API DESIGN
  36. 36. A STORY > LESSONS > API DESIGN How to: dictator driven contracts 1. Sit in ivory tower and postulate 2. Document perfect API (Swagger/OAS etc.) 3. Create said API 4. Publish said document to consumers 5. Repeat steps 1-4
  37. 37. A STORY > LESSONS > API DESIGN
  38. 38. A STORY > LESSONS > API DESIGN
  39. 39. A STORY > LESSONS > API DESIGN
  40. 40. Dictator Consumer Driven Contracts A STORY > LESSONS > API DESIGN
  41. 41. A STORY > LESSONS > API DESIGN
  42. 42. Benefits? A STORY > LESSONS > API DESIGN
  43. 43. You’ll know when you break a consumer A STORY > LESSONS > API DESIGN
  44. 44. You have a form of documentation A STORY > LESSONS > API DESIGN
  45. 45. You can test things independently A STORY > LESSONS > API DESIGN
  46. 46. Pact www.pact.io
  47. 47. Evolved from combining these two principles PACT
  48. 48. ● Open source (MIT) ● Multiple languages ○ Javascript ○ Go ○ Python ○ JVM ○ .NET ○ + more pact.io ● HTTP contracts ● Message contracts ● … protocol agnostic! PACT
  49. 49. PACT > WHO’S USING IT?
  50. 50. PACT > STEP 1: WRITE A CONSUMER TEST
  51. 51. PACT > STEP 1: WRITE A CONSUMER TEST
  52. 52. PACT > STEP 1: WRITE A CONSUMER TEST
  53. 53. Given “User A exists” When I Receive “a GET request for user A” With “content-type: application/json” Respond with “200 OK” And “User A’s details in the body” PACT > STEP 1: WRITE A CONSUMER TEST
  54. 54. Step 2: Share your pacts PACT > STEP 2: SHARE CONTRACT
  55. 55. PACT > STEP 3: VERIFY PROVIDER
  56. 56. PACT > NO SILVER BULLET No Silver Bullet
  57. 57. Are not functional tests Contracts tests PACT > NO SILVER BULLET
  58. 58. Are not load performance tests SLAs Contracts tests PACT > NO SILVER BULLET
  59. 59. Are not good for “public” APIs (many, unknown consumers) (Consumer) Contracts tests ANOTHER WAY @bethesque
  60. 60. PACT > NO SILVER BULLET
  61. 61. PACT > NO SILVER BULLET
  62. 62. PACT > NO SILVER BULLET Pact
  63. 63. PACT > NO SILVER BULLET Do I still need integration tests?
  64. 64. PACT > NO SILVER BULLET Probably, maybe, yes?
  65. 65. PACT > NO SILVER BULLET Integration test coverage Consequences of bug x Time to find bug x Time to release fix
  66. 66. DEMO
  67. 67. BEST PRACTICES Integrating with your CICD processes
  68. 68. BEST PRACTICES 68
  69. 69. BEST PRACTICES > CHALLENGES 69 1. Sharing and collaborating on pacts between teams 2. Managing contracts across code branches and environments 3. Orchestrating builds to know when it is safe to deploy
  70. 70. Introducing... The Pact Broker
  71. 71. BEST PRACTICES > PACT BROKER 71 Integrated collaboration tool to and share and manage pacts* between teams. ● API documentation that is guaranteed to be up-to date ● Visualisations of the relationships between your services ● Dashboards containing contract verification status ● Pact tagging and versioning ● Webhooks for integration and communication ● View pact diffs ● ...and more! All powered by a RESTful API
  72. 72. BEST PRACTICES > PACT BROKER 72 Did I mention README badges and integrations?
  73. 73. DEMO BEST PRACTICES > PACT BROKER > VISUALISATIONNS 73
  74. 74. DEMO BEST PRACTICES > PACT BROKER > VISUALISATIONNS 74
  75. 75. Replace lengthy integration test pipelines with “The Matrix” BEST PRACTICES > PACT BROKER > PIPELINES 75
  76. 76. BEST PRACTICES > PACT BROKER > CAN I DEPLOY? 76 ● Verify that a Pacticipant is safe to deploy prior to release ● Enables “matrix” style testing Consumer Head (1.0.1) Consumer Prod (1.0.0) Provider Head (2.0.0) ? ? Provider Prod (1.1.13) ? Already tested
  77. 77. BEST PRACTICES > PACT BROKER > CAN I DEPLOY? 77 Can also do with tags... Consumer Head (‘dev’) Consumer Prod (‘prod’) Provider Head (‘dev’) ? ? Provider Prod (‘prod’) ? Already tested
  78. 78. BEST PRACTICES > PACT BROKER > “THE MATRIX” 78 $ pact-broker can-i-deploy --app A --version 1 --app B --version 4
  79. 79. BEST PRACTICES > PACT BROKER > CAN I DEPLOY? 79
  80. 80. BEST PRACTICES > PACT BROKER > CAN I DEPLOY? 80 Computer says yes o/
  81. 81. DEMO BEST PRACTICES > PACT BROKER 81
  82. 82. BEST PRACTICES > CI/CD 82
  83. 83. BEST PRACTICES > CI/CD > PROVIDER 83
  84. 84. BEST PRACTICES > CI/CD > CONSUMER 84
  85. 85. Best Practices: Summary
  86. 86. Summary Best Practices ● Pact Broker is key for scaling ● Use webhooks for team communication ○ But… keep build triggers simple! ● Tag pacts to mirror your git branches ● Use can-i-deploy tool for deployment pre-flight checks ● Update pact tags after deployment to environment (e.g. prod)
  87. 87. Recapping... ● Business impact of integrated tests and environments ● Cause and explanation of those effects ● Alternative approach - isolation + contracts ● Contract-testing as an approach, Pact as a tool (in this order!) ● CDC has its own challenges ○ Not suitable for external/public APIS ○ effective collaboration is the key
  88. 88. Thank you - @matthewfellows - @pact_up Given “The presentation is over” Upon Receiving “A request for an answer” With “A valid question” Respond With “A valid answer”
  89. 89. What about Swagger / Open API?
  90. 90. A B mock/write C P pact swagger verify verify Pact+Swagger OTHER TOOLS
  91. 91. No Silver Bullet
  92. 92. What about systems maintained by other teams?
  93. 93. What about systems built in outdated technologies?
  94. 94. Scary outside world! Websphere? Mainframe
  95. 95. Thank you - @matthewfellows Given “The presentation is over” Upon Receiving “A request for an answer” With “A valid question” Respond With “A valid answer”

×