To TDD or not to TDD - that is the question
•Descargar como PPTX, PDF•
0 recomendaciones•306 vistas
A test-driven development experience report based on the 10+ year history of the Jenkins git plugin. Provides examples and heuristics for cases where test-driven development may not be the most effective use of time.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a To TDD or not to TDD - that is the question
Similar a To TDD or not to TDD - that is the question (20)
Más de Mark Waite
Más de Mark Waite (9)
Último
Último (20)
To TDD or not to TDD - that is the question
- 1. TO TDD OR NOT TO TDD – THAT IS THE QUESTION Test-driven development: lessons from the Jenkins Git Plugin Mark Waite 28 Feb 2018 Agile Boulder
- 2. AN EXPERIENCE TALK • My biases • My mistakes • My misperceptions This is a biased talk. I’m using my experiences and some data Biased by professional experience • 2003 – Extreme Programming - manager Loved it and lived it for 4 years at a 200 person company • 2007 – Scrum - manager Lived it for 6 years at a 5000 person company • 2013 – More scrum - director Lived it for 3 years at a 10,000 person company • 2017 – Technical evangelist at CloudBees We sell and support Jenkins to Enterprises Biased by personal experience • 2012 – My first commit to Jenkins git plugin (Java) • 2014 – Wrote many tests for git plugin and git client plugin • 2015 – Primary maintainer of git plugin and git client plugin • 2017 – Divided maintainer – I do legacy, Stephen does next gen Biased by beliefs • I believe in Test Driven Development
- 3. TEST-DRIVEN DEVELOPMENT • Red – write a small failing test • Green – write to pass the test • Refactor – simplify and improve Three simple steps A lifetime to learn to do it well I love tests. I write tests. I run tests. I fix tests. Photo CC BY-SA 4.0
- 5. WHAT IS JENKINS? Leading continuous integration server Open source, started in 2007 as Hudson Became Jenkins in 2011 150,000+ installations worldwide 1,300+ plugins Active community Strong integrations
- 6. WHAT IS GIT? Most popular version control system Linux kernel tracked in Git Microsoft Windows tracked in Git Core technology at GitHub, Bitbucket, VS Online, …
- 7. JENKINS GIT PLUGIN HISTORY Jenkins Git Plugin connects Jenkins to Git Historical Phases of Jenkins Git Plugin • Early adoption: 2007 - 2010 • Significant growth: 2010 – 2013 • Refactoring: 2013 - 2016 • Pipeline: 2017 - now
- 8. EARLY ADOPTION 2007 - 2010 Early adoption questions: • Is git relevant to Jenkins users? • Which git features matter most?
- 9. EARLY ADOPTION Unknowns • User value • Feature relevance • Performance measures Integration Hurdles • Few lines of code • Few components used • Few components depend on us
- 10. INSTALLS AND TESTS IN EARLY ADOPTION 0 20000 40000 60000 80000 100000 120000 140000 160000 Jan-08 Jan-10 Jan-12 Jan-14 Jan-16 Jan-18 Active Jenkins Git Plugin Installations 0 2000 4000 6000 8000 10000 12000 14000 Jan-08 Jan-10 Jan-12 Jan-14 Jan-16 Jan-18 Jenkins Git Plugin lines of code vs. test LOC LOC Test LOC • User needs are more important than tests • Discover user needs by delivering • Test interactively for fastest delivery
- 11. WIDESPREAD USE 2010 - 2013 Widespread use questions: • Easier to use? • Easier to diagnose? • Next features?
- 12. WIDESPREAD USE More confident • User value • Feature relevance • Key performance measure Integration Hurdles • More features • More lines of code • More components used • Few components depend on us
- 13. INSTALLS AND TESTS IN WIDESPREAD USE 0 20000 40000 60000 80000 100000 120000 140000 160000 Jan-08 Jan-10 Jan-12 Jan-14 Jan-16 Jan-18 Active Jenkins Git Plugin Installations 0 2000 4000 6000 8000 10000 12000 14000 Jan-08 Jan-10 Jan-12 Jan-14 Jan-16 Jan-18 Jenkins Git Plugin lines of code vs. test LOC LOC Test LOC • User needs are more important than tests • Discover user needs by delivering • Test interactively for fastest delivery • Feature and dependency growth is increasing the cost of testing
- 14. REFACTORING 2013 - 2016 Resolve implementation problems Git is winning the source control wars More regressions from refactoring Refactor: • add JGit • extract git client
- 15. REFACTORING Confident in • User value • Feature relevance • Key performance measures Integration Hurdles • More features • More lines of code • More components used • Many components depend on us
- 16. INSTALLS AND TESTS IN REFACTORING 0 20000 40000 60000 80000 100000 120000 140000 160000 Jan-08 Jan-10 Jan-12 Jan-14 Jan-16 Jan-18 Active Jenkins Git Plugin Installations 0 2000 4000 6000 8000 10000 12000 14000 Jan-08 Jan-10 Jan-12 Jan-14 Jan-16 Jan-18 Jenkins Git Plugin lines of code vs. test LOC LOC Test LOC • User needs are often failed by regressions • Discover user needs by delivering • Test automation for fastest delivery • Interactive testing still finds most bugs • Feature and dependency growth is increasing the cost of testing
- 17. PIPELINE 2017 - NOW Pipeline: • GitHub, Bitbucket optimizations • Simpler setup and maintenance • Higher performance
- 18. PIPELINE Significant new • User value • Features • Key performance improved Integration Hurdles • More features • More lines of code • More components used • Critical part of Jenkins
- 19. INSTALLS AND TESTS IN PIPELINE 0 20000 40000 60000 80000 100000 120000 140000 160000 Jan-08 Jan-10 Jan-12 Jan-14 Jan-16 Jan-18 Active Jenkins Git Plugin Installations 0 2000 4000 6000 8000 10000 12000 14000 Jan-08 Jan-10 Jan-12 Jan-14 Jan-16 Jan-18 Jenkins Git Plugin lines of code vs. test LOC LOC Test LOC • Regressions are less frequent • Discover user needs by delivering • Test automation prevents many regressions • Interactive testing still finds most bugs • Feature and dependency growth is increasing the cost of testing
- 20. WHEN CAN I SKIP TESTS? Tests are not as critical if: • Code will be discarded • Failures are easy to detect • Interactive testing is enough • Risk is low from broken code
- 21. WHEN SHOULD I ADD TESTS? Tests are more critical if: • Code will live a long time • Failures are hard to detect • Interactive testing is not enough • Risk is high from broken code
Notas del editor
- This is an experience report. It is based on my biases, my errors, and my misperception. I share my history not because it is especially relevant to test driven development, but because it may help you understand some of the biases that are implicit in the talk. Most of my time with test driven development was as a manager watching teams practice it. I was the manager of a team that transitioned from waterfall to Extreme Programming in 2003. That team of 10-15 people went “all in” with Extreme Programming. We tore down the cubicle walls. We installed big white boards. We encouraged continuous integration and rigorous test driven development in the Java desktop application we created and released every 3 months. We worked that way for 4 years. It was fun. After those 4 years, in 2007 we were acquired by a larger company. The product was moved to India and we were assigned to a project writing SharePoint extensions in C#. I spent a few years managing SharePoint based development in C# with the same developers who had spent 4 years writing test driven the Java based desktop application. After we discovered that SharePoint was a poor choice for the tasks we were trying to accomplish, I was reassigned to manage the build team for a large database application written in Java. We didn’t write tests ourselves, but we ran the tests created by others. That lasted for a few years and provided new experiences, including running builds for hundreds of developers. After those 6 years, I joined a different company as a director in 2013. The team was creating a Java based replacement for a rule processing engine. We spent 3+ years bringing that Java based rule processing engine into multiple releases. More new experiences, and eventually an assignment to manage a team of C programmers in their work on a message bus. In 2017, I joined CloudBees as a technical evangelist. I now spend my time showing people how to do cool things with Jenkins. I think test driven development is great. I’ve been in organizations where we wrote very few automated tests and I’ve been in organizations where we wrote many, many automated tests. I prefer more automated tests and I prefer tests that are written while the code is being written. We created our own python based continuous integration server in 2003, then switched to Hudson in 2007 when it became available. Hudson became Jenkins in 2011. I submitted tests to the Jenkins git plugin in 2014 when I became frustrated at the regressions I was detecting in new releases. In 2015 I became the primary maintainer of the plugin. In 2017, Stephen Connolly became a maintainer and added some really great multi-branch Pipeline features which he maintains. My history is only relevant to you if it helps you understand my biases. I think test driven development is great. I think everyone should do it almost all the time. However, others don’t always think that way. In particular, the original developers of the Jenkins git plugin don’t work that way. This talk uses history to look at those places where a lack of automated tests, and a lack of test driven development can be good, helpful, and workable.
- This is a period of fast learning and experimentation. There is less code to break and fewer dependencies
- I’m a test driven development proponent. I love test driven development. Yet here we see adoption and delight with code that is definitely not test driven. For the first 18 months of the existence of the plugin, there are no tests. None. All verification is done interactively before a release. How were they so successful? Because users are more important than tests. Interactive testing at this stage is much faster than writing tests.