2. ABOUT US 16 June 2011 Agile Mëtteg – Continuous Integration 2
3. PROFILE Created in 2004 Independent Software Development Company 16 June 2011 Agile Mëtteg – Continuous Integration 3
4. FIGURES 16 June 2011 Agile Mëtteg – Continuous Integration 4 2,5M€ en 2010 32 in 2011
5. MISSION Design, Develop and Customize “Software as Business & Operational Enabler” Fast & flexible solutions business value oriented Help IT and all Business & Operational Organizations to adopt the culture of “Software as Business & Operational Enabler” Simple & pragmatic methods making effective the collaboration of the actors of a project Easy and powerful tools for follow-up of relevant KPI 16 June 2011 Agile Mëtteg – Continuous Integration 5
6. OUR SERVICES MgtTeamServices 4 Applications enabling productivity Bespoke application Mobile application Based on package Consulting services Coaching & Support Training Resource delegation 1 Software Development OpsTeamServices Dev Team Services Agility Agility 2 3 1 Agility 16 June 2011 Agile Mëtteg – Continuous Integration 6 2 3 4
7. OUR MEANS 16 June 2011 Agile Mëtteg – Continuous Integration 7 80% > 4 years 56% > 8 years 31% > 12 years Agility Authorized Training Center in Luxembourg
13. CONTINUOUS INTEGRATION Continuous Integration in a few questions Why do I need it ? What is it ? What does it require ? How does it relate to the Agile principles ? 16 June 2011 Agile Mëtteg – Continuous Integration 13
14. SOFTWARE DEVELOPMENT: INDUSTRY? PROCESSES? What is Software Development? an industry mining, farming, construction, manufacturing, …etc. with clearly identified and well-defined processes, i.e. easily reproducible Really? No failed project? A lot of… 16 June 2011 Agile Mëtteg – Continuous Integration 14
15. SOFTWARE DEVELOPMENT: A LOT OF FAILED PROJECTS 16 June 2011 Agile Mëtteg – Continuous Integration 15
16. SOFTWARE DEVELOPMENT: A LOT OF FAILED PROJECTS 16 June 2011 Agile Mëtteg – Continuous Integration 16
17. « Houston, we’ve got a problem! » 16 June 2011 Agile Mëtteg – Continuous Integration 17
18. AGILE MANIFESTO vs SOFTWARE INDUSTRIALIZATION ? Agile Manifesto Individuals and interactions overprocesses and tools Working softwareovercomprehensive documentation Customer collaboration overcontract negotiation Responding to change overfollowing a plan 16 June 2011 Agile Mëtteg – Continuous Integration 18
19. SOFTWARE CRAFTSMANSHIP vs SOFTWARE INDUSTRIALIZATION? Manifesto for Software Craftsmanship Not onlyworking software, but alsowell-crafted software Not onlyresponding to change, but alsosteadily adding value Not onlyindividuals and interactions, but alsoa community of professionals Not onlycustomer collaboration, but alsoproductive partnerships 16 June 2011 Agile Mëtteg – Continuous Integration 19
20. SOFTWARE DEVELOPMENT: AN ART? Agile Manifesto and Manifesto for Software Craftsmanship were created by veterans of the software industry However, when summed up, one can conclude software development is more an art than an industrial process So, let’s compare with music… 16 June 2011 Agile Mëtteg – Continuous Integration 20
23. SOFTWARE DEVELOPMENT IS AN ART So, Software Development is an art! But building nearly anything is also an art Don’t you think? And, more importantly, Art is not without rules and best practices 16 June 2011 Agile Mëtteg – Continuous Integration 23
24. RULES AND BEST PRACTICES What’s the worst? Not following therules usually leads to a rather direct and abrupt failure project fails integration testing project is refused by infrastructure project fails user acceptance testing … 16 June 2011 Agile Mëtteg – Continuous Integration 24
25. RULES AND BEST PRACTICES What’s the worst? Not following the best practices augments, sometimes dramatically, the risks of failure reduces overall quality increases the time to market / time to deliver has typically a pervasive effect: can be ignored or remains unknown until it becomes really critical increases the maintenance and ownership costs 16 June 2011 Agile Mëtteg – Continuous Integration 25
26. RULES AND BEST PRACTICES What’s the worst? Both are terrible: sources of failure But one is harder to detect than the other Necessity to put in place and to define the structures and infrastructures required to check the quality at every level Because “the earlier, the best” (and less expensive) 16 June 2011 Agile Mëtteg – Continuous Integration 26
28. WHAT IS SOFTWARE QUALITY? “Quality is value to some person” (Gerald Weinberg, “Quality Software Management”) i.e. quality is inherently subjective; people experience the quality of the same software very differently this applies mainly to the quality of a software product, as perceived from an external view; and it can also comprise the quality of its running environment(s) but the quality of the source code can also have an impact on the efforts needed for having a software product that fulfills the requirements, the intrinsicquality Steve McConnell defines external and internal quality characteristics (‘”Code Complete”) 16 June 2011 Agile Mëtteg – Continuous Integration 28
29. IMPLICIT FACTORS FOR SOFTWARE QUALITY The software projects typically follow rather detailed requirement plans Though a set of characteristics often goes unmentioned These are the implied requirements that are expected of all professionally developed software 16 June 2011 Agile Mëtteg – Continuous Integration 29
30. IMPLICIT FACTORS FOR SOFTWARE QUALITY Conformance to implied requirements: Understandability Completeness Conciseness Portability Maintainability Testability Usability Reliability Efficiency Security 16 June 2011 Agile Mëtteg – Continuous Integration 30
31. GUARANTEEING CODE QUALITY How can we guarantee the level of quality of software during the « coding » phases? The same ways as in the industry Factory inspections: “static code analysis” in IT Incremental improvements: “release soon, release often” in IT Tests, tests, tests, tests, tests, tests, … 16 June 2011 Agile Mëtteg – Continuous Integration 31
32. GUARANTEEING CODE QUALITY Focus on Unit Tests and Test Driven Development (TDD) Unit Tests The first tests to write Written by the developers before the code Write the code afterward to make the tests succeed Must run in isolation, without any infrastructure Must be fast to execute How many unit tests per method ? One test per degree of “cyclomatic complexity” 16 June 2011 Agile Mëtteg – Continuous Integration 32
33. GUARANTEEING CODE QUALITY Cyclomatic complexity of a method? the number of linearly independent paths in the method If .. and .. then If .. or .. then If .. then If .. then .. else Do .. While While .. Do Switch 16 June 2011 Agile Mëtteg – Continuous Integration 33
34. GUARANTEEING CODE QUALITY Necessity to put in place the supporting Structures (practices): Unit testing Integration testing Performance testing Regression testing Acceptance testing Infrastructures (tools): Source Code management Issue tracking Build tools Continuous Integration server Quality reporting tools Agility Build Industrialization Build Industrialization Platform Agility 16 June 2011 Agile Mëtteg – Continuous Integration 34
35. GUARANTEEING CODE QUALITY How to put in place the supporting Structures (practices): Define and prioritize the quality requirements Refine and adapt the relevant quality criteria to the objectives Communicate rules and best practices: sharing, training and mentoring Verify the correct usage and level of adoption Review the results and improve the processes Infrastructures (tools): Select the tools adapted to the defined quality requirements Set them up based on the selected criteria (define measurements) Communicate on their usage: sharing, training and mentoring Generate quality reports and metrics Analyze the conformance of the results and adapt the tools Agility Agility 16 June 2011 Agile Mëtteg – Continuous Integration 35
36. Ci Rules and pre-requisites 16 June 2011 Agile Mëtteg – Continuous Integration 36
37. CI RULES AND PRE-REQUISITES (1/3) maintain a code repository automate the build the build must be self-testing regular code sharing i.e. frequent commits 16 June 2011 Agile Mëtteg – Continuous Integration 37
38. CI RULES AND PRE-REQUISITES (2/3) self-contained modifications i.e. commits don’t break the build process the build must be fast integration tests (for the least) in a clone of the production environment 16 June 2011 Agile Mëtteg – Continuous Integration 38
39. CI RULES AND PRE-REQUISITES (3/3) latest deliverables easily available to anyone who needs them easy access to the results of the tests automated deployments to a live test server continous deployment to production is the ideal achievement 16 June 2011 Agile Mëtteg – Continuous Integration 39
43. SOURCE CODE MANAGEMENT What are SCM tools ? Tools that allow sharing and versioning files … but really not their primary interest … … shared folders can do that too ! Tools to track and document the changes in the code, in such a way the developers can work in a task or issue oriented mode 16 June 2011 Agile Mëtteg – Continuous Integration 43
47. Feature branches for large changes“branch by abstraction” pattern works well with continuous integration (for non-distributed SCM’s like CVS, SVN, TFS…) 16 June 2011 Agile Mëtteg – Continuous Integration 44
91. CONCLUSIONS Guaranteeing code quality is an intensive task In time In money Continuous Integration pays back and offers a lot Has a high ROI Cost of ownership reduces over time It can be applied incrementally Agility should be the driving backbone for its adoption 16 June 2011 Agile Mëtteg – Continuous Integration 75
92. RESOURCES Gartner’s study ID “G00151721” http://condor.depaul.edu/~dmumaugh/readings/handouts/SE477/Gartner%20Reports/from_the_cio_trenches_why_so_151721.pdf Standish Group’s Chaos Reporthttp://www.standishgroup.com/services.php “Quality Software Management : Systems Thinking”Gerald Weinberg, 1991, ISBN 978-0932633729 “Code Complete”, Microsoft Programming SeriesSteve McConnell, 1993, ISBN 978-1556154843 16 June 2011 Agile Mëtteg – Continuous Integration 76
93. RESOURCES CVS http://www.nongnu.org/cvs/ Subversion (SVN) http://subversion.apache.org/ Maven http://maven.apache.org/ Ant http://ant.apache.org/ Nanthttp://nant.sourceforge.net/ Ivy http://ant.apache.org/ivy/ EasyAnthttp://www.easyant.org/ Gradlehttp://www.gradle.org/ Buildrhttp://buildr.apache.org/ Gant http://gant.codehaus.org/ Jenkins CI http://jenkins-ci.org/ Hudson CI http://hudson-ci.org/ Sonar http://www.sonarsource.org/ Microsoft Team Foundation Server http://www.microsoft.com/visualstudio/en-us/products/2010-editions/team-foundation-server/ 16 June 2011 Agile Mëtteg – Continuous Integration 77
94. RESOURCES Agile Partner: www.agilepartner.net &Blog: http://blog.agilepartner.net Trainings http://www.agilepartner.net/formations/coup-de-projecteur-sur/?lang=fr Agile Interest Group LU: www.aiglu.org Agile Tour Luxembourg8 November 2011 16 June 2011 Agile Mëtteg – Continuous Integration 78
95. CONTACTS Thank You 16 June 2011 Agile Mëtteg – Continuous Integration 79
96. DEBRIEFING Questions ? 5 fingers vote 1 = useless “I gained nothing. I completely lost my time!” 2 = useful “It wasn’t worth all the time spent on it. I lost most of my time” 3 = average “I gained enough to justify the time spent on” 4 = above average “Good value, I gained more than the time spent” 5 = excellent “Really useful session, time well spent” 16 June 2011 Agile Mëtteg – Continuous Integration 80
98. BRANCH BY ABSTRACTION “Branch by abstraction” all the changes are done in the same place, same location in the SCM no need to merge (hazardously) a lot of code from the feature branch history of the changes stays easy to follow 16 June 2011 Agile Mëtteg – Continuous Integration 82
99. BRANCH BY ABSTRACTION “Branch by abstraction” Cookbook: Introduce an abstraction over the core bits of the big thing you are going to change Update all the bits of code that were formerly using the thing directly to use it via the new abstraction Make a second implementation of the abstraction, with unit tests that specifically test its core functionality Update all the code to use the new implementation Deprecate the first implementation Delete the first implementation (there is no need to go back) Remove the abstraction (only if it is inelegant, not often the case) 16 June 2011 Agile Mëtteg – Continuous Integration 83
Notas del editor
That’s the answerswe are trying to give in thispresentation
Really? If itis the case, whyfailingprojects?
At firstsight, the Agile Manifestoseems to beagainst Software Industrialization and processes.Emphasize must begiven on the « over »: itdoesn’tmean « against ». It rathermeans « in favor of …»
The Manifesto for Software Craftmanshipalsoseems to beagainst Software Industrialization and processesEmphasize must begiven on the « not only »: itdoesn’tmean « not ».
Jazzmen seem to be totally freeHowever they tend to follow a lot of:Rules:Arrangements and compositionsRhythms, pitches, key signatures, transpositionsImprovisation considerations (context, balance, direction, …) Best practices: Themes: “Blues Form” and “Song Form”Scales: Blues, Bebop, Pentatonic, Symmetric, …Introduction, accompaniment, styles of composition
Developers seem to be totally freeHowever they tend to follow a lot of:Rules:Standards (« industry » or « de facto »)Specifications (IETF, W3C, OASIS, ISO …)Company levelBest practices: Design patterns and principles (yagni, kiss, dry, …)Clean code, coding conventionsLow coupling, high cohesion …Yagni: You Ain’tGonna Need itKiss: Keep It Straight SimpleDry: Don’t Repeat Yourself
Impliedrequirements (mostly):not oftenstatedexplicitly in the productrequirements
Static code analysis: analysis performed on the code without it being executed
Static code analysis: analysis performed on the code without it being executed
Static code analysis: analysis performed on the code without it being executed
Notice eachstep in the « Structures » isalsorelated to the correspondentstep in the « Infrastructures »
Source Code Management: CVS, PVCS, SourceSafe, Perforce, SVN, Git, Mercurial, Team Foundation Server uses a database (Microsoft SQL Server)Issue Tracker: JIRA,BugZilla, Trac, Team Foundation Server « Work Item Tracking » (integrated in Visual Studio)AutomatedBuild: Ant, Maven, Apache Ivy, Gradle, Team Foundation Server « AutomatedBuild »ContinuousIntegration Server: Hudson, Jenkins, CruiseControl, AtlassianBamboo, JetBrainsTeamCity, Team Foundation Server « Build Agent »QualityAnalysis and Reporting: Maven reports, Hudson plugins, Sonar, Team Foundation Server « Reporting »
Source Code Management: CVS, PVCS, SourceSafe, Perforce, SVN, Git, Mercurial, Team Foundation Server uses a database (Microsoft SQL Server)
TFS: Team Foundation Server (Microsoft)
Standardized: pick a format that all the developers on your project will use and, if possible, apply it for all the projects in your company.
The SCM doesn’t enforces the comments, but IDE tools can help:- In Java under Eclipse, the free Mylyn plugin does that –and more- automatically (shown here and in the next slides)- Similar mechanism in IntelliJ IDEAFocus on:the “Task List” view: shows the tasks from the issue tracking systemthe central view: shows the details of one task (also providing interaction with the issue tracking system from the IDE)The “Synchronize” view: shows only the files modified for the selected issue (in order to prepare a commit)The “Package explorer” view [as cherry on the cake]: shows only the modifications linked to the active task (instead of all the files in the project)
The SCM doesn’t enforces the comments, but IDE tools can help:- In Java under Eclipse, the free Mylyn plugin does that –and more- automatically (shown here and in the next slides)- Similar mechanism in IntelliJ IDEAFocus on:the “Task List” view: shows the tasks from the issue tracking systemthe central view: shows the details of one task (also providing interaction with the issue tracking system from the IDE)The “Synchronize” view: shows only the files modified for the selected issue (in order to prepare a commit)The “Package explorer” view [as cherry on the cake]: shows only the modifications linked to the active task (instead of all the files in the project)
Double clicking a task in the “Task List” view opens a central view that shows the details of the task, also providing interaction with the issue tracking system directly from the IDE
“Round” button to activate/deactivate a task. This button is also present next to each task in the “Task List” view (hard to see on this slide, but the active task is shown in bold)Only one task active at a given time
The “Package explorer” view: shows only the modifications linked to the active task (instead of all the files in the project)Filter can be on/off (also with a keyboard shortcut to quickly open a file not yet opened so far in the active task)In Bold, files that have been changed more frequently.
The “Synchronize” view: group the changed files by task (in order to prepare a commit in a task based mode)
The SCM doesn’t enforces the comments, but IDE tools can help:- In Java under Eclipse, the free Mylyn plugin does that –and more- automatically (shown here and in the next slides)- Similar mechanism in IntelliJ IDEAFocus on:the “Task List” view: shows the tasks from the issue tracking systemthe central view: shows the details of one task (also providing interaction with the issue tracking system from the IDE)The “Synchronize” view: shows only the files modified for the selected issue (in order to prepare a commit)The “Package explorer” view [as cherry on the cake]: shows only the modifications linked to the active task (instead of all the files in the project)
The SCM doesn’t enforces the comments, but IDE tools can help:- In Java under Eclipse, the free Mylyn plugin does that –and more- automatically (shown here and in the next slides)- Similar mechanism in IntelliJ IDEAFocus on:the “Task List” view: shows the tasks from the issue tracking systemthe central view: shows the details of one task (also providing interaction with the issue tracking system from the IDE)The “Synchronize” view: shows only the files modified for the selected issue (in order to prepare a commit)The “Package explorer” view [as cherry on the cake]: shows only the modifications linked to the active task (instead of all the files in the project)
Issue Tracker: JIRA,BugZilla, Trac, Team Foundation Server « Work Item Tracking » (integrated in Visual Studio)
AutomatedBuild: Ant, Maven, Apache Ivy, Gradle, Team Foundation Server AutomatedBuild
AutomatedBuild: Ant, Maven, Apache Ivy, Gradle, Buildr (Ruby), A-A-P (Python), Cake (Python), SBT (Scala) Team Foundation Server AutomatedBuild, NANT (.Net ANT)
ContinuousIntegration Server: Hudson, Jenkins, CruiseControl, AtlassianBamboo, JetBrainsTeamCity, Team Foundation Server « Build Agent »
QualityAnalysis and Reporting: Maven reports, Hudson/Jenkins plugins, Sonar, Team Foundation Server « Reporting »
Sonar IDE: Plugin to integrate into IDE (here with Eclipse)
In time: employee’s involvement (dev teams, management and administrators for the infrastructures)In money: hardware and serversIntensive task = should not be underestimatedROI: Return On Investment
Standardized: pick a format that all the developers on your project will use and, if possible, apply it for all the projects in your company.
Standardized: pick a format that all the developers on your project will use and, if possible, apply it for all the projects in your company.