Advocates of agile development claim that agile software projects succeed more often than plan-driven projects. Unfortunately, attempts to validate this claim statistically are problematic, because "success" is not defined consistently across studies. This paper addresses the question through a mathematical analysis of these projects. We model agile and plan-driven software projects with identical requirements, and show how they are affected by the same set of unanticipated problems. We find that that the agile project provides clear benefits for return-on-investment and risk reduction, compared to the plan-driven project, when uncertainty is high. When uncertainty is low, plan-driven projects are more cost-effective.

1. The Price of Uncertainty: Why Agile Projects Succeed when Others Fail cPrime, Inc. 4100 E. Third Ave, Suite 205 Foster City, CA 94404 650-931-1651 www.cprime.com

3. Education and certifications

4. Certified ScrumMaster and Scrum Professional

5. Project Management Professional

6. Doctorate in Physics, Princeton University

8. 15 Years Program / Project Management Experience

9. Education and certifications

10. Certified ScrumMaster

11. Does not have a Doctorate in Physics from Princeton University2

12. Today’s Agenda Introduction Case Study: Well Planned Failure Doing the Math Lessons Learned Case Study: How It Might Have Succeeded Conclusion 3

13. Outline Introduction Case Study: Well Planned Failure Doing the Math Lessons Learned Case Study: How It Might Have Succeeded Conclusion 4

14. Introduction Everyone talks about uncertainty,… … but no one does anything about it. That’s because you can’t eliminate it You can only reduce some of it,… … and cope with the rest Many claim that agile projects work better than waterfall projects when uncertainty is high Is this true? We will see… 5

15. Outline Introduction Case Study: Well Planned Failure Doing the Math Lessons Learned Case Study: How It Might Have Succeeded Conclusion 6

16. Case Study: Well Planned Failure - Context The year was 1995…. 6 Months have gone by since NetMarket performed the first secure credit card transaction on the web – sold a C.D. for $12.48 The top billboard hit was Coolio’sGangsta Paradise The #2 billboard hit was…… Waterfalls by TLC After 3 Months in the Bay Area 7

18. Governance Model included PMO, & 5 Primary teams: (Architecture, Development, Business Analysis / Process, Change Management / Training Testing)

19. 300+ resources, 1 location

20. 1.5 year Timeline

21. Complete Executive Buy-InCompany: National TeleCom Provider Program Objective : Develop a Custom, GUI based Provisioning and Service Application Methodology: Waterfall 8

23. Sourcing

25. Experienced App Engineers

28. Too SLOW

29. Too many manual interventions

30. Round the clock testing

31. Professional Trainers9

32. Case Study: Well Planned Failure -Gap Analysis What Was Delivered What They Wanted OOPS! 10

34. Extend Resourcing

36. Tired Experienced App Engineers

38. Too Much Money Spent

39. Dissention on Actual Value

40. Round the clock testing

41. Professional Trainers11

42. Outline Introduction Case Study: Well Planned Failure Doing the Math Lessons Learned Case Study: How It Might Have Succeeded Conclusion 12

43. The Theories Agile (Scrum) processes succeed where Waterfall processes fail This is a big claim It sounds arrogant Promoters are often pushy and ideological A carefully-planned waterfall process is the most efficient and successful way to run a project This is a big claim It sounds arrogant Promoters are often pushy and ideological 13

44. The Statistics Standish Report © 2009 Surveyed 50,000 projects Found 29% succeeded 53% late, over budget, lacked functionality or quality 18% failed Dr. Dobb’s Journal (2007—2008), by Scott Ambler 586 people responded to the survey Found 63% of waterfall projects succeed 72% of agile projects succeed Agile looks better But waterfall isn’t a disaster Can we trust the statistics? What should we believe? 14

45. Why we don’t Trust Statistics Statistics can lie There are lies, damn lies, and statistics Select your truth, find statistics to back it up “Who are you going to believe? Me, or your lying eyes?” Math does not lie Results are provable Can assumptions can be wrong? Yes Take nothing on faith So put theories to the test! 15

46. Doing the Math GEDANKEN EXPERIMENT 16

47. Put the Theories to the Test! Design a Gedanken Experiment to test the success of Scrum and Waterfall projects Build mathematical models for each Conduct the experiment Learn from the results 17

48. Experiment: Build a Data Warehouse / BI System A company wants to provide a reporting (Business Intelligence) capability for customers The initial concept calls for six reports Basic tasks Build production environment Configure servers with database, reporting software Create DB tables in various environments Develop ETL processes to transfer data to reporting DB Write reports 18

49. System Architecture OLTP (Online Transaction Processing) database Stores data from company’s business applications Replicated source DB Contains copies of OLTP data needed for reports Staging DB ETL (Extract-Transform-Load) process transforms source data into this DB, whose design is appropriate for report generation Report DB Replication of staging DB, used only by the reporting software Report server Reporting application. Generates reports created by developers. 19

50. Build the System in Two Ways Two types of project One big “Waterfall” project An “Agile” project with six iterations Only difference is scheduling! One big project versus six iterations Other agile practices are not considered Both are subject to same constraints Requirements have been set Team sizes are the same Funding is available for one year Both are subject to the same uncertainties See how they compare 20

51. What Goes Wrong Estimates are low All planned work takes 25% more time than expected Issue with source DB Source table used by Report #2 has 80 million records. Special processing for table adds 3 weeks to schedule. Upgrade Reporting vendor upgrades app 10 months into the project. Upgrade required to fix critical bugs, adds 3 weeks to schedule. Duplicate Data Several source tables for Report #3 have duplicate data. Handling this problem adds 3 weeks to the schedule. Production deployment is harder than expected. Problems add 3 weeks to the schedule. 21

52. Waterfall Project: The Planned Schedule Predict the project will complete in about 9 months Have room for three-month schedule buffer 22

53. Agile Project: The Planned Schedule Predict project will complete in 13 months Pessimistic: Assumes 20% more effort per report than for Waterfall Project Report #1: Takes 3 months, including initial server setup Reports 2—5: Take 2 months each Report #6: Won’t finish in Year 1 23

54. Comparison: Plans for Waterfall and Agile Projects Waterfall Project takes 9 months, Agile Project takes 13 Agile Project has more overhead Waterfall Project is more efficient Waterfall Project delivers all functionality in funding period Agile Project runs out of money before completion Requires de-scoping (remove Report #6) Waterfall Project is clear winner 24

55. Waterfall Project: The Actual Schedule Add effects of uncertainty to 9-month schedule Add 25% across the board: Expand to 11 months Add four 3-week delays: Expand to 14 months The money ran out at 12 months The project was cancelled The project failed It delivered nothing It produced no revenues The entire investment was wasted Money Spent + No Results = Lay off project team? 25

57. Add 25% to all tasks

58. Add 3-week slips to first four reports

59. Total schedule expanded to 19 months

60. Three reports were delivered within Year 126

61. Schedule Comparison: Waterfall to Agile Project Uncertainty impacted both projects Schedules lengthened 14 months for the Waterfall Project (+44%) 19 months for the Agile Project (+36%) Neither project delivered requested scope in Year 1 27

62. Value Comparison: Waterfall to Agile Project Within the budgeted one-year period Agile Project delivered three working reports Waterfall Project delivered no reports By end of Year 1 Agile Project brought in revenues Waterfall Project brought in nothing Implications for Staff Retention / Growth Agile project ROI encourages Waterfall project’s zero ROI discourages Implications for Year 2 Agile Project’s first-year revenues encourages extension Waterfall Project stays canceled 28

63. Outline Introduction Case Study: Well Planned Failure Doing the Math Lessons Learned Case Study: How It Might Have Succeeded Conclusion 29

64. Lessons Learned We can learn a lot from this gedanken experiment Lessons revolve around how schedule relates to Uncertainty Risk Return on investment 30

65. Uncertainty Affects Return on Investment When uncertainty is low, ROI is a calculation Project takes X months, costs $Y, will yield $Z revenue Estimate Net Present Value, Internal Rate of Return, etc. When uncertainty is high, ROI is a gamble Projects might not finish Oops! It was late by factor of 3, then company went out of business Funding might disappear Business priorities changed. We’ll do rear-view mirrors, get out of tire business. Customer interests can change Last year’s best-selling Pet Rock is this year’s gravel High uncertainty = high risk of wasting entire investment 31

66. Manage Uncertainty by Reducing Risk… Think of this visit to the doctor Patient: “Doc, my back hurts when I lift heavy weights. What should I do?” Doctor: “Lift smaller weights.” Don’t make a few big investments over long periods and expect big returns There may be no returns You may lose all of your investment 32

67. …and Speeding up Delivery of Value Make small investments for short periods to get small returns Risk is smaller with small investments Losses are less painful, when they occur Uncertainty is reduced due to shorter time periods Less time and less scope for problems Flexibility is greater More opportunities to change direction Large projected ROI is worthless if project never completes Better to deliver some value, soon, than risk large value, never Deliver increments of value as soon as possible Early ROI in pieces is better than big ROI at end Efficiency, later is seldom as important as value, soon 33

68. Summary of Lessons Learned In high-uncertainty projects Risk of failure is high Schedules can become meaningless All-or-nothing plans invite disaster Better to plan for the schedule to be wrong Tailor strategy to perform even when schedule is toast Change your plan to deliver value Do not estimate schedule, plan to deliver all value at end Ask, “How can I deliver the most value in… The next [month | three months | six months] Deliver planned scope in useful increments, ASAP Value sooner is better than value later Some value this year is better than a cancelled project 34

69. Outline Introduction Case Study: Well Planned Failure Doing the Math Lessons Learned Case Study: How It Might Have Succeeded Conclusion 35

70. Case Study: How It Might Have Succeeded Divide project into short iterations Deliver value with each iteration Work closely with customer to validate deliverables Adjust course based on feedback Worst case: Cancel project early Before wasting complete investment 36

71. Outline Introduction Case Study: Well Planned Failure Doing the Math Lessons Learned Case Study: How It Might Have Succeeded Conclusion 37

72. Conclusion High-uncertainty projects drive “agile process frameworks” (such as Scrum) Long-term, fixed-scope projects break when uncertainty is high Small iterations bring big benefits Risk reduction Improved ROI through early delivery 38

73. Crossing the Gap to an Agile World The transition from Waterfall to Scrum can be difficult No part is easy “Easiest” part is changing how development work is done Hardest parts include Changing customer, stakeholder expectations about schedule commitments Changing business process around the development work Suggestions to ease the transition Pick “low-hanging fruit” first Plan more short projects, instead of long projects Get training and mentoring for migration to Scrum 39

74. Discussion One “Candy Point” per speaker (while they last) War Stories! What projects failed? Why? Success Stories! What projects succeeded? Why? Bonus questions (2 Candy Point for first right answer): Who was Albert Einstein? What does “gedanken” mean? What is the key difference between the Special and General Theories of Relativity? cPrime, Inc. www.cPrime.com 650-931-1650 Educating. Consulting. Leading. 40