The document discusses measuring the costs of requirements to improve project success rates. It recommends taking a balanced approach to measurement that aligns with organizational goals. Key measures include requirements defects, changes requiring rework, customer satisfaction, and cycle times. Tracking measures over time can help increase process maturity from initial to optimized levels. Consistency, benchmarks, and a SMART approach to measurement are emphasized.

1. The Dollar$ are in the Detail$:
Measuring the Cost of Requirements
Glenn R. Brûlé
Executive Director of Client Solutions, ESI International
gbrule@esi-intl.com
http://www.linkedin.com/in/toglennsnetwork
www.esi-intl.com

2. Agenda
Breaking the Bank SAFE Count on it…
building talent, driving results 2

3. Breaking the Bank

4. Consistently Challenged
100%
80%
60% Failed
Challenged
40%
Successful
20%
0%
1994 1996 1998
2000
2002
2004
2006
2009
Chaos Reports ~ 2009 – 75% of our Projects are NOT successful!
building talent, driving results

5. Writing checks we can’t cash?
Why do we have to measure our
costs?
– Maintain/Grow market share
Are the products meeting customer
expectations?
– Quality?
– Costs of product to consumers vs.
the cost to build the products
– Can we increase the profitability?
– Are we building the right products?
– Are we efficient at building the right
products?
Right people
Right processes
Right tools?
building talent, driving results

6. A Balanced Portfolio
Organizational
Goals and
Objectives
Customer Value Analysis
Goods &
Portfolio Management
Services
Core Competencies
Business
Policies & Rules
Process
Tasks &
Activities
building talent, driving results 6

7. So What’s the Point?
Increased quantifiable probability of
Organizational Goals
providing the desired deliverables and and Objectives
expectations to meet overall organizational
goals and objectives through a clear Goods & Services
understanding of our core
competencies Business Policies &
Rules
Improved goods and services by virtue of Process
standardized methodology/approach and
more complete overall work, less iterations
by understanding the output from our
customer value analysis activities Tasks &
Activities
Increased flexible structure checks and
balances that ensure we are compliant with
governance, internally or externally
building talent, driving results 7

8. So What’s the Point?
Better requirements and more closely
aligned and prioritized to work being Organizational
Goals and
performed Objectives
Goods & Services
Stronger, well-aligned requirements to
alleviate rework down the line Business Policies
& Rules
Having the right resources with the right Process
competencies, completing the right tasks
that they are capable of is critical to ensure
traceability of deliverables to overall goals
Tasks &
and objectives. Activities
Finally, any one of these areas are areas
for quantifiable consideration
building talent, driving results 8

9. Nickels and Dimes…
Organizational goals and objectives Processes
– Increase market share by 15% from 7% – Increase efficiencies within our own process to
– Increase revenue by 10% decrease cycle time or number of iterations or the
– Decrease the time it takes to respond to a customer costs of each iteration within an SDLC
inquiry from 10 minutes to 2 minutes Tasks
Goods and services – Are we focused on too many tasks at the
– Increase customer sat by 15 % same time
– Decrease defects in products by 10% from 15% – Are the tasks prioritized
– Are the tasks related to a real business value
Business Policies and Rules
– Are the right people performing the right
– Sampling of compliance with rules
tasks?
– Are they prohibitive or restrictive, or to ad-hoc –
could we improve or optimize performance here.
building talent, driving results 9

10. Requirements Management and
Development Assurance Model
© ESI International, Inc. 2009 building talent, driving results 10

11. Top Down Metrics?
GO
Enterprise
Analysis
Enterprise Architecture
SOA
Change Management
Portfolio Program
Governance
Management Management
ITIL, RUP, AGILE, LEAN, Six Sigma, CMMI
Process Project
Business Analysis
Management Management
BUSINESS IT
© ESI International, Inc. 2009 building talent, driving results 11

12. SAFE

13. Organizational Starting Point
CMMI Model
1. Initial 2. Repeatable 3. Defined 4. Managed 5. Optimizing
Requirements Standard practices Process is Measurements Continue process
Management and and templates formalized collected and improvement
Development (predictable) assessed
practices performed
inconsistently
building talent, driving results 13

14. Measures for Maturity
CMMI
Levels 4-5
CMMI
Levels 2-3
CMMI
Level 1
building talent, driving results 14

15. Measures for Maturity
Phase III
Phase II Mature
Continuous
Maturing Improvement
Capability
Phase I:
Rollout
Stage
building talent, driving results 15

16. An inventory for “measurable”
CMMI Level Framework Environment Resources Business Alignment
Standards and Organization and Competency and Career
Level II Enterprise Analysis
Methodologies Structure Development
Level III Metrics Governance Training and Development Portfolio Management
Level IV Tools Assessment Coaching and Mentoring Program Management
Customer Relationships
Level V Knowledge Management
building talent, driving results 16

17. Rules of the Road
Remember…this is a journey…..
1. Measure to your level of maturity.
2. It is best when a baseline can be obtained.
• If not, then just start and use initial findings that as your baseline.
• Look at 5-10 projects of a representative sample
• Change requests, help desk tickets, iterations, approval processes,
3. Know what your measures mean; i.e. what value the measurement provides to the
organization.
4. Consistency is important. - stick with what you can put into place
• Even if you learn that the measure needs to be altered; it might be prudent to continue to measure
consistently for a period of time vs. changing it without getting good baseline information.
5. Benchmark externally whenever possible.
• What are the industry standards to compete on these levels – this is a true – thinking outside of the
box type of concept!
building talent, driving results 17

18. Count on it…

19. A Penny for your thoughts…
What are you tracking and measuring? How often?
Who sets the standards?
How realistic are they?
How clear are they?
How quickly do you need to report?
How often?
What variance is acceptable? (+/-)
At what level of variance must you take action?
What rewards and “penalties” will enforced?
© ESI International, 2006.
building talent, driving results 19

20. Determine an approach that is
SMART!
Specific
Measurable
Agreed Upon
Realistic
Timely
© ESI International, 2006.
building talent, driving results 20

21. Measures for Each Level
Phase I: Rollout Stage Level of Organizational Change, i.e. Number of People Trained, Number of
Organizations using RMD processes
Number of Requirements Failing an ‘Acid Test’ or other test to determine a ‘good’
requirement.
Phase II / Maturing Number of Requirement Defects (Poor or Missing Requirements)
Capability: Number of Requirements Changes resulting in Project ‘Rework’
Customer Satisfaction of Deliverable
Mature / Continuous % Requirements/Defects compared to other Projects
Process % Requirements/Change Controls compared to other Projects
Improvement: Overall Cycle Time (value of good requirements to satisfactory and speedy
completion of deliverable)
Defects in post-production environment
Prediction of future activities accurately
Six Sigma – optimization tool
Phase III
Mature Continuous
Phase II Improvement
Maturing
Capability
Phase I:
Rollout Stage
building talent, driving results 21

22. Have a “controlled” approach
All methodologies/approaches can leverage
metrics
Have a Process of comparing actual performance
with planned performance, analyzing variances,
evaluating alternatives, and taking corrective
action as needed
Concerned with three elements of all
aspect of the overall development of
deliverables. From GO to tasks and
activities.
– Performance
– Cost
– Time
Ward, Project Management Terms, p. 48
Meredith & Mantel, Project Management, p. 508
© ESI International, 2006.
building talent, driving results 22

23. Some Finer Points of
Measurement
Consider amount of time/effort Business
Architecture
spent in each phase of your SDLC
Verification &
Elicitation
Validation
Consider the number of iterations
of each phase or overall iterations
Consider the number of change
requests over the course of your Development Analysis
SDLC
Design
Consider the amount of “rework in
each phase
building talent, driving results 23

24. Cost of Different Types Requirements
Total Cost of
Requirements + Total
Cost of Effort = Total
Cost of Solution
building talent, driving results 24

25. Summary:
Consider the depth and breadth of
“measurables” from the “top down”
Match your approach to
measurements according to the level
of overall requirements maturity
Be SMART about your approach
This is a journey…metrics provide the
roadmap to know when you’ve
achieved your goal!
building talent, driving results 25

26. Questions?
Glenn R. Brûlé
Executive Director of Client Solutions, ESI International
gbrule@esi-intl.com
http://www.linkedin.com/in/toglennsnetwork
www.esi-intl.com