Why and how measure the performance of business analysts

1. MEASURING THE
PERFORMANCE OF BUSINESS
ANALYSTS
Adriana Beal
Beal Projects - http://bealprojects.com

2. Adapted from the original presentation made at
the WI BADD 2010, to work without narration.

3. Why measure the performance of
business analysts?
bealprojects.com
3

4. Traditional view of individual
performance measurement
Source: When the uncountable counts: An alternative to monitoring employee performance
by K. W. Platts, M. Sobotka Jul 15, 2010. Prod. #: BH392-PDF-ENG
bealprojects.com
4

5. Common complaints
 tunnel employee vision
 reduce urge to experiment
 cause sub-optimization
 lead to gaming, creative accounting, and fraud
bealprojects.com
5

6. Alternative viewpoint
Study developed in one of the largest suppliers of
electricity and natural gas in Germany:
“A formalized individual performance
measurement system may not be required to
achieve high performance standards.”
(K. W. Platts, M. Sobotka - When the uncountable counts:
An alternative to monitoring employee performance)
bealprojects.com
6

7. Other means to achieve high
performance
 Inspiring leadership
 Positive working environment
Encouragement to experiment and suggest new
solutions for business problems.
(K. W. Platts, M. Sobotka - When the uncountable counts:
An alternative to monitoring employee performance)
bealprojects.com
7

8. Easier when your problems are of a
of a very straightforward nature...
E.g., “reduce network outage by replacing
components that started to fail with unexpected
regularity.”
Intrinsic motivation, clear responsibilities, and
striking agreement about what needs to change
removes the need of measuring performance at
the individual level.
bealprojects.com
8

9. ... much harder when looking at
more complex problems
“Reasons why
projects fail”
bealprojects.com
9

10. Many potential causes...
bealprojects.com
10

11. Why measure
To go from
“we have a
problem”
to
“there is a performance
problem, and out of 15
potential causes, here
are the 3 that we need to
attack first.”
bealprojects.com
11

12. Alternative viewpoint
(Adriana Beal, Oct 2010)
Partial agreement with Plats and
Sobotka:
 No need to measure individual
performance to control or
motivate behavior.
 There are better ways to
motivate people to achieve high
performance.
bealprojects.com
12

13. Alternative viewpoint
(Adriana Beal, Oct 2010)
However, there is a much better
reason to measure individual
performance...
bealprojects.com
13

14. Why measure?
To increase understanding of what affects
business analysis processes and results,
to identify areas where action is needed, or
there is opportunity to scale successes, and
to make the right decisions about where to
invest time and resources.
bealprojects.com
14

15. What NOT to measure
What can't be used for learning and
improving performance
What can't be expressed in
numbers and units of measure, or
require subjective criteria
What you could learn using
a more economical measurement
bealprojects.com
15

16. What NOT to measure
What can't be used for learning and
improving performance
“Total number of requirements
(e.g., a size metric)”
Things that you could learn using
a more economical measurement
Source:“Measurably improving you requirements”
modernanalyst.com: http://tiny.cc/0lyp3
bealprojects.com
16

17. What NOT to measure
What requires subjective criteria
Things that you could learn using
“Requirement stability (i.e., what is
a more economical measurement
the probability of the requirement changing?)”
Source:“Measurably improving you requirements”
modernanalyst.com: http://tiny.cc/0lyp3
bealprojects.com
17

18. What NOT to measure
“% of customers satisfied with our software
as a service tool” - by monthly phone survey
Things that you could learn using
a more economical measurement
bealprojects.com
18

19. What NOT to measure
“% of customers satisfied with our software
as a service tool” - by monthly phone survey
Things that you could learn using
a more economical measurement
conversion rate subscription
after trial period renewal rate
bealprojects.com (Already available in your billing system)
19

20. What to
measure
A good indicator
helps you
understand where
you are, which way
you are going, and
how far you are
from where you
want to be.
bealprojects.com
20

21. What to measure
Goal
Question
Measure
bealprojects.com
21

22. What to measure
Goals & questions may be about...
Efficiency Goal
Effectiveness Question
Impact
Measure
Best practices
bealprojects.com
22

23. Is our work being done on time and
with high quality?
Efficiency Effectiveness
Requirements cycle Defects in requirements
length artifacts
Waste
Best practices
Adherence to change
Improvement on on- management
time delivery standards & procedures
bealprojects.com
23

24. Are we succeeding in delighting our
internal/external customers?
Impact
 Positive feedback from customers
 Requests for change attributable to the BA work
Quality problems in implemented solution
attributable to the BA work
Progress made in actual delivery date versus
requested and promised date
bealprojects.com
24

25. Creating meaningful performance
measurement systems
Comprehensive view: inputs,
processing steps, outputs
Multiple perspectives: individual, process,
department, organization
Few KPIs (3-5)
Lagging and leading indicators
(no more than 7-10 measures in total)
bealprojects.com
25

26. A hierarchy of objectives and KPIs
Ensure 100% regulatory
compliance
IT
Implement regulatory
BA
compliance reqs on time
Deliver acceptable regulatory
bealprojects.com
compliance reqs on time
26

27. M Measuring for the wrong
reasons
 “keep score” Creativity and
 punish people progress
 micro-manage inhibitors!
 find something wrong
with the work done
bealprojects.com
27

28. “The consequences of a failed performance
should be personal development,
new perspectives, improved judgment,
skill enhancement, and all-around learning.”
(Samuel A. Culbert in: GET RID OF THE PERFORMANCE REVIEW!)
bealprojects.com
28

29. Learning from measurement
Source: Daniel Powell (Requirements Evaluation Using Behavior Trees - Findings
from Industry)
Adapted from Tim Olson (Measurement Driven Project Management, 2004 CMMI
Conference)bealprojects.com
29

30. Finding the “bright spots”
Defects 17 8 2 .5
per 100 reqs
bealprojects.com
30

31. Finding the “bright spots”
By investigating the strategies of your top
performers, you could discover, for example, that
the most successful BAs are spending time
creating glossaries and data models for their
projects, while the average BA is simply
describing software behavior.
Identify and promote the winning
strategies to the rest of your team.
bealprojects.com
31

32. What really matters for high
performance...
(According to 4 decades of research)
bealprojects.com
32

33. What really matters for high
performance...
 Interesting and meaningful work
 Autonomy
 Personal growth
 Clear and reasonable expectations
 Consistent & fair evaluation
 Frequent and usable feedback
bealprojects.com
33

34. A well-designed performance measurement
system can support every element of high
performance, helping:
 establish clear and reasonable expectations
about our performance,
 identify opportunities for personal
development,
 facilitate consistent and fair evaluation of the
work done, and
 make our work more interesting and meaningful
by making our contributions more visible.
bealprojects.com
34

35. You know that
a performance
measurement
system is
working...
... when you see performance data serving as the basis for
frequent discussions about performance levels and trends,
and BAs starting to develop their own strategies to deal with
performance issues.
bealprojects.com
35

36. Measuring the Performance of
Business Analysts
WI BADD 2010
For additional resources please visit
http://bealprojects.com
Adriana Beal
adriana@bealprojects.com
bealprojects.com
36