Measuring the right things for the right reasons to support team experimentation, learning, and improvement

1. March 27, 2022
Angela Dugan and Sara Caldwell
3Cloud Solutions
Measuring Up

2. Angela Dugan
Delivery Director,
Agile Practice
Manager, Team
Coach
Values = Connection | Generosity | Trust
Passion = Helping humans be the best they
can be
Location = Chicago
Quirks = Obsessed with Halloween, raises
chickens!
Email = adugan@3CloudSolutions.com
URL = https://3cloudsolutions.com
Twitter = @OakParkGirl
Sara Caldwell
Lead Agile Coach,
Operational Health
Enabler
Values = Kindness | Adaptability
Passion = Helping humans be the best they
can be
Location = St. Louis
Quirks = Loves Old Rescued Dogs, Jigsaw
puzzles
Email = scaldwell@3CloudSolutions.com
URL = https://3cloudsolutions.com

3. Metrics

4. Twitter: @OakParkGirl Email: Angela@PolarisSolutions.com

6. Why are we here?

7. © 2021, 3Cloud, LLC., All Rights Reserved.
8
Chat with us!
What Metrics Are You Currently Collecting?

9. Metrics Gone Wrong

13. © 2021, 3Cloud, LLC., All Rights Reserved.
14
Chat with us!
We are curious… are any of the metrics that you are using a little
suspect now?

15. Metric Mindset

16. The Hawthorne (Observer) Effect
Goodhart’s Law
METRIC MINDSET
Cognitive Neuroscience Effects

17. The Hawthorne
(Observer)
Effect
That which is measured
will improve…

18. Goodhart’s
Law
When a measure
becomes a target, it
ceases to be a good
measure

19. 20
This Photo by Unknown Author is licensed under CC BY

20. Consume by the
Team
Surrounded by
Conversation
You Get What You
Measure
METRIC MINDSET
4 Principles for Viewing Metrics
View Complimentary
Metrics

21. Team Metrics
Help team understand how capable they are at
understanding problems and designing realistic
solutions
Help team learn how to execute better, based on
the outcomes of their experiments
Help team pinpoint bottlenecks and weak points, in
areas where they can make changes
Provide near immediate feedback that supports the
team in setting and managing realistic expectations

22. Team Metrics We Love

23. Flow/Responsiveness
• Work in Progress (WIP)
Value
• HEART metrics
Quality
• Team Health
Some Things to Measure

24. Work in
Progress
(WIP)
# of active items

25. WIP –
What It
Costs
Lower WIP = Lower Lead Times
Lower Lead Times = Faster Delivery
Faster Delivery = Faster Feedback
Faster Feedback = Build Right Thing
Build Right Thing = Less Waste & Risk

26. Stop Starting,
Start finishing

27. WIP Gone
Wrong
• Not respecting WIP limits
• Team doesn't swarm
• WIP limit isn’t the thing
28
From Dandy People

28. (Business)
Value
Tangible value delivered to the
organization, end user, customer

29. There is no paint by number
Identify Business Value Themes
• Generate/Save Money
• Increase renewal business
• Improve Customer Satisfaction
• Maintains Compliance
Identify unit of measure
Don't forget to validate the hypothesis later!
WHAT EVEN IS BUSINESS VALUE, REALLY?!?
Business Value

30. Happiness
•How does the user feel about the product?
•Measures: surveys, rating, reviews, etc
H
Engagement
•How often do users return?
•Measures visits per user, session length, actions
E
Adoption
•Conversion from 1st time user to regular user
•Measures number of new and returning users
A
Retention
•Churn rate measure
•Measures return number and frequency
R
Task Success
•Can users achieve their goal or task quickly and easily?
•Measures completed tasks amount & time, error rates
T
MEASURING USER EXPERIENCE
HEART Metrics

31. Business Value
Metrics Gone
Wrong
• Identifying Business Value in a silo
• Letting ‘pet projects’ or “the
loudest voice” win
• Assigning too granularly (story or
task level)
• Validating the value was
experienced
32

32. Team Health
Motivated and respected team members deliver amazing things

33. PROTECTING YOUR BIGGEST ASSET
• List of indicators based
on the team’s
environment
• Helps focus
improvement efforts
• Involves different health
indicators
• Surrounded by
conversation
Team Health Check
Spotify Team
Health Check

34. • How
• Run workshops
• Create a graphical summary of the result
• Use the data to help the teams improve
• There are several variants of health
checks
• Only “perfect time interval” is set by the
team
PROTECTING YOUR BIGGEST ASSET
Team Health Check
© 2021, 3Cloud, LLC., All Rights Reserved.
35

35. Team Health
Gone Wrong
Surrounding Team Health
conversations about how ‘good’ a
leader is or how ‘bad’ a team is
Talk about the person voting, and
not the problem
Have incentives for “all greens”
36

36. Closing

37. © 2021, 3Cloud, LLC., All Rights Reserved.
38
Chat with us!
What Did You Learn Today?

38. Consume by the
Team
Surrounded by
Conversation
You Get What You
Measure
METRIC MINDSET
Take Away - 4 Principles for Viewing Metrics
View Complimentary
Metrics

39. Sara and Angela wrote a (little) Book of Agile Metrics!
The Little Book of Agile Metrics
In our FREE guide, learn how to use Agile
to transform the way your teams work.
• Know what common metrics are
actually telling you
• Learn how to interpret the changes
and trends in your agile metrics
• Discover new metrics you did not
even know you needed to be looking
at!
OR reach out directly at adugan@3cloudsolutions.com or scaldwell@3cloudsolutions.com

40. More great reads…

41. While you’re here, browse our
open positions at 3Cloud!
Have questions or stories of
your own to share?

42. Please don’t forget to share
feedback with us about this
talk!

43. March 27, 2022
Thank you!
© 2021, 3Cloud, LLC., All Rights Reserved.
44
Connect with us on LinkedIn or email to say hi! adugan@3cloudsolutions.com or scaldwell@3cloudsolutions.com