Twenty-five years ago, Peter Senge wrote “The Fifth Discipline”, considered the seminal text for how to build a learning organization. With obvious benefits, and the recipe needed for success, why don't we see more learning organizations? That was twenty-five years ago!
As Ash Maurya pointed out in his new book, Scaling Lean, “The goal isn't learning, the goal is traction.” Without a process that helps us turn learning into momentum, a culture of learning gets us nowhere. Without a strategy to overcome the challenges of distributed decision-making, we still make most decisions in ignorance.
Let's dust off these old ideas in light of all the discoveries we've made over the last decade in Lean Startup, Agile, and Continuous Delivery.
What are the critical elements that are missing in our organizations that prevent us from building a learning organization? What are the key obstacles to success?
In this talk, we'll breakdown the concept of a learning organization into discrete system components and analyze the requirements like engineers. Then we'll discuss a strategy for overcoming the challenges and iteratively transforming our organizations into learning organizations. From the building blocks of culture, to the design of organizational architecture, we'll build a roadmap for learning how to learn together.
Want to learn your way to being an AWESOME company? Learn how to become a learning organization.
7. News Flash!
For the first time in our industry,
we have all the tools necessary
to pull this off.
8. Data-Driven Software Mastery
Five Talks
Stop Getting Crushed By Business Pressure
How to Build a Learning Organization
Top 5 Reasons Why Improvement Efforts Fail
Make a F.O.C.O.L. Point!
10. Five Disciplines of a Learning Organization
Personal Mastery
Mental Models
Shared Vision
Team Learning
Systems Thinking
These disciplines are emergent with science.
11. , Developer & Consultant for 15+ years
Specialized in Statistical Process Control (SPC)
and Supply Chain Optimization from Lean Manufacturing (I <3 cool data)
Continuous Delivery infrastructure, design & automation strategies
Janelle Klein
Who Am I?
How to Measure the PAIN
in Software Development
Janelle Klein
leanpub.com/ideaflow
Data-Driven
Software Mastery
I’m also a hobbyist Cognitive Scientist
20. The amount of PAIN was caused by…
Likeliness(of((
Unexpected(
Behavior(
Cost(to(Troubleshoot(and(Repair(
High(Frequency(
Low(Impact(
Low(Frequency(
Low(Impact(
Low(Frequency(
High(Impact(
PAIN(
21. What Causes Unexpected
Behavior (likeliness)?
What Makes Troubleshooting
Time-Consuming (impact)?
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
Most of the pain was caused by human factors.
What causes PAIN?
22. What Causes Unexpected
Behavior (likeliness)?
What Makes Troubleshooting
Time-Consuming (impact)?
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
What causes PAIN?
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
Most of the pain was caused by human factors.
23. PAIN occurs during the process of
understanding and extending the software
Complex(
So*ware(
PAIN
Not the Code.
Optimize “Idea Flow”
24. My team spent tons of time working on
improvements that didn’t make much difference.
We had tons of automation, but the
automation didn’t catch our bugs.
25. My team spent tons of time working on
improvements that didn’t make much difference.
We had well-modularized code,
but it was still extremely time-consuming to troubleshoot defects.
26. The hard part isn’t solving the problems
it’s identifying the right problems to solve.
“What are the specific problems
that are causing the team’s pain?”
27. measures the time spent on:
Idea Flow
x
Troubleshooting
x
Learning
x
Rework
Quality Risk Familiarity Risk Assumption Risk
28. Idea Flow Mapping Tools
(Open Source, Supported GA ~June 2016)
github.com/ideaflow/tools
29. “What caused the pain
in this case?”
Categorize the Problems with #HashTags
#ReportingEngine
#Hibernate
#MergeHell
30. 1. Test Data Generation
2. Merging Problems
3. Repairing False Alarms
1000 hours/month
Add up the Pain by Category
31. 0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Chaos Reigns
Unpredictable work stops
fitting in the timebox
Troubleshooting
Progress
Learning
Analyzing Long-Term Trends
42. Learn & Adapt
Just because we repair the car…
doesn’t mean we know how to drive.
43. Quality Target
Lower Control Limit
Upper Control Limit
X"
This is “Out of Control”
Lower Variability = Better Control
Steering Wheel = Process Control
44. “Pain Control” in Software Development
Optimal Friction
Upper Control Limit
X"
“Out of Control”
20min
0m
Average Pain per Incident
This is Better
Target
Control Limit
Categorize the Pain Type
45. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
Improve Quality of Decisions
Idea Flow Learning Framework
46. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
Target - The direction of “better”
Target: Optimize the Rate of Idea Flow
Idea Flow Learning Framework
47. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
Input - The constraints that limit our short-term choices…
Idea Flow Learning Framework
48. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
Output - The pain signal we’re trying to improve
Idea Flow Learning Framework
49. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
Focus on the biggest pain…
F ocus!
Idea Flow Learning Framework
50. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
1. Visibility - Identify the specific patterns
1.
Visibility
Idea Flow Learning Framework
51. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
2. Clarity - Understand cause and effect
2.
Clarity
Idea Flow Learning Framework
52. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
3.
Awareness
3. Awareness - Stop and think to adjust habits
Idea Flow Learning Framework
53. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
4. Run Experiments to Learn What Works
Idea Flow Learning Framework
54. Optimal Friction
Upper Control Limit
X"
“Out of Control”
20min
0m
Average Pain per Incident
This is Mastery
Target
Control Limit
Categorize the Pain Type
This is the Process of Mastery
56. “The Art of the Scientist” Workshop, Ash Maurya
Ash:
“What’s the most important thing that scientists do?”
“Run Experiments!”
“Analyze Data!”
Getting Involved with Lean Startup
57. “The Art of the Scientist” Workshop, Ash Maurya
NO!
?
?
?
?
?
?
?
Getting Involved with Lean Startup
58. “The Art of the Scientist” Workshop, Ash Maurya
Ash:
“The most important things scientists do is build models.”
Getting Involved with Lean Startup
61. Test the Model
(Experiments)
Refine the Model
(Patterns)
False&
Predic,on&
Scien,fic&
Model&
Modeling
(Inductive)
Experimentation
(Deductive)
Empirical Data
The Scientific Method
We can’t ignore
this part.
62. Test the Model
(Experiments)
Refine the Model
(Patterns)
False&
Predic,on&
Scien,fic&
Model&
Modeling
(Inductive)
Experimentation
(Deductive)
Empirical Data
Science is a Feedback Loop
that Fuels Discovery
Creating an explicit model makes
our beliefs testable and our discoveries additive.
We can’t ignore
this part.
63. “The Art of the Scientist”
Lean Startup Workshop, Ash Maurya
Ash:
“The goal isn’t learning, the goal is traction.”
64. Optimize the Rate of Flow in a Supply Chain
Focus on
Limiting
Constraint
(Goldratt’s Theory of Constraints, “The Goal”)
66. Optimal Friction
Upper Control Limit
X"
“Out of Control”
20min
0m
Average Friction per Customer
This is Traction
This is the Process of Lean Startup
High
Friction
Min
Friction
“No”
Threshold
Customer
Happiness
Categorize the “No” Type
67. “The Idea Flow Factory”
(supply chain model)
Optimize the Rate of Idea Flow
Across the Software Supply Chain
69. Optimal Friction
Upper Control Limit
X"
“Out of Control”
20min
0m
Average Pain per Incident
This is Traction
Target Rate
Control Limit
Categorize the Pain Type
This is the Process of Idea Flow Mastery
70. We can apply the entire suite of cost & risk reduction tools
from Lean Manufacturing!!
Supply Chain Optimization
Statistical Process Control (SPC)
Optimal Friction
Upper Control Limit
X"
“Out of Control”
20min
0m
Average Pain per Incident
Target
Control
Limit
Throughput Accounting
71. Test the Model
(Experiments)
Refine the Model
(Patterns)
False&
Predic,on&
Scien,fic&
Model&
Clarity
(Why)
Awareness
(How)
Visibility
(What)
Use Science to Fuel “Explicit Mastery”
Model “Decision Principles”
72. What’s a Decision Principle?
1. How do I evaluate my situation?
2. What should I optimize for?
Answers Two Questions
73. The Haystack Principle
Lots of unvalidated changes
Easier to find the needle.
Decision Principle: The Haystack Effect
Optimize for small manageable haystacks.
74. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
Use Science to Fuel “Explicit Mastery”
Systematically codify Decision Principles
from lessons learned
79. Observe Pattern
“Do I know this pattern?”
Metaphors are patterns of conceptual “Shapes”
80. Three Different Types of Metaphors
Object Patterns
(Thing)
Context/Relationship Patterns
(Space)
Process Patterns
(Time)
We “see” the world through a fabric of metaphor.
82. “This looks like
a tech debt thing.”
1. Categorize patterns by concept type
83. “This looks like
a painful tech debt thing.”
2. Categorize variations by intensity.
Pain level = 8
84. = Little Debt
= Medium Debt
= Big Debt
Our Beliefs about Developer Pain
85. PAIN occurs during the process of
understanding and extending the software
Complex(
So*ware(
PAIN
Pain occurs over TIME.
86. What do you think happens when we
make this transition?
Process Patterns
(Time)
Object Patterns
(Thing)
87. What Causes Unexpected
Behavior (likeliness)?
What Makes Troubleshooting
Time-Consuming (impact)?
Take a Closer Look at the Patterns
Familiarity Mistakes
Stale Memory Mistakes
Semantic Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
Most of the pain was caused by human factors.
88. “What caused the pain
in this case?”
We Start to Experience the World Differently
#ReportingEngine
#Hibernate
#MergeHell
92. The Clarity Principle
Our understanding deepens when we explain
the similarities and differences of experiences
and the relationships of cause and effect
102. Circle Leader
Circle Member
Focus: What’s the problem to solve?
Observe: Ask questions about the facts
Conclude: What’s causing the pain?
Optimize: How do we reduce pain?
Learn: What questions does this raise?
Observation
Questions
Make a F.O.C.O.L Point!
Summarize lessons learned in a
#HashTagged Blog.
Deep Reflection in Open Mastery Circles
107. Me: "So first, George, can you describe what you were trying to do before you
encountered the conflict?”
George: "This was back when I was getting the token to work in the email. I
wrote the code to generate the token, edited the email template to include the
token in the email link, but the token wasn't showing up in the actual email."
Me: "So what did you do next?"
George: At first, I thought there was something wrong with the token variable
code, but the problem turned out to be a misspelled variable name in the
email template. Doh!"
Focus: What Major Factors Caused the Pain?
Observe: Ask Questions About the Facts.
109. Painful Consequences: Consequence
Conclusions: What Caused the Pain?
Ran experiments by sending emails over and over again
(23 emails with a missing tokenId)
Redeployed application when uncertain about the cause
Ambiguous Clue
Manual Execution (slow)
110. Optimize: How could we have avoided the pain?
What’s an alternative strategy
that could have mitigated these risks?
George: My main strategy was to do a small #VerticalSlice by getting the
“email send” part working, before adding the token logic for the security part. I
understand how the #ManualExecution and an #AmbiguousClue caused the
friction, but I'm not really sure what I would do differently if I could do it again.
Me: "George, can you describe your original strategy to mitigate risk?”
111. Optimize: How could we have avoided the pain?
What’s the strategy pattern we’re
trying to implement?
Alternative Strategy:
APIs for each email workflow
Wire up email last
Strategy
112. Optimize: How could we have avoided the pain?
What’s the strategy pattern we’re
trying to implement?
Alternative Strategy:
APIs for each email workflow
Wire up email last
Strategy
113. Optimize: How could we have avoided the pain?
Alternative Strategy:
APIs for each email workflow
Wire up email last
Situation
What’s the situation we need to recognize
in order to decide to use the strategy?
114. Situation Strategy Consequence
Developer Journeys
#HiddenOutputs
Create
#ObservableInterfaces
Avoid #AmbiguousClues &
#ManualExecution
Optimize: How could we have avoided the pain?
115. Learn: What questions does this raise?
What questions
should we ask ourselves in the future?
In what situations
should we ask the question?
119. Experiment
Time
Our Perception of Time is WAY OFF
Setup Experiment Analyze Results &
Decide Next Experiment
Execute
waiting - time goes slow
doing - time zooms by
121. The Code Sandwich Principle
The thickness of the
sandwich increases
troubleshooting
difficulty
Behavior Complexity
Observability
Ease of Manipulation
Optimize for experimentation.
123. The bigger the cloud of possibilities,
the more we need a strategy.
124. The Clarity Principle
Our understanding deepens when we explain
the similarities and differences of experiences
and the relationships of cause and effect
127. The Awareness Principle
We can only avoid a mistake
if we predict the consequences
of the decision in the moment.
128. 18:120:00
George’s Painful Experience
I know I made a big haystack,
I’ll do better next time.
This is hindsight bias, it’s not that easy!
The Experience Review
129. The Challenge
We tend to make auto-pilot decisions and do what’s familiar
How do we break decision habits?
130. Imagine your brain is a
decision-making engine
written in code.
Breakpoint
Stop and Think!
Changing Decision Habits
132. The Checklist Manifesto
Atul Gawande
Scaling Lean (Lean Startup)
Ash Maurya
+
Change Decision Habits with Strategy Experiments
133. 14:230:00
I want to avoid this…
Thinking Checklist
Is my current approach likely to cause a big haystack?
Situation: start of subtask
Let’s Make a Checklist!
“What question could I ask my future self to
recognize similar risks in the future?”
“In what situation would I ask the question?”
134. 0:00
Stop and Think:
Is my current approach likely to
cause a big haystack?
Predict: Small haystack
Strategy Experiments
135. 18:120:00
Stop and Think:
Is my current approach likely to
cause a big haystack?
Predict: Small haystack
False Prediction
Strategy Experiments
136. 18:120:00
Stop and Think:
Is my current approach likely to
cause a big haystack?
False Prediction
Strategy Experiments
High-Risk Situations
1. Unraveling sweater
2. Integration-heavy change
3. High state variation
4. Minimum scope is big
Q: Is my current approach likely to
cause a big haystack?
Start of Subtask
139. The Awareness Principle
We can only avoid a mistake
if we predict the consequences
of the decision in the moment.
140. Three Stages of Mastery
Visibility
Clarity
Awareness
See
Explain
Predict
141. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
Use Science to Fuel “Explicit Mastery”
Systematically codify Decision Principles
from lessons learned
142. Five Disciplines of a Learning Organization
Personal Mastery
Mental Models
Shared Vision
Team Learning
Systems Thinking
These disciplines are emergent with science.
143. How to Measure the PAIN
in Software Development
Janelle Klein
These are the Blueprints!
++
146. Input:
Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term looplong-term
loop
1.
Visibility
2.
Clarity
3.
Awareness
F ocus!
Output: “Friction” in Idea Flow
How do my decisions affect your experience?
Use Science to Fuel “Explicit Mastery”
148. Bigger Social Network of Brains
Collaboration to Perform a Bigger Function (Organization)
Component
Component
Hub
Hub
Hub
The Challenge: Knowledge & Decision-Making are Distributed
155. 4. Optimize the Whole
2. Design the Strategy
1. Identify Obstacles
(Focus)
3. Learn What Works
(Traction)
Steering
Wheel
Create a Steering Wheel to Drive the Organization!
$$ $$
156. Focus: Quality of Improvement Decisions
Focus Traction
$$$ $$
Focus: Quality of Investment Decisions
Risk
Translator
(interface)
Management
(Coordination)
Engineering
(Execution)
Open Mastery Learning Framework
Idea Flow Pain Sensor
157. If your Interested in doing this:
Free Support Network for
Data-Driven Software Mastery
158. Open Mastery Learning Platform
Idea Flow Mapping Tools
Team Mastery Platform
Team
Joe
Sally
Mark
Eric
Community
Collaboration Platform
Anonymized
Data
Project
Tiger
Project
Bear
(REST)
Shared #HashTag Glossary
of Patterns & Principles
with Examples
159. The Conversation
is on Slack
We’re debating the patterns
and principles of Idea Flow in Wikipedia
It’s a Developer Party!
We’re sharing our ideas
over Blogs
(and managers are invited)
160. Industry Peer Mentorship Network
Companies
Community
Groups
HQ in
Austin
Open Mastery
Austin
meetup.com/Open-Mastery-Austin
161. Janelle Klein
openmastery.org @janellekz
Check out openmastery.org for details.
Read my Book.
Think About It.
FREE with
Reading GroupBuy It
How to Measure the PAIN
in Software Development
Janelle Klein