Informal learning measurement (social networks, performance support, coaching) is on the rise. It is important to think about measuring this investment and comparing it against formal programs to determine its mix and place. In addition, the root cause of low learning impact is lack of manager support. This webinar will address why this happens and how to overcome it. In this webinar, you will: 1. Define informal learning and a learning taxonomy to incorporate it. 2. Discuss when and how to measure informal learning. 3. Understand the reasons for scrap learning and low manager support. 4. Discuss tools and methods to increase manager support and optimize learning impact.

1. How to Measure Informal Learning and Engage
Managers to Optimize On-the-Job Impact
You can listen to today’s webinar using your computer’s speakers
or you may dial into the teleconference.
If you would like to join the teleconference,
please dial 1.866.469.3239 and enter access code 669 720 923.
You will be on music hold until the seminar begins.
#CLOwebinar

2. How to Measure Informal Learning and Engage
Managers to Optimize On-the-Job Impact
Speaker: Jeffrey Berk
Chief Operating Officer
KnowledgeAdvisors, Inc.
Moderator: Daniel Margolis
Managing Editor
Chief Learning Officer magazine
#CLOwebinar

3. Tools You Can Use
• Q&A
– Click on the Q&A panel (?)
in the bottom right corner
– Type in your question in the
space provided
– Click on “Send.”
#CLOwebinar

4. Tools You Can Use
• Polling
– The poll will appear on the
right side of your screen
– Select the best option for
each question
– Click on “Submit”
#CLOwebinar

5. Frequently Asked Questions
1. Will I receive a copy of the slides after the webinar?
YES
2. Will I receive a copy of the webinar recording?
YES
Please allow up to 2 business days to receive these materials
#CLOwebinar

6. How to Measure Informal Learning and Engage
Managers to Optimize On-the-Job Impact
Daniel Margolis
Managing Editor
Chief Learning Officer magazine
#CLOwebinar

7. How to Measure Informal Learning and Engage
Managers to Optimize On-the-Job Impact
Jeffrey Berk
Chief Operating Officer
KnowledgeAdvisors, Inc.
#CLOwebinar

8. Informal Learning and Manager Engagement
Jeffrey Berk
Chief Operating Officer
KnowledgeAdvisors

9. Informal Learning Measurement Approaches
What is “Informal Learning”
Research on Informal Learning tactics
Measurement in Informal Learning
April 18, 2011 © 2010, KnowledgeAdvisors 8

10. Informal Learning
Informal Learning: Knowledge
transfer that occurs without the
assistance of structured curriculum.
Source: KnowledgeAdvisors
April 18, 2011 © 2010, KnowledgeAdvisors 9

11. What Informal Learning is Supported?
45
# of respondents
40 38
35
29 28
30
24
25
20
15
10 7
5
0
Mentoring & Communities of Virtual Knowledge Performance Other
Coaching (M&C) Practice (CoPs) Sharing (VKS) Support Systems
(PSS)
73% 56% 54% 46% 13%
% of respondents supporting each type of informal learning
Source: KnowledgeAdvisors Informal Learning Research, 2009
April 18, 2011 © 2010, KnowledgeAdvisors 10

12. How Much is Spent on Informal Learning?
% supporting each method
80%
73%
70%
60% 56% 54%
50% 46%
40%
30%
20%
13%
10%
0%
Mentoring & Communities Virtual Performance Other
Coaching of Practice Knowledge Support
Sharing Systems
Avg
$70K $85K $110K $170K
spend
Source: KnowledgeAdvisors Informal Learning Research, 2009
April 18, 2011 © 2010, KnowledgeAdvisors 11

13. Informal Learning Measurement
Are your Execs asking for Are you measuring business benefits?
quantitative measures?
While only 21% of respondents indicated that their
Executives are asking for quantitative measures, twice that
number expect to be asked soon.
Source: KnowledgeAdvisors Informal Learning Research, 2009
April 18, 2011 © 2010, KnowledgeAdvisors 12

14. Measurement Correlates to Budget
Measurement L&D Spend /
COPs M&C VKS EPSS
discipline person
Those who
measure $958 $168,000 $84,000 $250,000 $250,000
Those who don’t
measure $1004 $68,929 $82,679 $97,917 $129,286
Overall $996 $71,042 $83,026 $109,615 $167,750
% delta ( over ) -5% 143% 6% 155% 93%
• The group measuring Informal Learning spends less per person on L&D
annually.
• However, their Informal Learning spend is notably higher than the non-
measurement group in most areas.
• We might conclude that measuring impact of Informal Learning enables
organizations to demonstrate outcomes and justify greater allocation of
L&D $ to these critical areas.
Source: KnowledgeAdvisors Informal Learning Research, 2009
April 18, 2011 © 2010, KnowledgeAdvisors 13

15. Informal Learning Measurement : What?
Solution Experience Benefits
Availability/ Value / Business Results
Accessibility Belonging / ROI
Performance
Fulfills objectives Quality
impact
Application to
Effectiveness Engagement
the job
Right solution Professional
Usability
for needs growth
Support Systems (Manager, Resources, Rewards)
Source: KnowledgeAdvisors Informal Learning Measurement White Paper, 2010
April 18, 2011 © 2010, KnowledgeAdvisors 14

16. Informal Learning Measurement : When?
At the conclusion
5. Complete 1. Become
and/or aware /
recommit consider
4. Use, 2. Search,
Periodically learn, access,
practice, sign in
contribute or join
3. Engage
At each interaction
Source: KnowledgeAdvisors Informal Learning Measurement White Paper, 2010
April 18, 2011 © 2010, KnowledgeAdvisors 15

17. Informal Learning Measurement : How?
Systems Based Tools Based People Based
• Web analytics • Needs analysis • Observation
• Supporting system • Usability studies • Interviews
statistics • Feedback links • Focus groups
• Performance data • Content ratings • Case studies
• Surveys
• Assessment
High Tech High Touch
Source: KnowledgeAdvisors Informal Learning Measurement White Paper, 2010
April 18, 2011 © 2010, KnowledgeAdvisors 16

18. Informal Learning Measurement
Easy wizard for administration
of informal learning measurement
Micro Poll
Experience
Pop
Up Experience
Robust
reports
April 18, 2011 © 2010, KnowledgeAdvisors 17

19. Manager Engagement Measurement Approaches
What is “Scrap Learning”
Research from the War on “Scrap”
Measurement to Combat the “Scrap”
April 18, 2011 © 2010, KnowledgeAdvisors 18

20. Scrap Learning
60% of learning is not applied to the job…
That’s Scrap. Scrap learning is learning not applied on
the job. It is the amount of training that is wasted.
Source: KnowledgeAdvisors , Metrics that Matter
What causes scrap learning?
April 18, 2011 © 2010, KnowledgeAdvisors 19

21. Causal Factors of Scrap
Data from thousands of on‐the‐job evaluations shows
the major barriers to impact occur on‐the‐job
60.00%
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%
content not prevented or no opportunity other high other
practical discouraged from priorities
using
This is where the power of training
Source: KnowledgeAdvisors , Metrics that Matter
leaks out of the organization
April 18, 2011 © 2010, KnowledgeAdvisors 20

22. Manager Support is Needed
Manager support comes both before and
after training. Key steps include:
Evaluate learner Get involved after
readiness Training training
Follow up on
Expectation setting
expectations
Provide performance
resources
Measurement is a difference maker to achieve manager
support and reduce scrap learning!
April 18, 2011 © 2010, KnowledgeAdvisors 21

23. Research: Learner Readiness
Only 21% of organizations pre-evaluate learners “most of the time”
or “all of the time” prior to sending them to training.
Do you pre-assess learners to
determine their readiness for
training?
April 18, 2011 © 2010, KnowledgeAdvisors

24. Research: Expectation Setting
75% of companies indicate that managers set post-training
performance expectations with learners less than 25% of the time.
How often do you set expectations
prior to training?
April 18, 2011 © 2010, KnowledgeAdvisors

25. Research: Manager Involvement
A large minority (44%) of managers have little involvement in how
their employees use training on the job. Scrap learning can be
reduced by getting managers involved.
April 18, 2011 © 2010, KnowledgeAdvisors

26. Research: Manager Follow Through
Fewer than 35% of managers follow-through to validate that training
actually accomplished anything.
April 18, 2011 © 2010, KnowledgeAdvisors

27. Research: Resource Provisions
Fewer than 25% of managers provide resources to support learners
application of training on the job.
April 18, 2011 © 2010, KnowledgeAdvisors

28. Research: Manager Support Summary
Here are the general estimates of
manager support for selected activities:
21% 52%
Evaluate learner Get involved after
readiness Training training
25% 35%
Follow up on
Expectation setting
expectations
25%
Provide performance
resources
Measurement is a difference maker to achieve manager
support and reduce scrap learning!
April 18, 2011 © 2010, KnowledgeAdvisors

29. Measure Learner Readiness Before
Implement a pre-evaluation to managers.
Keep it simple and concise.
Case: Telecom Co.
When: Strategic, costly programs
How: LMS sends pre-evaluation to manager
via integration with Metrics that Matter
Purpose: Used for L&D to obtain ‘right ‘
learners in class and remind managers of their
role in the learning process.
Source: KnowledgeAdvisors , Metrics that Matter
April 18, 2011 © 2010, KnowledgeAdvisors 28

30. Measure Learner Readiness During
Implement end of class, follow-up
and manager evaluations to critique
support tools.
Case: Payroll Processor
When On customer service program
How: Post event evaluations
revealed high satisfaction but low
manager report
Purpose: Used to convince
management the problem wasn’t the
training. They got funds to do
manager performance coaching as a
result.
Source: KnowledgeAdvisors , Metrics that Matter
April 18, 2011 © 2010, KnowledgeAdvisors 29

31. Sharing between Learner and Manager
Answer Sharing
Answer Sharing is to automatically
route student responses to their
manager or coach.
The learner can identify who they
would like to route their responses
to via an email address.
When the learner completes the
evaluation for a training program,
then the manager/coach then
receives an email with the
responses to those tagged
questions.
April 18, 2011 © 2010, KnowledgeAdvisors 30

32. Goal Setting and Periodic Check Ins
Check-in to Change
Program managers can enable and oversee a process over time
that facilitates the collaboration between program participant and manager.
Employees are asked to enter goals. These goals are then stored and tracked
for both the employee and the manager to review. Automated ‘Check-ins’
occur to facilitate feedback on goal status, the attainability of their goals, and
also share possible obstacles and successes amongst learner and manager.
This is done via an online portal
April 18, 2011 © 2010, KnowledgeAdvisors 31

33. Contact Information
For further information on the information contact:
Jeffrey Berk
Chief Operating Officer
+1 312 676-4411
jberk@knowledgeadvisors.com
www.knowledgeadvisors.com

34. Join Our Next CLO Webinar
Preparing Your Workforce for Tomorrow’s
Challenges
Thursday, April 21, 2011
CLO Webinars start at 2 p.m. Eastern / 11 a.m. Pacific
Register at www.clomedia.com/events
Join the CLO Network: http://network.clomedia.com/
#CLOwebinar