For many years, we could proudly report the cost of learning (cost per learning hour, the ratio of L&D professionals per 1000-employee, or how many people our high-priority programs reached); however, it was very difficult to demonstrate the actual value of learning to the business. After years of focused efforts, we now use a pragmatic and cost-effective approach to measure the business impact of learning. This document covers: • The pillars of our effectiveness evaluation framework • Moving the conversation with stakeholders from cost to value • Developing a pragmatic measurement approach for a learning program: challenges to expect and goals to set

1. 1Copyright © Capgemini 2015. All Rights Reserved
How to Gather the Right Evidence to Demonstrate the
Business Value of learning?
Dr. Regis P Chasse
Director of Curriculum, Capgemini University

2. 2Copyright © Capgemini 2015. All Rights Reserved
Measuring our Learning Programs
Course participant is twice more
likely to become a certified PM
43% increase in profitability of
projects run by PMs after
participating in the course
50% of participants became more
effective managers, as rated by
their subordinates
95% of participants’ teams were
more satisfied and engaged after
their managers attended the
training
BEFORE
Measured efficiency
i.e. cost of learning
hour, delivery mix,
learning staff per FTE
Only partially measured effectiveness
i.e. participant feedback only, seldom
feedback from managers, and only as
afterthought once program was built and
running
UNIVERSITY BOARD
need a strutured approach to
measuring effectiveness of
learning
NOW

3. 3Copyright © Capgemini 2015. All Rights Reserved
The goal of EvaluateIt is to bring the OU evaluation
to the next level
…help the
Capgemini
LEARNING
NETWORK
to implement a consistent
and robust evaluation
method to its training
programmes
… help the
BUSINESS
to get assurance
through evidence of
getting best value for
the learning
investment
EvaluateIt will …
… help the
UNIVERSITY
and LOCAL L&D
implement a continuous
improvement programme
rather than run expensive and
infrequent re-design cycles
…Help the
PARTICIPANTS
continue to enhance their
learning through assessment
and provide their inputs on
the quality of the learning
through timely and effective
feedback
… close the
gap in ALIGN
around evaluation
where we do not have
a consistent
evaluation framework
Bring people together to develop their
capabilities and achieve the full potential
of our Group and its businesses
University VISION
Bring people together to develop their capabilities and achieve the
full potential of our Group and its businesses
EvaluateIT GOAL
Understand the impact of the training Optimize and refine
learning programs + validate financial investment

4. 4Copyright © Capgemini 2015. All Rights Reserved
EvaluateIt evaluation framework
Like It
Relevance, Experience and Intent to apply
Show ItKnow It Impact It
Knowledge
Acquisition and
Retention
Behavior and
Individual
Performance
Adoption and
Business
Performance
Purposeful exclusion of ROI  to focus on individual performance and business outcomes
* Model inspired from Kirckpatrick, Bersin and TDRp
*

5. 5Copyright © Capgemini 2015. All Rights Reserved
How to develop a measurement approach for a
learning program
Which level of
measurement
to implement
How to collect
the data at
each level
Frequency
Design program
effectiveness
report out
Measurement
Approach
Learning
Program
Design
Run Analyze Refine Communicate
Business
Case
Challenge Execute Evaluate
Integrate
measurement
approach in
design of
learning
program
Run program
and
associated
measurement
approach
Analyze
measurement
findings, provide
insights and
recommendations
for improvements
Business
sponsor
Business
objectives
Expected
business
outcome
Key KPI to be
impacted
Expected
individual
performance
gains
Present
recommendations
to stakeholders,
and integrate in
„refined” business
case as needed
Make the
story
known!
Connect

6. 6Copyright © Capgemini 2015. All Rights Reserved
Lessons Learned
1) Do not apply all 4 levels of measurement for all programs:
o LikeIT and ShowIT are pivotal, and can be implemented at minimal cost, and
provide enormous information about your programs
o KnowIT and ImpactIT are more complex and costly  be selective on which
programs to implement them, e.g . large scale porgrams with significant
business impact
o ROI is typically not an issue for business stakeholders, business outcomes are
enough.
2) Invest in change management and education of learning teams
3) Establish an effectiveness measurement team, with key capabilities
(evaluation design, survey taking & interpretation, interviews, tests, analysis)
4) Create a plan for communicating with stakeholders before/during/after
measurement
Communicate BROADLY about the findings!!

7. 7Copyright © Capgemini 2015. All Rights Reserved
Let’s discuss!
• Further clarification on framework or steps?
• Can this approach be used in your organization?
• How would you adapt the measurement framework for your
organization
• Any barriers to implementating the framework?
• ...?

8. 8Copyright © Capgemini 2015. All Rights Reserved
Capgemini Business Case Template
Business Objective XXXXXXXX
Business Impact & KPI XXXXXXXX include associated business KPI impacts
Flywheel Topic Which flywheel topic does this initiativesupport (Competitiveness, Growth, Innovation, Talent) & how?
Target Group Name of target group
 SBUs affected:
 Target group in Capgemini total:
 Grade(s) in target group (A, B, C, D, E, F):
 # of participants to be trained in 2 years:
 % trained people of target group total:
 How will these people be selected:
 How quickly do we need to reach % trained people:
 What happens to the people left untrained at end of 2 year period:
Individual Performance
Objectives
XXXXXXXXX include individualperformance impact
Root Cause(s) Root causes of performancegap:
 Capability:
 Work environment:
 External factors:
Learning Objectives Needed Skills & Knowledge
 Awareness:
 Understanding:
 Demonstrating:
 Mastering:
Proposed Solution Positioningin Global Curriculum (must be blended):
High level view of Learning map:
Description of the Solution:
Delivery Channel What delivery channel(s) do you intend to use (Campus / Glocal / Virtual)? – select all that apply
For Virtual please select all that might apply – Virtual Classroom (Adobe Connect), Virtual Classroom (Virtual Campus), Mobile,
Yammer, eLearning (Rapid, L1,L2 or L3), videos, others……
Is this initiativea candidate to trigger migration of the curriculumto the Virtual Campus: Yes / No
•Curriculum: XXXXXX
•Name of new program: XXXXXXX

9. 9Copyright © Capgemini 2015. All Rights Reserved
Capgemini Business Case Template, continued
Prioritization For your proposed solution what are the key solution priorities(No 1 & No 2):
Speed to delivery / Quality / Cost
Visibility Is this a high visibilityor high risk project: Yes / No (if Yes, design needs signoff from COMEX, risk will need mitigationplan)
Evaluation What evaluation mechanism should we use to demonstrate each of the 4 levels (if required):
 Like It (e.g. course evaluation, action lists, expected impacts stated by participants) - mandatory
 Know It (e.g. knowledge test, multiple choice etc) - optional
 Show It (e.g. 6 month interviews, scenario based testing, individual KPI review) - optional
 Impact It (e.g. business unit or project KPI review, % BU / project team trained) – aligned with above business objective
Delivery Date What is the explicit delivery date agreed with business (who in business has agreed to this)?
[Needed for all learning solutions,irrespective of delivery channel]
Project Planning Start & end date (Design & Build):
(For Virtual Learning Planning the University needs to plan over the year given the availability of resources)
Investments Use the Excel spreadsheet below for your calculationand copy/paste it into this template:
Signed off by: Date:
Executive Sponsor Curriculum:
Urgent & relevant Business need
Global Content Owner:
Up-to-date Content
Curriculum Director:
University perspective on Global Curriculum
Director Business to Learning:
University Perspective on clear Business Case
University Board
Group Funding Decision

10. 10Copyright © Capgemini 2015. All Rights Reserved
Capgemini Pre-During-Post Examples
How to measure the effectiveness of the content delivered in campus
learning events?
Bringing participants to International Physical Campus learning events is a significant
investment for the Group. It is important that the effectiveness of the courses delivered
during those events is carefully and more systematically measured.
Therefore, it has been agreed that the effectiveness of the International physical campus
learning events will be optimized and measured as follows:
Prior to events:
• Set clear expectations with participants and their managers up-front in the form of a
“contract”: Through upfront communications, it will be explicit that the participants are
expected to apply the content they will learn during the events. In addition, it is a duty of
the participants’ managers to support them in applying it. To that effect, participants will
be expected to be fully engaged during the entire course (including pre and post work),
and accept to be assessed on their progress.
During the events:
• Ensure participant engagement during the course: The University Qualified facilitators,
University team and HR will ensure participant engagement during the events. They will
provide appropriate feedback and coaching to individuals who are not fully participating.
• Gather immediate feedback at the end of the course: systematically, the University will
gather participants’ feedback, on quality the training received. The feedback will also
include a participant self assessment for them to share with their managers. It will be on
what they have learned, its’ relevance to their role, their intent and plans to apply it.
• Test participants at the beginning and at the end of the course when appropriate: When
knowledge acquisition is part of the objectives of a course, testing can be used at the
beginning of the course as a way to understand where the gaps of knowledge are.
Therefore, for specific courses, participants may be tested at the beginning (baseline)
and at the end (outcome) of a course. For more advanced courses with clearly defined
prerequisites, participants may be tested only at the end of a course.
After the events:
• Assess effectiveness 2 or 3 months after the program: The University will either send a
survey to participants or organize a virtual session to review how the content has been
applied in the field and what the business outcomes resulted from the event.
• Analyze certification success rate: Analyze the success rate of employees gaining
certifications after attending courses targeted at becoming certified.
• Analyze employee engagement: Once a year, the engagement of the key populations
who attended the international physical campus events will be compared
How to measure the effectiveness of the content delivered in FVLJs
programs?
Participants in virtual facilitated learning journeys represent a significant number of Group
employees and , therefore, there is a high visibility for these programs. It is important that the
effectiveness of these programs is carefully measured.
It has been agreed that the effectiveness of the FVLJs will be optimized and measured as
follows:
Prior to events:
• Set clear expectations with participants and their managers up-front in the form of a
“contract”: Through upfront communications and the use of Introduction sessions (“Week
0”), it will be explicit that the participants are expected to apply the content and
principles they will learn during the learning program.
• As specified by learning program entry criteria, participants will be expected to complete
assignments (web-based training, reading) and submit certification applications .
• In addition, it is a duty of the participants’ managers to support them in applying it. To
that effect, participants will be expected to be fully engaged during the entire course
(including pre and post work), and accept to be assessed on their progress.
Management awareness will be implemented through the registration and approvals
process.
During the events:
• Ensure participant engagement during the course: The University facilitators, University
team and HR will ensure participant engagement during the events. They will provide
appropriate feedback and coaching to individuals who are not fully participating.
• Gather immediate feedback at the end of each virtual checkpoint and at the end of the
program: systematically, the University will gather participants’ feedback on quality the
training received, its’ relevance to their role, and their intent and plans to apply it.
• Test participants knowledge throughout the learning journey: During the FVLJ, polling
questions can be used within live events and social collaboration sites as a way to gauge
the acquisition of knowledge. Additionally, assigned self-assessments can foster
understanding on the behalf of the participant.
After the events:
• Assess effectiveness 2 or 3 months after the program: A survey will be sent to
participants to assess the program, how the content has been applied in the field, and
what business outcomes resulted from the program.
• Analyze certification success rate: Analyze the success rate of employees gaining
certifications after attending courses targeted at becoming certified.
• Continued engagement: Social collaboration sites can be used for periodic scheduled
“check-ins” with program participants.

11. 11Copyright © Capgemini 2015. All Rights Reserved
EvaluateIt evaluation framework
Like It
Relevance, Experience and Intent to apply
Show ItKnow It Impact It
Knowledge
Acquisition and
Retention
Behavior and
Individual
Performance
Adoption and
Business
Performance
Purposeful exclusion of ROI  to focus on individual performance and business outcomes
Knowledge tests
after the course
(can be used for
spaced repetition)
Survey of managers or
participants for feedback, or
other indicators on
observed changes 3-6
months after the course
KPIs collected as part of the
operational processes of the
organization. Adoption of the
course in different business units is
also analyzed. Performed once or
twice a year.
Feedback from
participants at the
end of a course
1
2 3 4