Fordham -How effective decision-making is within the IT department - Analysis...
Training effectiveness
1. Training Effectiveness:
Is the Cost Justified?
A Six Sigma Black Belt Project
Daniel Bloom SCRP
Managing Consultant
Daniel Bloom & Associates, Inc.
Human Capital Consultants
Daniel Bloom and Associates, Inc. 1
2. Evolutionary Nature of
Workplace
Industrial Age Information Age
Daniel Bloom and Associates, Inc.
CompetitivenessCompetitiveness
InnovationInnovation
Human CapitalHuman Capital
Expense
Non-Owned
Assets
From Presentation by Russ Moen of Express Personnel
HR Tampa Monthly Meeting
Collabo
ration
Collabo
ration
05/06/2009 2
3. Premise #1Premise #1
In fiscal year 2008, AmericanIn fiscal year 2008, American
corporations spent $56,200,000,000 oncorporations spent $56,200,000,000 on
providing training to their employeesproviding training to their employees
-- Workforce Management WebsiteWorkforce Management Website
Daniel Bloom and Associates, Inc.05/06/2009 3
4. Premise #2Premise #2
In today’s competitive talent market, the
primary goal for many FTE’s is to enhance their
portfolio of career skills that they can bring to
their current or future employer. Especially true
with late Boomers, Gen X, Gen Y
Daniel Bloom and Associates, Inc.05/06/2009 4
5. Premise #3Premise #3
Two types of “Training” being offered
Hard Skills or Training – Designed to increase an
Employee’s ability to complete a process
Soft Skills Training or Education – Designed to increase
an Employee’s Career Portfolio and enable them to
become more engaged employees.
Daniel Bloom and Associates, Inc.05/06/2009 5
6. Management PerspectiveManagement Perspective
Equally Critical Issue is whether:
“We are conducting this training for “X” amount
of hours per year.
Are we getting our monies worth for the time and
funds invested to produce the training?”
Daniel Bloom and Associates, Inc.05/06/2009 6
7. Premise #4Premise #4
Equally Critical Issue is in which scenario training is
being requested:
“Mr. Manager are you asking for this training
due to a skill problem or a will problem?”
Will problem is a management issue and
carries no ROI
Will problem is NOT a training issue
Daniel Bloom and Associates, Inc.05/06/2009 7
8. Traditional Assessments
Evaluation MethodsEvaluation Methods
Kirkpatrick ModelKirkpatrick Model
Phillips ROI ModelPhillips ROI Model
Total Cost of OwnershipTotal Cost of Ownership
Simple Return on InvestmentSimple Return on Investment
DuPont Equation Attribute ModelDuPont Equation Attribute Model
05/06/2009 Daniel Bloom and Associates, Inc. 8
9. Traditional Assessments
Evaluation MethodsEvaluation Methods
Success Case ModelSuccess Case Model
Kolb’s Learning StylesKolb’s Learning Styles
People Performance Model (Performance vs. Potential)People Performance Model (Performance vs. Potential)
Joahri Window (Known vs. Unknown)Joahri Window (Known vs. Unknown)
Participatory Training Evaluation ModelParticipatory Training Evaluation Model
Solomon Four ModelSolomon Four Model
05/06/2009 Daniel Bloom and Associates, Inc. 9
11. Proposed Question
Using the power of the World Wide Web
via LinkedIn and Yahoo we posed the
following question to approximately
100,000 individuals worldwide
05/06/2009 Daniel Bloom and Associates, Inc. 11
12. Proposed Question
“I am currently working on completing my training for a
six-sigma black belt. Part of that process is to complete a
major project. The project that has been agreed on with
the instructor is one devoted to Training Effectiveness
and how to evaluate it. The end result will be a report and
a dashboard that will graphically portray the various ROI
methods and provide the user with the capability to see
how they are doing across several methods. We intend
to talk to some companies to gather their training data to
be entered into the matrices. We are also considering
including a Training Balanced Scorecard. We are open to
input or for companies that might like to participate.”
05/06/2009 Daniel Bloom and Associates, Inc. 12
13. Data Points
Critical Data Points
Each participating organizations was asked to provide
data points which were central to the ability of the project
to draw its conclusions about the effectiveness of the
offered training:
Sales Revenue
Total Number of Training Hours
Total Number of FTE and PTE’s Trained
Type of Training
Results from Training Evaluations
05/06/2009 Daniel Bloom and Associates, Inc. 13
14. Measure
Critical Data Points
Three Segments of the Ishikawa Cause and Effect
Diagram are of Concern
Process /Methods
Type of Training
Feedback
Materials (Feedback Format)
People
05/06/2009 Daniel Bloom and Associates, Inc. 14
15. Analyze
Data EvaluationData Evaluation
Internet Project Description PostingInternet Project Description Posting
100,000 individuals received the postings100,000 individuals received the postings
25 respondents (.03 percent response)25 respondents (.03 percent response)
General consensus isGeneral consensus is
““We just give the training, we never evaluate it”We just give the training, we never evaluate it”
05/06/2009 Daniel Bloom and Associates, Inc. 15
16. Analyze
ASTD BENCHMARK FORUM 2005 REPORTASTD BENCHMARK FORUM 2005 REPORT
Average Percentage of Training Program Evaluation LevelsAverage Percentage of Training Program Evaluation Levels
91.3% Evaluated Reaction Responses (Happy Smile91.3% Evaluated Reaction Responses (Happy Smile
Sheets)Sheets)
53.9% Evaluated Learning Responses (Did the FTE Learn53.9% Evaluated Learning Responses (Did the FTE Learn
from the training)from the training)
22.9% Evaluated Learning Transfer Levels22.9% Evaluated Learning Transfer Levels
7.6% Evaluated the Impact Levels7.6% Evaluated the Impact Levels
3.2% Evaluated the projected Return on Investment3.2% Evaluated the projected Return on Investment
2.1% Evaluated the actual Return on Investment2.1% Evaluated the actual Return on Investment
05/06/2009 Daniel Bloom and Associates, Inc. 16
17. Training Return on
Investment
Based on Tool implemented by DuPont
Corporation
Assessed on what you want out of the
training for the long run.
05/06/2009 Daniel Bloom and Associates, Inc. 17
18. Improve
Model asks corporations to assess training from a
different perspective
Aligns training to the expected outcomes
Outcomes based on alignment with the corporate
goals
Outcomes based on the anticipated goals of the
training
05/06/2009 Daniel Bloom and Associates, Inc. 18
19. Improve
Model asks corporations to assess training from a
different perspective
Outcomes ranked in the order of importance
Based on a similar assessment used to introduce a
new employee benefit by the DuPont Corporation
Form modified for use with a training program
05/06/2009 Daniel Bloom and Associates, Inc. 19
20. Improve
Outcomes were looked at from some Training Criteria
Areas
Performance Levels
Increased Peer Pressure
Employee Motivation
Performance Assessment
Focused Process Improvement
05/06/2009 Daniel Bloom and Associates, Inc. 20
21. Improve
Outcomes also looked at some additional Training
Criteria Areas
Bottom Line Impact
Corporate Values
Competitive Advantage
Employee Loyalty
Compensation Benefits
05/06/2009 Daniel Bloom and Associates, Inc. 21
22. Improve
Outcomes also looked at some additional Training
Criteria Areas
Building Trust
Relationship Building
Networking Benefits
Follow-up Coaching
05/06/2009 Daniel Bloom and Associates, Inc. 22
23. Improve
DuPont Attribute Training Evaluation Model
Final Step is to evaluate your training from three views
Each is done from a monetary perspective
Views
Corporate Goal for Training
Actual Performance Results
Training Impact ( Corporate Goal – Actual Performance)
Return on Investment Calculation
Training Impact – Training Cost = Return on Investment
05/06/2009 Daniel Bloom and Associates, Inc. 23
25. Control
05/06/2009 Daniel Bloom and Associates, Inc. 25
Goal: To develop a Standard Training
Model which will achieve critical metrics
within the business organization
Purpose of the Training
FTE Involvement at the highest level possible
Management Buy-In to the Training Results
26. Control
05/06/2009 Daniel Bloom and Associates, Inc. 26
Goal: To develop a Standard Training
Model which will achieve critical metrics
within the business organization
Increased Client/FTE Engagement Levels
Direct increase in revenue stream attributed
to the training outcome
Can be shown in a Kaplan-Norton Balanced
Scorecard Model
33. Conclusions
Majority of Corporations globally are not carrying
evaluations out to the point of being able to determine
the economic impact of their training on the bottom line
Majority of corporations globally confine their evaluations
to basically determining the FTE reaction to the
presented training
Corporations need to find a new method to accurately
evaluate the training impact that is faster, better and
cheaper
The DuPont Attribute Training Model can be the solution
05/06/2009 Daniel Bloom and Associates, Inc. 33
35. Have Further Questions :
Daniel Bloom SPHR,SSBB,SCRP
CEO, Managing Consultant
Daniel Bloom & Associates, Inc.
PO Box 1233
Largo, FL 33779
(727) 581-6216
dan@dbaiconsulting.com
http://www.dbaiconsulting.com
35
Find us on
Editor's Notes
Thank you for the opportunity to speak to you this morning. The concepts we are discussing with you this morning are the outputs from our Six Sigma Black Belt project Completed at St Petersburg College during the time period of January to May 2009.
Who Am I? The nature of the role of human capital within your organizations has changed as the nature of the workplace has changed
Every year no matter what the nature of the economic times, American Corporations spend vast amounts of funds on training and development of the employees.
We are operating in a chaotic workplace Four mindsets, one goal Three of the generations have a very specific view of what they want out of the training opportunities
We would all agree that there are two types of training
Management is reviewing our training efforts Concerned with whether or not we are getting our monies worth for the time spent
Need to confirm the reason behind the training Skill versus Will
We are all familiar with the traditional methods for rating our training opportunities This slide and the next list the more common tools we reviewed during the project
The first step in the project was reviewing the root causes for problems in our training efforts Efforts were reviewed from the perspective of How we delivered the training (Process/Method) Course Materials People involved Use of technology and layout of the training space
To determine our scope of the research we utilized social media to a large extent to find resources to assist with the final outputs
We posted a simple question to the social media world
We were seeking very precise data from 5 perspectives
In the view of the Training Root Cause Analysis we looked at four potential root causes of training roadblocks. In the final analysis we were primarily concerned with three out of the four The use of machines to deliver the training did not necessarily have a major impact on the outputs
These 25 respondents represented 6 organizations Represented various industries Represented various sized organizations We are thankful for the assistance of AAA South First American Title KLA-Tencor Lazy Days RV Superstore Marriott Corporation PSCU Financial Services
We also analyzed data from the 2005 ASTD Benchmark Forum on Training Evaluation levels
We looked to see if there was a better way to evaluate training going forward that maybe did not rely on the Kilpatrick or Phillips tools
The DuPont Attribute Model is concerned with different aspects of the training efforts
The model looks at the outcomes of the training programs
We were concerned with 5 Macro views of the training programs
In addition we were concerned 9 other areas coming out of the training programs
We began the process with a look at three segments of outputs The future state of your organization following training The actual performance improvement following training The impact of the training on the organization We also looked at a ROI calculation
It is our belief that determining the effectiveness of your training comes from the influences of the corporate goals, the training results and the training impacts
Our goal was to create a management tool that would develop a series of metrics to determine our initial question
The goal from this tool was to achieve critical metrics regarding your training programs The results were demonstrated in a Kaplan-Norton Balanced Score Card Model
The Balanced Score Card model showed the integration of all the parties to the training efforts within the organization
Let’s begin our look at the Model with the starting point – the organizational training goals The training is evaluated on the basis of the improvement areas from the view of the current state, the future state and the business priority Each is rated on a scale of 1-10 with 10 being a must have
In the second stage of the tool we broke the higher level improvement areas down into finer detail We began with a look at whether the training positively impacted the corporate improvement area The second area was to establish an estimate whether management expected to see the results of the training in 3,6 or 23 months following the conclusion of the training Using the same 1-10 scale the management is asked to determine whether the training area was effective in meeting the corporate goal
We are not saying that ROI is unimportant It is a critical aspect of whether training is worth the expenditure We based our responses on the basis of a 75% confidence level that management was right in their initial assessment
Created a dashboard for showing our results Carried over from the first two worksheets it shows what the corporate training goal was and how close we came to achieving the goal In our example it showed we had a way to go to achieve the outcome we sought, using the data we inputed into the tool
The final worksheet becomes a record of the training offered Provides management with a visual perspective on What course were offered Where the training was given The platform used to deliver the training The number of hours provided The number of FTE’s involved Total instructional hours delivered
Based on our research and analysis utilizing the six sigma methodology we derived four conclusion regarding our question as to whether our training programs were successful and effective. If our training is effective then the cost is in fact justified