This document discusses evaluating HRD (human resource development) interventions and programs. It covers major areas of HRD like training, organizational development, and career development. Strategic HRD involves needs assessment, design, implementation, and evaluation. Kirkpatrick's model is described for evaluating training at four levels: reaction, learning, behavior, and results. Methods of evaluation include questionnaires, tests, observations, interviews and analyzing performance/economic data. Ensuring transfer of training to job performance is key to an effective evaluation.

1. Evaluating HRD
Interventions
-Please sit with your group members-

2. Chapter 1 - Review
HRD is a set of systematic and planned activities designed by an
organization to provide its members with the opportunities to learn necessary
skills to meet current and future job demands
Major Areas:
 Training and development (T&D)
 Organizational development (OD)
 Career development (CD)
Strategic HRD involves:
• Needs Assessment
• Design
• Implementation
• Evaluation
2
Allen Grabo – VU University Amsterdam

3. Chapter 1 – Example Question
What capabilities are required of a successful Human
Resource Manager in the modern world?
• Strategic consultant:
• Designer and inventor:
• Change mediator:
• Career advisor:
• Researcher:
• Facilitator: A HRD manager facilitates structured learning
experience for the employees to enhance productivity.
3
Allen Grabo – VU University Amsterdam

4. Chapter 1 - Review
What capabilities are required of a successful Human Resource Manager in the modern
world?
• Strategic consultant: The main role of a HRD manager is to guide a company in formulating and
planning its strategies in order to achieve organizational goals. HRD managers carry out necessary
training programs to improve performance goals of the team.
• Designer and inventor: A HRD manager performs various functions of planning to designing
strategies and executing them.
• Change mediator: A HRD manager acts as a change agent at the time of mergers, acquisitions or
takeovers as well as at the time of installing new technologies. People often resist change and
reluctant to accept it. HRD managers help employees to cope up with the change.
• Career advisor: A HRD manager assists the employees to improve their skills and competencies in
order to achieve their career goals. Various learning sessions, workshops, seminars and
conferences are arranged by the HRD managers in this purpose.
• Researcher: Acts as a researcher by assessing HRD practices and procedures with the help of
statistical data in order to determine the effectiveness of the employees and communicate the results
to the organization.
• Facilitator: A HRD manager facilitates structured learning experience for the employees to enhance
productivity.
4
Allen Grabo – VU University Amsterdam

5. Chapter 2 - Review
What factors influence how people behave in the
workplace?
• External
• Work environment
• Coworkers
• Performance outcomes
• Internal
• Motivation
• Knowledge, skills and abilities
5
Allen Grabo – VU University Amsterdam

6. Chapter 2 – Example Question
Describe at least 3 external factors that influence how
people behave in the workplace.
6
Allen Grabo – VU University Amsterdam

7. Chapter 2 – Example Answers
Describe at least 3 external factors that influence how
people behave in the workplace.
• Economic factor:
• Technological factor:
• Competition in Industry:
7
Allen Grabo – VU University Amsterdam

8. Chapter 2 – Example Answers
Describe at least 3 external factors that influence how
people behave in the workplace.
Economic factor:
Technological factor:
Competition in Industry:
8
Allen Grabo – VU University Amsterdam

9. Chapter 2 – Example Answers
Describe at least 3 external factors that influence how people
behave in the workplace.
• Economic factor: If the economy is booming and employees have plenty
of jobs, then the behavior at the work would be overall happier. On the
contrary, there will be fear and anxiousness among employees if the
economy is facing recession. That anxiety leads to lower productivity and
hamper employee performance.
• Technological factor: Day by day advancement in the technology made
employees feel regressive and less motivated. Employees resist changing
and avoiding learning new things. To avoid this situation, a manager should
impart timely updated training to employees.
• Competition in Industry: Increased competition within an industry may
raise questions on the survival of the company which leads to frustration
and less motivated employees. However, employees feel motivated in a
stable industry where it is easy to attract customers.
9
Allen Grabo – VU University Amsterdam

10. Chapter 3 - Review
Learning: A change in behavior, cognition, or affect that occurs as a result of
one’s interaction with the environment
Training creates an environment that maximizes
learning:
• Trainee characteristics
• Kolb’s Learning Styles are most widely used
• Motivation is key
• Training design
• How to implement training (active vs passive, whole vs part)
• Over-learning
• Feedback
• Transfer of training
• Aim for fidelity – either psychological or physical
• Give people a chance to use the skills in practice
10
Allen Grabo – VU University Amsterdam

11. Chapter 3 – Example Question
From the strategies discussed in this chapter, which two are
most commonly used by your team? Can you give an
example of how it helped you learn more effectively?
• Rehearsal
• Elaboration
• Organizational
• Comprehension
• Affective
11
Allen Grabo – VU University Amsterdam

12. Chapter 3 – Example Answers
Rehearsal: Repeating items and copying texts or articles.
• Students memorizing topics during examinations are a classic
example.
• Helps to get accustomed to new information. Helps with retention
and understanding
• Has global benefits
• Allows for the retention of large amounts of information
Elaboration: Visualizing an image, writing down and summarizing
• When memorization is difficult, keywords or mnemonics can help
• Breaks down complex topics
• Focus on the conceptual rather than rote repetition
12
Allen Grabo – VU University Amsterdam

13. Break Time
See you in 15 minutes.
13
Allen Grabo – VU University Amsterdam

14. Evaluating HRD Programs
Recall that the strategy for effective interventions
consists of four parts:
• Needs Assessment
• Design
• Implementation
• Evaluation
Assume you have done all this successfully.
14
Allen Grabo – VU University Amsterdam

15. Now what?
15
Allen Grabo – VU University Amsterdam
Assessment
Assessneeds
Design Implementation Evaluation
Prioritize
needs
Define
objectives
Select
evaluation
criteria
Develop
lessonplan
Develop/acquire
materials
Select
trainer/leader
Select methods
andtechniques
Schedulethe
program/intervention
Determine
evaluation
design
Conduct
evaluation
of program
or intervention
Deliver the
HRDprogram
or intervention
Interpret
results

16. Evaluation - Questions
The systematic collection of descriptive and judgmental information
necessary to make effective training decisions related to the selection,
adoption, value, and modification of various instructional activities.
• How do you evaluate training and HRD?
• What measures can be used to evaluate training?
• Is there one best way to evaluate training?
• How can the value of HRD be expressed in terms of
costs and benefits, or Euros?
16
Allen Grabo – VU University Amsterdam

17. Goals of Evaluation
• Determine whether a program is accomplishing its
objectives
• Identify the strengths and weaknesses of HRD programs
• Determine the cost-benefit ratio of an HRD program
• Decide who should participate in future HRD programs
• Identify which participants benefited the most or least from
the program
• Gather data to assist in marketing future programs
• Establish a database to assist management in making
decisions
17
Allen Grabo – VU University Amsterdam

18. Requirements in HRD research
Pretest and posttest:
• Allows the trainer to see what has changed after the training
Control Group:
• Group of employees similar to those who receive training
• Don’t receive training at the same time as those who are trained
• Receive the same evaluation measures as the group that is trained
Ideal scenario:
• Training group and the control group have similar scores before training
• Scores for the training group increase after training, while those of the control
group remain constant.
• This provides fairly strong evidence that the training (and not some other factor)
was responsible for the changes on the outcome measures
18
Allen Grabo – VU University Amsterdam

19. Ethical Issues
Confidentiality
Informed consent
Control group assignment
• Use a lottery to assign to control groups
• Assure training will be provided to control group later as
well if found to be effective
Pressure to produce positive results
19
Allen Grabo – VU University Amsterdam

20. Major problems in evaluation
Mono-method bias:
• If both reports in a before-and-after evaluation come from the
same person at the same time (say, after training), conclusions
may be questionable
• Respondents may be more concerned about being consistent in
their answers than about providing accurate responses
Social desirability bias:
• Respondents may report what they think the researcher (or
boss) wants to hear rather than the truth
• Employees may be fearful or embarrassed to admit that they
learned nothing in a training program
20
Allen Grabo – VU University Amsterdam

21. Methods
21
Method Advantages Disadvantages
Interview Flexible
Opportunity for clarification
Depth possible
Personal contact
High reactive effects
High cost
Face-to-face threat potential
Labor intensive
Trained observers needed
Questionnaire Low cost
Honesty increased if anonymous
Anonymity possible
Respondent sets pace
Variety of options
Possible inaccurate data
On-job responding conditions not
controlled
Respondents set varying paces
Return rate beyond control
Direct
Observation
Non-threatening
Excellent way to measure
behavior change
Possibly disruptive
Reactive effect possible
May be unreliable
Trained observers needed

22. Methods
22
Method Advantages Disadvantages
Written Test Low purchase cost
Readily scored
Quickly processed
Easily administered
Wide sampling possible
May be threatening
Possible low relation to job performance
Reliance on norms may distort individual
performance
Possible cultural bias
Simulation/
Performance
Test
Reliable
Objective
Close relation to job
performance
Time consuming
Simulation often difficult
High development cost
Archival
Performance
Data
Reliable
Objective
Easy to review
Minimal reactive effects
Criteria for keeping or discarding records
Info system discrepancies
Indirect
Needs to be converted to usable form
May be expensive to collect

23. Three types of data :
Individual performance
System-wide performance
Economic Data
23
Allen Grabo – VU University Amsterdam

24. Individual Performance
• Test scores, number of units produced
• Timeliness, quality of performance
• Attendance, Attitudes
24
Allen Grabo – VU University Amsterdam

25. System-wide performance
• Productivity
• Rework / Scrap
• Customer and client satisfaction
• Timeliness
25
Allen Grabo – VU University Amsterdam

26. Economic Data
• Profits
• Product liability
• Avoidance of penalties
• such as fines for noncompliance with laws and regulations
• Market share
26
Allen Grabo – VU University Amsterdam

27. Effectiveness of an HRD program
Successful outcomes:
• Increased productivity
• Timeliness
• Cost savings
ROI = Results / Training Costs
• If ROI < 1, training costs more than benefits accrue
• If ROI > 1 benefits accrue
• The greater the ratio, the higher the benefit for the organization
27
Allen Grabo – VU University Amsterdam

28. Kirkpatrick’s Method
Reaction
• Did trainees like program?
• Did trainees think it valuable?
Learning
• Did they learn what the training objectives said they
should learn?
Job Behavior
• Did they use learning back on job?
Results
• Has HRD improved organization’s effectiveness?
28
Allen Grabo – VU University Amsterdam

29. Kirkpatrick’s Method
• Most organizations do not collect information on all four types of
outcomes
• About one-third of organizations use Kirkpatrick’s model
• Some feel it only measures what happens immediately after
training
• Others feel it is more of a taxonomy of outcomes, rather than a
framework for evaluating trainings
29
Allen Grabo – VU University Amsterdam

30. Evaluating Training Transfer
• Employees can gain new knowledge and skills through training
programs, but learning alone does not mean that a training is
effective
• Positive transfer of training = extent to which the learning that
results from training transfers to the job and leads to relevant
changes in work performance (Goldstein & Ford, 2002)
• The “transfer problem”: Estimates suggest that only 10 % of
training expenditures transfer to the job (for an overview, see
Grossman & Salas, 2011)
30
Allen Grabo – VU University Amsterdam

31. Evaluating Training Transfer
Near Transfer
• The ability to directly apply on the job what has been learned in
training, with little adjustment or modification
Far Transfer
• Expanding on what has been learned in new and creative ways
31
Allen Grabo – VU University Amsterdam

32. Baldwin & Ford’s Transfer of Training
32
Ability
Personality
Motivation
Trainee
characteristics
1
2
3
4
5
6
Training Inputs
Conditions
of TransferTraining Outputs
Learning
and
retention
Support
Opportunity
to use
Work
environment
Principles of
learning
Sequencing
Training content
Training design
Generalization
and
maintenance

33. Increasing Transfer
• Develop (and follow) clearly stated learning objectives for the
training
• Maximize the similarity between the training situation and the
job situation
• Provide ample opportunity during training to practice the task
• Identify and label important features of a task
• Make sure trainees understand general principles
33
Allen Grabo – VU University Amsterdam

34. Increasing Transfer
• Use a variety of situations and examples, including both positive
and negative models of the intended behavior
• Provide support back in the work environment, including clear
goals, checklists, measurement, feedback, and rewards for
using the new behaviors on the job
• Provide ample opportunity to perform what is learned back on
the job:
• The #1 reason cited for low transfer (listed by over 64 percent of trainees)
is “lack of opportunity to apply on the job.”
34
Allen Grabo – VU University Amsterdam

35. Issues with Evaluation
How often are HRD programs actually evaluated?
• For most company-sponsored training - at the end
Less than half of executive MBA programs are evaluated properly
• “If it’s Harvard, it must be good!”
Most common measures focus on participant reactions
• Not always useful
• Need to compare performance before and after training
Conducting an evaluation is not easy
• Many external factors can affect whether employee performance
improves
• Makes it difficult to evaluate the impact of just the training
• HRD managers can be afraid of criticism and program cuts
35
Allen Grabo – VU University Amsterdam

36. Summary
HRD does not take place in a vacuum
Research shows that HRD impacts:
• Financial performance
• Turnover
• Absenteeism
• Organizational learning
Insights from training transfer research show:
• HRD interventions matter
• But, training alone rarely works
36
Allen Grabo – VU University Amsterdam