Aaron Wolowiec, MSA, CAE, CMP, CTA delivered this presentation during the ISAE 2017 Summer Session in Carbondale, IL on Wednesday, July 19, 2017.

1. Evaluations That Matter
Aaron Wolowiec
Founder | Event Garde
EventGarde.com

2. Name
Organization
One question you have about
Evaluations That Matter

3. 09000 3 87654321520 04 98765432103 9876543210987654321021 987654321098765432100Hours Minutes Seconds
Participant Introductions

4. 09000 2 87654321510 04 98765432103 9876543210987654321021 987654321098765432100Hours Minutes Seconds
Participant Introductions

5. 09000 1 87654321500 04 98765432103 9876543210987654321021 987654321098765432100Hours Minutes Seconds
Participant Introductions

8. Learner Objectives
Ask the right questions.
Determine the optimal platform.
Promote and incentivize participation.
Analyze the data.
Plan better meetings and events.

9. Learner Objectives
Ask the right questions.
Determine the optimal platform.
Promote and incentivize participation.
Analyze the data.
Plan better meetings and events.

10. Learner Objectives
Ask the right questions.
Determine the optimal platform.
Promote and incentivize participation.
Analyze the data.
Plan better meetings and events.

11. Learner Objectives
Ask the right questions.
Determine the optimal platform.
Promote and incentivize participation.
Analyze the data.
Plan better meetings and events.

12. Learner Objectives
Ask the right questions.
Determine the optimal platform.
Promote and incentivize participation.
Analyze the data.
Plan better meetings and events.

13. Learner Objectives
Ask the right questions.
Determine the optimal platform.
Promote and incentivize participation.
Analyze the data.
Plan better meetings and events.

15. 09000 3 87654321520 04 98765432103 9876543210987654321021 987654321098765432100Hours Minutes Seconds
Evaluation Question

16. 09000 2 87654321510 04 98765432103 9876543210987654321021 987654321098765432100Hours Minutes Seconds
Evaluation Question

17. 09000 1 87654321500 04 98765432103 9876543210987654321021 987654321098765432100Hours Minutes Seconds
Evaluation Question

22. Smile sheet defined.
A set of questions provided to learners
after training – or after a portion of
training – asking for learners’
perceptions of the value of the training.

23. Smile sheet problems.
>> Research on traditional smile sheets
finds that they are unrelated to learning
results.
>> Subjective responses are not carefully
analyzed to minimize their most damaging
effects.

24. Smile sheet problems.
>> Likert-like responses hurt learners’
decision-making as they respond.
>> Numerical averaging of smile-sheet data
is inappropriate and conveys results in a
way that hurts stakeholder decision-making.

25. >> We must use good learning
measurement to get valid feedback so
that we can refine our understandings,
improve our learning designs, and live
up to our promise – so that we can
maximize the benefits of learning. <<

26. Smile sheet guidelines.
>> Are based on the learning research.
>> Are actionable.
>> Help learners make better smile-sheet
decisions.
>> Communicate more clearly to
stakeholders.

27. Step 1 >>
Ask the right questions.

28. Example 1.
I learned new knowledge and skills from this
training.
A. Strongly disagree
B. Disagree
C. Slightly disagree
D. Neutral
E. Slightly agree
F. Agree
G. Strongly agree

29. Example 2.
The training will have a significant impact on (check all
that apply):
A. Increasing quality
B. Decreasing costs
C. Decreasing cycle time
D. Increasing productivity
E. Increasing sales
F. Decreasing risk
G. Increasing employee satisfaction
H. Increasing customer satisfaction

30. Example 3.
In regard to the concepts taught in the course, how motivated will you be
to utilize these skills in your work?
A. I will NOT MAKE THIS A PRIOIRTY when I get back to my day-to-day
job.
B. I will make this a PRIORITY – BUT A LOW PRIORITY – when I get
back to my day-to-day job.
C. I will make this a MODERATE PRIORITY when I get back to my day-
to-day job.
D. I will make this a HIGH PRIORITY when I get back to my day-to-day
job.
E. I will make this one of my HIGHEST PRIORITIES when I get back to
my day-to-day job.

31. Example 4.
Using job aids or checklists can be an effective way to ensure you
properly apply skills to your job. Which of the following are true? Circle
as many items as are true.
A. We did NOT RECEIVE ANY WORTHWHILE job aids, checklists or
similar reminders to direct our on-the-job actions
B. We RECEIVED ONE OR MORE WORTHWHILE job aids, checklists
or similar prompting mechanisms
C. During the course, we USED ONE OR MORE WORTHWHILE job
aids, checklists or similar prompting mechanisms in REALISTIC
PRACTICE EXERCISES
D. Between sessions of our course, IN OUR WORKSITES, WE
UTILIZED one or more WORTHWHILE job aids, checklists or similar
prompting mechanisms IN A REAL-WORLD JOB TASK.

33. Smile sheet guidelines.
>> Are based on the learning research.
>> Are actionable.
>> Help learners make better smile-sheet
decisions.
>> Communicate more clearly to
stakeholders.

36. Step 2 >>
Determine the optimal platform.

37. Step 3 >>
Promote and incentivize participation.

38. Step 4 >>
Analyze the data.

39. Step 5 >>
Plan better meetings and events.

40. Focused Conversation

41. Questions you had about
Evaluations That Matter

43. Aaron Wolowiec
Founder | Event Garde
EventGarde.com
Stay In Touch