Presented by Veronica Michaelsen and Stephanie Sarandos Vertical- Medical Computer-based testing (CBT) offers a variety of advantages over paper-based testing including improved security and ease of longitudinal tracking of student performance over all phases of the curriculum. However, CBT, like any other innovation, should be implemented thoughtfully. Any successful implementation strategy must account for student acceptance and provide sufficient practice that the extraneous cognitive load is minimized. While extraneous cognitive load should decrease with use, it has also been shown that student opinion toward CBT improves with exposure. Following use of CBT, students perceive the system to be easier to use and they express more confidence in the technology. Still, there is an increased level of anxiety whenever a new technology is introduced. Students’ first experience to CBT should be deliberately designed to address common concerns. Practice quizzes can improve student acceptance of and option toward computer based testing but even these should be developed thoughtfully. In this session, we will briefly describe our approach to the issue of student acceptance and how we utilized student concerns from a pilot to inform our full-scale implementation. Participants will have the opportunity to discuss their strategies and plans with the goal of developing a consensus around the subject of student orientation to maximize acceptance of CBT. At the end of the session, participants will be able to: 1) Participants will become aware of the types of cognitive load that can impact student performance during CBT. 2) Participants will become familiar with the literature on student acceptance of educational technologies. 3) Participants will have the opportunity to share concerns about and strategies for student acceptance of CBT. 4) Participants will take part in building a consensus around best practices for student orientation to CBT.

1. 1ExamSoft Assessment Conference | #ASSESS2015
ExamSoft Assessment Conference
#ASSESS2015

2. 2ExamSoft Assessment Conference | #ASSESS2015
Student Acceptance
 Students have a strong preference for writing on
exams
 Students worry about technology failures
 Students acclimate rapidly to CBT
 Use improves acceptance
 Ease of Use is key factor in determining intent to use

3. 3ExamSoft Assessment Conference | #ASSESS2015
23% Opt out
Hochlehnert et. al. 2011
Students have a strong preference for writing on exams

4. 4ExamSoft Assessment Conference | #ASSESS2015
7% Opt out
Hochlehnert et. al. 2011
Students worry about technology failures

5. 5ExamSoft Assessment Conference | #ASSESS2015
Students acclimate rapidly to CBT
Sheader, 2006
Increased preference with exposure

6. 6ExamSoft Assessment Conference | #ASSESS2015
Use improves acceptance
Deutsch, 2012
Students endorse use after a single exposure

7. 7ExamSoft Assessment Conference | #ASSESS2015
Ease of Use is key factor in determining intent to use
Wijerathne, 2013
70% thought CBT was more efficient
Sheader, 2006
48% thought CBT was faster
37% thought there was no difference

8. 8ExamSoft Assessment Conference | #ASSESS2015
Cognitive Load Theory
 Working Memory - limited
 Long-Term Memory - unlimited

9. 9ExamSoft Assessment Conference | #ASSESS2015
Types of Cognitive Load
Intrinsic
Germane
Extraneous

10. 10ExamSoft Assessment Conference | #ASSESS2015
George Washington University SMHS

11. 11ExamSoft Assessment Conference | #ASSESS2015
Mock Quiz
 Distributed at installation
 Dual purpose:
Familiarize students with platform
Collect Evaluation data
 Sent to 188; downloaded by 102; completed by
98

12. 12ExamSoft Assessment Conference | #ASSESS2015
Pilot Quiz
 10 Question Exam Review
In class
Familiar session format
 Sent to 188; downloaded by 151; completed by
120
 Post-Assessment Evaluation
Distributed via Qualtrics on the same day

13. 13ExamSoft Assessment Conference | #ASSESS2015
Pilot Quiz
0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00%
Technology glitch
Writing on Exam
Crossing out Options
Acclimation
Preference
Exam Navigation
Software Installation
iPad-specific considerations
Eye Strain
Scrolling
Question/Direction-specific
Waiting to Start

14. 14ExamSoft Assessment Conference | #ASSESS2015
Pilot Quiz

15. 15ExamSoft Assessment Conference | #ASSESS2015
Pilot Quiz
0
0.5
1
1.5
2
2.5
3
3.5
4
pre T1 T2 T3 T4
2017
2018

16. 16ExamSoft Assessment Conference | #ASSESS2015
Pilot Quiz
1
1.5
2
2.5
3
3.5
4
T1 T2 T3 T4
Comfort

17. 17ExamSoft Assessment Conference | #ASSESS2015
1
1.5
2
2.5
3
3.5
4
4.5
5
T1 T2 T3 T4
Preference

18. 18ExamSoft Assessment Conference | #ASSESS2015
Conclusions
 It is possible to pre-emptively address some
student concerns
 Students remain sensitive to technology failures
 Students perceive CBT as faster with use
 Students acclimate rapidly to CBT
 As students grow more comfortable, they prefer
CBT