Slides for the virtual workshop delivered as part of the Western Conference on Science Education 2013.

1. student
generated
content
for
enhanced
engagement
and
learning
Paul
Denny
paul@cs.auckland.ac.nz
The
University
of
Auckland
The
Western
Conference
on
Science
Educa>on
London,
Ontario,
Canada
|
July
9th
–
11th
2013
Simon
Bates
simon.bates@ubc.ca
The
University
of
Bri>sh
Columbia
Ross
Galloway
ross.galloway@ed.ac.uk
The
University
of
Edinburgh

2. Overview
for
the
session
1)
About
PeerWise
2)
Hands-­‐on
session
3)
Research
highlights
4)
Q&A
student
generated
content
for
enhanced
engagement
and
learning

3. 1)
About
PeerWise
student
generated
content
for
enhanced
engagement
and
learning

4. a
web-­‐based
MCQ
repository
created
by
students

5. InsItuIons
signing
up
per
year:
2009:
22
2010:
66
2011:
204
2012:
266
2013
(Jan-­‐Jun):
214
Growing
content
repository:
Courses:
2,500
Logins/month:
75,000
Ques>ons:
600,000
Answers:
12,000,000

6. Student
ownership
over
learning
resource
Student
familiarity
with
social
soQware
Leveraging
student
energy
and
creaIvity

7. Why
student
authored
ques>ons?
• A
student
contribu>ng
a
new
ques>on
would
develop
the
ques>on
stem:
QuesIon
Student
authored
ques>on
HUBS192,
2010
University
of
Otago

8. Why
student
authored
ques>ons?
AlternaIves
• And
a
set
of
plausible
alterna>ves
• Of
course,
the
correct
answer
must
be
indicated

9. Why
student
authored
ques>ons?
AlternaIves
• And
a
set
of
plausible
alterna>ves
• Of
course,
the
correct
answer
must
be
indicated

10. Why
student
authored
ques>ons?
ExplanaIon
• And
an
explana>on,
in
their
own
words
• Useful
for
students
who
answer
incorrectly

11. “You
don't
really
understand
how
much
or
how
li5le
you
know
about
a
concept
un9l
you
try
to
devise
a
good,
original
ques9on
about
it”
“The
aspect
I
found
truly
useful
was
the
crea9on
of
ques9ons,
which
reinforced
much
of
[my]
understanding
while
also
ac9vely
making
me
clarify
and
solidify
my
thought
processes
(especially
the
explana9on
parts)”
Why
student
authored
ques>ons?
Student
feedback

13. Badges
Points
Leaderboards

14. 2)
Hands-­‐on
session
student
generated
content
for
enhanced
engagement
and
learning
Your
turn:
open
a
new
window
in
your
browser

15. Google:
peerwise
Start
typing:
Bri5sh
Columbia…...
hWp://peerwise.cs.auckland.ac.nz/at/?ubc_ca
Or
go
direct
to
the
URL:

17. Registra>on

18. Step
1
–
choose
a
name
Our
recommendaIon:
please
use
your
first
iniIal
and
your
surname
(e.g.
rgalloway)

19. Step
2
–
enter
a
password

20. Step
3
–
enter
the
Course
ID
7634

21. Step
4
–
enter
your
Iden>fier
Any
3-­‐digit
number
between
100
and
500

22. That’s
it!

23. now
log
in…

24. Choose
the
“WCSE
2013”
course

25. 1)
Unanswered
ques>ons:
“view”
2)
Choose
ques>on
3)
Select
answer
Answering
quesIons

26. 1)
Your
ques>ons:
“view”
2)
Create
new
ques>on
3)
Preview
4)
“Make
changes”
or
“Save
ques>on”
CreaIng
a
quesIon

27. 3)
Research
highlights
student
generated
content
for
enhanced
engagement
and
learning

28. Research
highlights
• Rela>onship
between
ac>vity
and
performance
• The
quality
of
student-­‐authored
ques>ons
• Influencing
student
behaviour
with
rewards

29. Ac>vity
and
learning
• Generally,
students:
– Par>cipate
beyond
minimum
requirements
– Engage
in
community
learning,
correc>ng
errors
– Create
problems,
not
exercises
– Provide
posi>ve
feedback

30. Correla>on
with
end
of
course
outcomes

31. Correla>on
with
end
of
course
outcomes

32. Correla>on
with
end
of
course
outcomes

33. Ques>on
quality
Comprehensive
categorisa>on
of
>50%
of
repository
for
two
successive
academic
years
Principal
measures
to
define
a
‘high
quality
ques>on’
-­‐
cogni>ve
level
of
ques>on
-­‐
explana>on
quality
-­‐
other
binary
criteria

34. Category Description
6 Create (synthesise ideas)
5 Assess
4 Analyse (multi-step)
3 Apply (1-step calcs.)
2 Understand
1 Remember
Cogni>ve
level
of
ques>on

35. 0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
1 2 3 4 5 6
Taxonomic Category
PercentageofSubmittedQuestions
First semester N = 350
Second semester N = 252
Results: Question level Physics 1A / 1B 2011

36. • Badge
/
achievement
system
• Randomised
controlled
trial
(n
>
1000)
Authoring
ques>ons
Evalua>ng
ques>ons
Answering
ques>ons
Other
Do
rewards
work?

37. No
difference
between
groups
with
respect
to
answer
correctness
+
13%
+
22%
DistribuIon
of
days
of
acIvity
differs
significantly
between
groups:
p
<
0.001
DistribuIon
of
answers
per
student
differs
significantly
between
groups:
p
<
0.001

38. Implica>ons
• Badges
can
have
a
measurable
posi>ve
impact
on
some
student
ac>ons
• No
nega>ve
effects
on
ac>vity
or
percep>ons:
a
low-­‐risk
proposi>on

39. Please
join
us!
hWp://www.peerwise-­‐community.org/
These
slides
hWp://bit.ly/WCSE_PW

40. 4)
Q
&
A
student
generated
content
for
enhanced
engagement
and
learning

41. Publica>ons
“Student-­‐generated
content:
using
PeerWise
to
enhance
engagement
and
outcomes
in
introductory
physics
courses”
(S.
P.
Bates,
R.
K.
Galloway,
K.
L.
McBride),
In
2011
Physics
Educa>on
Research
Conference
(NS
Rebello,
PV
Engelhardt,
C
Singh,
eds.),
Amer.
Inst.
Physics,
volume
1413,
2012.
(Physics
Educa>on
Research
Conference,
Omaha,
NE,
2011)
“Student-­‐generated
content:
Enhancing
learning
through
sharing
mulIple-­‐choice
quesIons”
(J.
A.
Hardy,
S.
P.
Bates,
M.
M.
Casey,
K.
W.
Galloway,
R.
K.
Galloway,
A.
E.
Kay,
P.
Kirsop,
&
H.
A.
McQueen),
submiqed
to
Interna>onal
Journal
of
Science
Educa>on
“Assessing
the
quality
of
a
student-­‐generated
quesIon
repository”
(S.
P.
Bates,
R.
K.
Galloway,
J.
Riise,
and
D.
Homer)
in
review
with
Physical
Review
Special
Topics:PER
“The
Effect
of
Virtual
Achievements
on
Student
Engagement”
(P.
Denny),
In
Proceedings
of
the
SIGCHI
Conference
on
Human
Factors
in
Compu>ng
Systems
(CHI
'13),
pp
763-­‐772.
ACM,
2013.