Slides from webinar on 'Rubrics and online assessmentt' given by Eloise Tan in the 2012 Online Assessment and Feedback Module at Dublin City University.

1. /
http://softchalkconnect.com/lesson/files/QA
RNsW98cnVGeh/Rubrics_Lesson_print.ht
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University
ml

2. LI502: Assessment and Feedback in the
Online Environment
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

3. Rubric Design and Deep Learning
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

4. Overview
 Rubrics: definition, components (based on
(Meeusen, 2011) http://softchalkconnect.com/lesson/QARNsW98cnVGeh
 Rubrics in the online environment
 Rubrics for online assessments: online
discussions, wikis, blogs
Class admin
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

5. What is a rubric?
Do you use them for
your assessments?

6. Definitions of rubrics from (Meeusen,
2011) http://softchalkconnect.com/lesson/QARNsW98cnVGeh
Criterion –referenced
tools
 “used to evaluate  “a scoring guide for
student performance evaluating student
against a standard of work”
mastery, not just to
compare students”
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

7. What rubrics can do (Palloff & Pratt,
p.33)
• Way to provide feedback
• Define characteristics of a high quality
assignment
• Help student understand expectations
• Establish range of performance categories
• Provide a concrete way of evaluating their
own performance (self-assessment &
reflection)
• Takes guesswork out of grading for
lecturers
• Aims to achieve reliability, validity, and
transparency within marking process
• Helps students to manage time
• Helps lecturers clarify what is important to
them in the assessment: “What do I want
students to learn from this assessment?”
@t_eloise Dr. Eloise Tan, Learning Innovation
•
Unit, Dublin City University Opens up discussion on marking criteria,
student learning

8. Objectives of rubrics
Validity
(Assesses what
it says it will
assess)
Reliability
Transparency
(Assesses
(Makes clear
consistently
to students
across
the criteria)
submissions)
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

9. What could be some challenges or
unintended consequences of using
rubrics?
 Enter into chat or raise hand to use Talk button
 Chat room comments:
 Lead to mechanical approach to assessments
 Opens up room for conflict relating to
assessments
 Could constrain student creativity
 Marking is subjective, hard to pin down criteria
 Takes a lot of time to design
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

10. Responses to comments
 Lead to mechanical approach to assessments
 We know students already are strategic about their assessments,
rubrics help focus their time/effort.
 Opens up room for conflict relating to assessments
 Rubrics provide a co-constructed tool that can be used by
students to dispute their grades. This could be a positive because
they are relating their performance to pre-determined criteria and
not making subjective arguments about a grade.
 Could constrain student creativity
 Yes, and rubrics could also be seen as a framework through
which students can strategically approach their assessment,
instead of hoping that their strategy works.
 Marking is subjective, hard to pin down criteria
 Yes, that is the point of rubrics. To make marking valid, reliable,
and transparent.
 Takes a lot of time to design
 Definitely! Many tools to help though as we will see…
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

11. Holistic v. Analytic Rubrics
 Holistic rubrics  Analytic rubrics
provide a single score provide specific
based on an overall feedback along
impression of a several dimensions
student’s performance
on a task.
@t_eloise Dr. Eloise Tan, Learning Innovation
http://www.schreyerinstitute.psu.edu/pdf/rubricbasics.p
Unit, Dublin City University
df

12. Example of Holistic v. Analytic
 Webtour
 http://softchalkconnect.com/lesson/files/QARNsW
98cnVGeh/Rubrics_Lesson_print.html
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

13. Remember
What you do not include in
a rubric is just as important
as what you put in.
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

14. Parts of a rubric (Meeusen, 2011)
 Scale: the scale of points to be assigned in
scoring a piece of work on a continuum of quality.
Higher points are typically assigned for the best
work.
 Descriptors: the descriptions for each level of
performance that contain criteria and standards
by which the performance will be judged
 Criteria: Criteria describe the conditions that any
performance must meet to be successful. Criteria
should describe both strengths and errors.
 Standards: standards specify how well criteria
@t_eloise Dr. be met.
must Eloise Tan, Learning Innovation
Unit, Dublin City University

15. Rubrics and online assessment
 Tools to design rubrics:
 Rubistar http://rubistar.4teachers.org (free)
 Rubrics in Moodle 2.2
http://www.youtube.com/watch?v=KXavtUhDINA
 Page compiling online rubric tools:
http://www.teach-nology.com/web_tools/rubrics/
 Open Educational Resources: Foundations for
rubrics can be found online. Example :
http://school.discoveryeducation.com/schrockguid
e/assess.html
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

16. Self, peer assessment and
rubrics
 Could I involve students in setting the rubrics?
 Could students use the rubric to self assess /
peer assess? (Having rubrics readily available
promotes self-assessment and reflection)
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University http://tellio.blogspot.com/

17. Designing rubrics for online
assessments
 Am I clear on what I am assessing in the activity?
What is my learning objective for the
assessment?
 Will technical skills / challenges affect students’
ability to carry out the assessment or meet
successful criteria?
Careful of:
 Tendency to quantify
 Lack of time / preparation/ scaffolding of technical
skills and/or new ways of writing for students
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

18. Evaluating your rubric – some
tips
1. Does it assess what you want it to?
For an online discussion forum – are you assessing for critical thought displayed in the
post? Or are you assessing for number of contributions?
2. Do students have the skills necessary / or the opportunity to gain the
skills necessary to achieve success through your rubric?
Have you provided tutorials for how to post, use a wiki, etc?
3. Have you paid careful attention to what goes into and what stays out
of your rubric?
Is it important to allocate marks for style, presentation, referencing for the assessment?
Might be different for different levels.
4. Do the marks allocated for criteria reasonably correlate to the
amount of time students should spend on that criterion?
5. Have you shared your rubric with students in advance? Or better yet
involved them in the design?
6. Have you included room in your rubric for feedback
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

19. Sites to help you evaluate your rubric
 Useful set of questions from the Office of Provost
at Rensselaer Polytechnic Institute
 http://provost.rpi.edu/node/32
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

20. Webquest and Discussion
 Search for rubrics online for 1) online discussions,
2) wikis , 3) blogs, or 4) any online assessment
you would like to discuss
 Come back and share/discuss/evaluate rubrics
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

21. Online discussion forum rubrics
 Rubric that combines frequency with quality
 http://www.ankeqiang.org/IEP/Online-Eval.pdf
 Rubric that details the different types of posts for
forums (responses, initial post, content
contribution)
 http://www.udel.edu/janet/MARC2006/rubric.html
 Also available in our core texts: Crisp, Palloff and
Pratt
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

22. Wikis (can be used for group
work)
 Rubric for self / peer assessment of team
members in a group
 https://wiki.umn.edu/pub/TeachingWithWikis/Asse
ssingWikis/team_rubric.pdf
 Rubric for the collaboration process
 https://wiki.umn.edu/pub/TeachingWithWikis/Asse
ssingWikis/wiki_rubric.pdf
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

23. Related to last webinar:
 Rubrics as self-
assessment and as
plagiarism prevention
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

24. Current Featured Resource on
Moodle
 http://www.docstoc.co
m/docs/3647772/Tips-
For-Effective-Rubric-
Design-How-to-
design-a-rubric
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University

25. Questions?
@t_eloise Dr. Eloise Tan, Learning Innovation
Unit, Dublin City University