Blackboard rubrics: The good, the bad, and the ugly
1. Blackboard Rubrics:
The good, the bad, and the ugly
Adel Gordon & Pepijn Kalis
University of Northampton | Blackboard
10th April 2013
2. Introductions
• Adel Gordon @adelgordon
– University of Northampton
– Learning Technologist
– Been at Northampton for 13 years, an LT for 6
• Pepijn Kalis pepijn.kalis@blackboard.com
– Blackboard EMEA
– Sr. Specialist
– Working for Blackboard for 5 years
2
3. Rubrics
• What are rubrics?
– Scoring guide
– Authentic assessment tool
– Working guide for students
3
5. Chocolate Chip Cookies
4 3 2 1
Delicious Good O.K. Poor
Chocolate chip in every Chips in about 75% of Chocolate in 50% of Too few or too many
Number of Chips bite bites bites chips
Chewy Chewy in middle, crisp Texture either Texture resembles a
Texture on edges crispy/crunchy or 50% dog biscuit
uncooked
Golden brown Either light from Either dark brown Burned
Color overcooking or light from from overcooking or
being 25% raw light from
undercooking
Home-baked taste Quality store-bought Tasteless Store-bought flavor,
Taste taste preservative
aftertaste – stale,
hard, chalky
Rich, creamy, high-fat Medium fat contents Low-fat contents Nonfat contents
Richness flavor
5
8. Rubrics for teaching
• Clarify learning goals from the offset
• Design materials and activities that address those goals
• Communicate those goals to students
• Guide feedback on students’ progress
• Assess products to degree to which the goals are met
Andrade, 2005
• Enable timely grade allocation whilst justifying them at the
same time
• Feedback > Feed Forward …
8
9. … teaching Rubric design based on
learning outcomes
Make adjustments to
teaching based on PLAN
reflections
Emphasise the
use of rubrics
REFLECT TEACH
Identify common
areas of strengths and
weaknesses
ASSESS
Look for
Score student work
patterns
using rubric
9
Adapted from Stevens & Levi, 2013
10. Rubrics for learning
• Learning
– Face to face
– Online
– Experiential
• Learner
– Surface
– Strategic
– Deep thinker
10
11. In practice
Rubrics set out Rubrics let students Develops/drives a
expectations to aid the know how their grade level of
student to understand was calculated and professionalism and
what they’re being where they could enables me to provide
graded against improve their work more timely feedback
Rubrics give the Makes calculating the
Moderation can take
marker confidence overall grade easier
place immediately and
that you can been when using a
feedback is ready to
more objective then multifaceted approach
be released speedily
subjective to assessment criteria
11
12. Overview of workflow for assessment
process
Provisional
Record
Student First Internal grade & External
grade in
Submission marking moderation feedback to moderation
SIS
students
12
13. … the bad,
• Reporting
– An actual report
However, this report works when working to identify patterns of
strengths and weaknesses in achievement of assessment criteria
across a set of students.
13
14. .. and the ugly
• Intuitiveness
– Importing/exporting
– Saving
• Usability
– Saving
– Integration
14
16. Blackboard vs Turnitin
Feature Blackboard rubric Turnitin rubric
Group submissions
Tutors can add
personalised feedback to
all/any of the marking
criteria
Multiple markers
Export marked work with
rubric summary
University of Manchester
16
17. References
Andrade (2005), Teaching with Rubrics: The Good, the Bad, and the Ugly. College Teaching, [online] Available
at: http://www.uri.edu/assessment/uri/guidance/documents/Andrade_2005_Teachingwithrubrics.pdf
Stevens, D. D. and Levi, A. J., 2013. Introduction to Rubrics. Virginia: Stylus Publishing
University of Manchester, Rubrics – What are they? Why and how should I use them? [online] Available from:
http://www.elearning.eps.manchester.ac.uk/rubrics-what-are-they-and-why-and-how-should-i-use-them/
Utah Education Network – Original Chocolate Chip Cookies. [online] Available at:
http://www.uen.org/Rubric/rubric.cgi?rubric_id=2730
17
18. Contact
• Adel Gordon
– University of Northampton
– Adel.Gordon@northampton.ac.uk
– http://blogs.northampton.ac.uk/learntech
• Pepijn Kalis
– Blackboard
– Pepijn.Kalis@blackboard.com
18
Notas del editor
Where as a lot of student feel that rubrics help with their learning by setting out goals, tutors perspectives of using rubrics tend to lean towards the grading advantages... they serve a good purpose of marking, enabling them to mark consistently and with more objective focus. Andrade reported that in areas where resistance/hesitation to rubrics existed resulted from a lack of understanding of the learning and teaching benefits of a rubric. It was felt that tutors would be more receptive if they understood how rubrics can enhance learning and teaching.
Teaching model adapted slightly from Introduction to Rubrics by Stevens and Levi. Northampton CAIEROS- module development process- teaching teams start from scratch - learning and teaching specialist - learning technologist - librarian - student- support and align assessment with learning outcomes, activities- use a rubric to layout criteria, aligned with LOs* appropriate/understandable languageA few studies that I've read promote the design of rubrics for assessment with the students. Positing that students are more likely to use one if they played a part in designing it than when it see it only as a way for the tutor generate grades. > takes us back to "emphasising the use of rubrics to students"... Jessica Poettecker student union - Develop Partnerships students. Involved them in the development of their learning... Including them in the developments of rubrics can allow for thisI would like to look into the use of a rubric for the module development process for programme/module design and evaluating programmesAt Northampton we use a module development process where teaching teams start from scratch to develop a module. Throughout this process the teaching team, together with a learning and teaching specialist, learning technologist, student and librarian set out learning outcomes, activities etc to support and align assessment with outcomes. Using a rubric to lay out the criteria for the assessments - aligned with learning outcomes - makes everything really simple as long appropriate/understandable language is used in the marking criteria.
Learning:Theme throughout all types of learning is similar – the teaching model is used where refinements are made throughout the overall end to end process learning is improved.TailoredTransparent goalsReducing uncertainty by identifying critical issues in an assignmentEvaluating their own performance in order to get immediate feedbackFocus efforts for subsequent assignments"Instructional illuminators" Popham, 1997Learner:Surface learner – bare minimumStrategic – do what is expected to get best grade they canDeep thinker – everything plus more
The definitions defined in the marking criteria provide detailed explanations of what a student must do to demonstrate a skill, proficiency, or criterion in order to attain a particular level of achievement. Excellent, very good, good satisfactory etc...Scoring, or marking, work based on these definitions makes the students job of interpreting the tutors judgment more easy to understand the targets for their learningWhen marking in teams consistency needs to be monitored. In the Tii case study on marker was marking much more harshly than the others which, because of problems with moderation (later), wasn't picked up until after grades were released.
Practicalities of using the report within the workflow…Outcomes system… ?
It works, but mostly it feels more like a workaround or something that is an after thought that hasn’t quite been integrated properlyNext slide... Demo...
Go into Adel-apr-13... 2 browsers?Demo Kaltura, Turnitin, journal, presentation.
Marking by more than one tutor – Bb - use smart viewsTii – Turnitin groups ExportingBb – No, reporting issueTii – GradeMark paper download includes rubricUse either Blackboard or Turnitin rubrics for:Individual assignmentsSetting an indefinite number of criteria and score rangesSetting marks by points or percentage or no marks (feedback only)Specifying feedback for each possible score/criterion in advanceAutomatically adding up marksManually overriding marks if requiredRe-use with future assessmentsExporting and importing rubrics (to allow sharing with others)*LimitationsBlackboard rubrics cannot be printed by academics once completed – for example – for moderation purposes. (A change request has been submitted to Blackboard for this but this will not be available until at least 2013.)If adding comments to a Blackboard rubric, remember to save your work often – it does not auto-save so will otherwise be lost in the event of a time-out or computer crash.