This document has been produced in the context of the Open Educational Resources project “Discovering Collections of Social Science Open Educational Resources, currently undertaken by C-SAP (Higher Education Academy Subject Centre for Sociology, Anthropology and Politics, www.c-sap.bham.ac.uk). The project aims to create a collection of social sciences-based research methods resources.

1. Draft document – looking at the rating and reviewing features of Open Educational
Resources repositories (15 December 2010, version 1)
This document has been produced in the context of the Open Educational Resources project
“Discovering Collections of Social Science Open Educational Resources, currently
undertaken by C-SAP (Higher Education Academy Subject Centre for Sociology,
Anthropology and Politics, www.c-sap.bham.ac.uk). The project aims to create a collection
of social sciences-based research methods resources. For more information about the
project and work-in progress, see also our blog http://csapopencollections.wordpress.com/
Repository Rating feature Review/comment feature
Connexions Yes (on a scale of 1 to 5) Option to email the module author but no
public reviews
Curriki Yes Expert review, where materials are assessed
in three categories: appropriate pedagogy,
content accuracy, and technical
completeness.
Humbox No Yes (see below for a more extensive
overview)
Jorum No No (however, users are encouraged to use
the forum to discuss learning and teaching
resources)
LeMill No Yes (through using the “discussion” button)
MedEd Portal No All submissions are peer-reviewed, provides
information about lessons learnt, limitations
of the resource, intended learner/faculty
audience etc.
MERLOT Yes (on a scale of 1 to 5) Yes (see below for a more extensive
overview of the feature)
OER commons Yes (on a scale of 1 to 5) Yes
OpenLearn Yes (on a scale of 1 to 5) The review option is actually a poll with
three questions:
1. How satisfied are you with the overall
quality of this Unit?
2. How interesting overall did you find this
Unit?
3. How difficult overall did you find the
academic level of this Unit?
There is no option to offer any other type of
feedback
QUEDOC Yes (not formally, however The repository is based around a wiki – users
as the repository is wiki- are welcome to add reviews
based, users can edit the
pages and could add their
This content is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK:
England & Wales http://creativecommons.org/licenses/by-nc-sa/2.0/uk/ C-SAP, 2010
1

2. own rating system)
TEMOA Yes (on a scale of 1 to 5) No
Examples of institutional OER repositories like Open.Michigan, OpenYale, MIT OpenCourseware etc.
are not included in the above table as they as a rule do not offer opportunities for reviews and/or
comments, presumably operating on the assumption that the institutional links of the resource
creators guarantee the quality of the resource.
Humbox
The reviews seem to be an excellent feature of the repository, especially as a significant
number of the resources have been peer-reviewed. At the same time, it would be hard for a
an average user to appreciate the review fucntionalityof the repository – the reviews are
not searcheable, they do not show up on the personal pages of the users and they are not
visible from the front page if someone is just browing the resources.
The reviews cover a number of issues, including:
- a generic description of the resource, including an indication of whether the
resource is part of a larger collection
- any relevant technical issues (for instance, technical requirements, difficulty with
opening some files, broken links etc.)
- an indication of ways in which the resource could be best used – target audience,
usefulness of the resource for particular teaching methods, adaptability across
different disciplines, possibilities for repurposing
- Comments on the quality of the resource and suggestions for areas of improvement,
if relevant
- Comments from reviewers re4gardibng their plans to use/adapt the resource for
their own teaching
- Reactions from the original contributor to the feedback provided by the other
reviewers
MERLOT
The Merlot repository offers the facility for review and comments. While commenting is
open to anyone registered with the repository (for the purposes of registration, only an e-
mail address is needed), only peer reviewers vetted by the institution can submit their
feedback on the resources. To become a peer reviewer for MERLOT, individuals must be an
instructor in an institution of higher education and have relevant teaching experience
including familiarity with learning technology. They need to undergo training (online-based)
to familiarise themselves with MERLOT processes and policies on Peer Review.
This content is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK:
England & Wales http://creativecommons.org/licenses/by-nc-sa/2.0/uk/ C-SAP, 2010
2

3. Peer reviews are structured into three sections: content quality, Potential Effectiveness as a
Teaching Tool and eased of use. The reviewers will first offer a rating for each section (on a
scale from 1 to 5) and then provide written feedback for each section, focusing on the
strengths and weaknesses of the resource. They will also indicate how long it took them to
review the resource.
Content quality: relevance and accuracy of the material, depth of the resource; how
comprehensive the material is, any significant issues with the material (spelling mistakes,
sexist language etc.).
Potential effectiveness as a teaching tool: any required prerequisite skills or level of
knowledge, suggestions for improvement, potential inconsistencies; any issues related to
assignment; the need for any additional material; additional effort needed in terms of
repurposing the resource so that it can be used in class; any assumptions made by the
authors of the resource.
Ease of use for both students and faculty: ease of navigation, use of multimedia; inactive or
broken links; how interactive the resource is; suggestions on how the material could be
better organised, interactivity of the resource.
This content is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK:
England & Wales http://creativecommons.org/licenses/by-nc-sa/2.0/uk/ C-SAP, 2010
3