This technical seminar explains how OERScout uses text mining techniques to autonomously mine domain specific metadata for search purposes, how it utilises a faceted search approach to zero-in on resources and how it incorporates the desirability framework to recommend useful resources for academic purposes. The seminar also gives a technical overview of OER and explores the current OER search dilemma.

1. OERScout Technology Framework
A Novel Approach to Open Educational Resources Search

2. Technical Seminar
19th July 2013
Wawasan Open University and DISTED College
Ishan Abeywardena
MSc, MSc (Brunel), BSc (Bangalore), MIEEE, MBCS, MIET, MTA
Deputy Dean and Senior Lecturer
School of Science and Technology
Wawasan Open University
Penang, Malaysia

3. Acknowledgments
This research project is funded:
• as part of a doctoral research through the Grant (# 102791) generously made by the International
Development Research Centre (IDRC) of Canada through an umbrella study on Openness and Quality in Asian
Distance Education.
• by the Education Assistance Program (EAP) of Wawasan Open University, Malaysia.
Ishan Sudeera Abeywardena acknowledges the support:
• by Tan Sri Dato’ Emeritus Professor Gajaraj Dhanarajan, Dato’ Dr Wong Tat Meng,
Professor Dato’ Dr Ho Sinn Chye and Professor Dr Tham Choy Yoong of Wawasan Open University;
• by Dr Chan Chee Seng and Dr S. Raviraja of University of Malaya;
• by Professor A. Kanwar and Dr. V. Balaji of the Commonwealth of Learning (COL), Vancouver, Canada through
an Executive Secondment (4th – 25th May 2012);
• by the Faculty of Computer Science and Information Technology, University of Malaya, 50603, Kuala Lumpur,
Malaysia where he is currently pursuing his doctoral research in Computer Science;
• by the School of Science and Technology, Wawasan Open University, 54 Jalan Sultan Ahmad Shah, 10050,
Penang, Malaysia where he is currently employed;

4. Seminar Outline
• Introduction to Open Educational Resources;
• Review of the Current OER Search Dilemma;
• Parametrically measuring the Desirability of
OER using D-index;
• Faceted Search of OER using OERScout.

5. A Technical Perspective

6. Definition
Current Status
The “O” in OER
Copyright
Access
Curation
Sustainability
Impact
Future Direction
Abeywardena, I.S. (2012). A report on
the Re-use and Adaptation of Open
Educational Resources (OER): An
Exploration of Technologies
Available. Commonwealth of
Learning.

7. “web-based materials,
offered freely and openly
for use and re-use in
teaching, learning and
research” (Joyce, 2007).
Joyce, A. (2007). OECD Study of OER: Forum Report, OECD. Retrieved December 12, 2011 from
http://www.unesco.org/iiep/virtualuniversity/forumsfiche.php?queryforumspages_id=33.

8. “teaching, learning and research
materials in any medium, digital
or otherwise, that reside in the
public domain or have been
released under an open license
that permits no-cost access, use,
adaptation and redistribution by
others with no or limited
restrictions”
(UNESCO Paris OER Declaration, 2012)
UNESCO. (2012, June 22). 2012 PARIS OER DECLARATION. Retrieved June 13, 2013, from unesco.org:
http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/CI/WPFD2009/English_Declaration.html

9. OER: Just another means of getting there…

11. four ‘R’s model:
 Reuse
 Redistribute
 Revise
 Remix
Hilton, J., Wiley, D., Stein, J., & Johnson, A. (2010). The four R‘s of openness and ALMS Analysis: Frameworks for open educational resources.
Open Learning: The Journal of Open and Distance Learning, 25(1), 37-44.
O

12. Increasing openness of the four ‘R’s: adapted from
(Hilton et al., 2010)

14. http://www.youtube.com/watch?v=8YkbeycRa2A
Lets watch a video on CC

16. Unported
Ported

17. Access (ALMS)
Hilton, J., Wiley, D., Stein, J., & Johnson, A. (2010). The four R‘s of openness and ALMS Analysis: Frameworks for open educational
resources. Open Learning: The Journal of Open and Distance Learning, 25(1), 37-44.
• Access to editing tools
• Level of expertise
required to revise or
remix
• Meaningfully editable
• Source file access

18. Content repositories
Portal repositories
Content and portal repositories
McGreal, R. (2010). Open Educational Resource Repositories: An Analysis. Proceedings: The 3rd Annual Forum on e-Learning
Excellence, 1-3 February 2010, Dubai, UAE.
Curation

19. Sustainability
Downes, S. (2007). Models for Sustainable Open Educational Resources. Interdisciplinary Journal of Knowledge and Learning Objects , 3.
1. Endowment Model
2. Institutional Model
3. Governmental Model
4. Partnerships and
Exchanges Model
5. Membership Model
6. Donations Model
7. Conversion Model
8. Contributor-Pay Model
9. Sponsorship Model

20. basis for training the global masses
increased access
reduced costs
increased equity
Impact

21. Future?

22. Lets watch a video on
mooc
http://www.youtube.com/watch?v=eW3gMGqcZQc

24. The Current OER Search Dilemma

25. • Current State of OER
• The Dilemma
• Some Existing
Solutions
– Pearson’s Project
Blue Sky
– GLOBE
– LRMI
• However…
Abeywardena, I.S., & Chan, C.S.
(2013). Review of the Current OER
Search Dilemma. Proceedings of the
57th World Assembly of International
Council on Education for Teaching
(ICET 2013), Nonthaburi, Thailand.

26. Current State of OER

27. So…how do I find the material I need for my teaching?

29. Google“AdvancedSearch”resultsforOERonChemistry(24thMay2012)

30. Literature
• ...The problem is in finding the resources, and more correctly finding the “right”
resources. Using a regular search engine like Google to find content is not always a
viable option as it will generate too many answers. There is, hence, a need to easily
find relevant content...” (Hatakka, 2009)
• “searching this way (using existing search engines such as Google) might be a long
and painful process as most of the results are not usable for educational purposes”
(Pirkkalainen & Pawlowski, 2010)
• No single search engine is still able to locate resources from all the OER
repositories (West & Victor, 2011)
• One of the major barriers to the use and re-use of OER is the difficulty of finding
quality OER matching a specific context (Dichev & Dicheva, 2012)
• “…the problem with open content is not the lack of available resources on the
Internet but the inability to locate suitable resources for academic use” (Unwin,
2005).

31. Native search mechanisms perhaps?

32. Identify which material to look for (e.g. integration, C++ programming)
Identify the search queries (e.g. “undergraduate mathematics”)
Locate repository (word of mouth, some link somewhere, go to the more popular
repositories)
Run multiple queries to find resources
Read each resource to identify the usefulness (openness, access, relevance)
Identify useful resources
Repeat steps 3-6 on multiple repositories (hundreds to thousands…..)
Native Search in Repositories
Locate repository (word of mouth, some link somewhere, go to the more popular
repositories)
Run multiple queries to find resources
Read each resource to identify the usefulness (openness, access, relevance)
Identify useful resources

33. Frustrated??

34. Some Existing Solutions
• Federated Search: BRENHET2; OpeScout;
Global Learning Object Brokered Exchange
(GLOBE); and Pearson’s Project Blue Sky.
• Semantic Search: OER-CC ontology; the
“Assistant” prototype; the “Folksemantic”
project; and “Agrotags”.

35. Metadata
IEEE-LOM
DCMI

36. Federated Search
Institution
Repos
Institution Institution
Repos Repos Repos Repos Repos
Organisation Consortium
Federation
User
Metadata

37. Apples
Processed Products
Nutrition
Varieties
Juice
Sauce Cider
McIntosh
Pink Lady
Honeycrisp
Disease Prevention
Fiber
Vitamins
Semantic Search

38. Yeah…but which one do I choose?

39. http://www.pearsonlearningsolutions.com/pearson-bluesky/

40. http://www.globe-info.org/

41. http://creativecommons.org/tag/learning-resource-metadata-initiative

42. However…the issue is

44. Desirability of OER using D-index

45. • Usefulness of OER
• The Desirability of OER
• Measuring the
Desirability using D-
index
Abeywardena, I.S., Raviraja, R., & Tham,
C.Y. (2012). Conceptual Framework for
Parametrically Measuring the Desirability
of Open Educational Resources using D-
index. International Review of Research in
Open and Distance Learning, 13(2), 104-
121 (ISI-indexed publication).

46. How useful is a resource really?

47. Measures of usefulness
Openness
Accessibility
Relevance

48. Less useful resources are less desirable for teaching and learning needs….
What is Desirability?

49. i.e. Desirability
D-index = (level of access x level of openness x relevance) / 256

50. Openness
Permission Value
Reuse 1
Redistribute 2
Revise 3
Remix 4
The level of openness based on the four R’s of openness
Mapping the CC licenses to the 4 R’s
Permission Creative Commons (CC) licence Value
Reuse None 1
Redistribute Attribution-NonCommercial-NoDerivs (CC BY-NC-ND)
Attribution-NoDerivs (CC BY-ND)
2
Revise Attribution-NonCommercial-ShareAlike (CC BY-NC-SA)
Attribution-ShareAlike (CC BY-SA)
3
Remix Attribution-NonCommercial (CC BY-NC)
Attribution (CC BY)
4

51. Access
Access to editing
tools
Level of expertise required to
revise or remix
Meaningfully
editable
Source-file access Value
LOW HIGH NO NO 01
LOW HIGH NO YES 02
LOW HIGH YES NO 03
LOW HIGH YES YES 04
LOW LOW NO NO 05
LOW LOW NO YES 06
LOW LOW YES NO 07
LOW LOW YES YES 08
HIGH HIGH NO NO 09
HIGH HIGH NO YES 10
HIGH HIGH YES NO 11
HIGH HIGH YES YES 12
HIGH LOW NO NO 13
HIGH LOW NO YES 14
HIGH LOW YES NO 15
HIGH LOW YES YES 16
ThelevelofaccessbasedontheALMSanalysis

52. Relevance
Search rank Value
Below the top 30 ranks of the search results 1
Within the top 21-30 ranks of the search results 2
Within the top 11-20 ranks of the search results 3
Within the top 10 ranks of the search results 4
The level of relevance based on search rank (Vaughan, 2004)
• Users will only consider the top ten ranked results for a particular search
as the most relevant;
• Users will ignore the results below the top 30 ranks.
Vaughan, L. (2004). New measurements for search engine evaluation proposed and tested. Information Processing and Management
40, 677–691.

53. Calculation
Original search results (OER Commons example)
Calculation of D-index
After application of D-index

54. Sample Search
Search Rank Title CC Lisence File Type
1 18.01 Single Variable Calculus CC BY-NC-SA PDF
2 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text
3 18.01 Single Variable Calculus CC BY-NC-SA PDF
4 18.013A Calculus with Applications CC BY-NC-SA HTML/Text
5 18.02 Multivariable Calculus CC BY-NC-SA PDF
6 Single Variable Calculus CC BY-NC-SA PDF
7 Calculus Online Textbook CC BY-NC-SA PDF
8 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text
9 18.075 Advanced Calculus for Engineers CC BY-NC-SA PDF
10 MATH 140 - Calculus I, Summer 2007 CC BY-NC-SA Protected
Top 10 search results returned by MERLOT for the keyword “calculus”

55. Application of D-index
Rank After
Applying D-
index
Original
Search
Rank Title CC Lisence File Type D-index
1 2 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text 0.75
2 4 18.013A Calculus with Applications CC BY-NC-SA HTML/Text 0.75
3 8 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text 0.75
4 14 Multivariable Calculus CC BY HTML/Text 0.75
5 19
MATH 10250 - Elements of Calculus
I, Fall 2008 CC BY-NC-SA HTML/Text 0.56
6 20 18.022 Calculus CC BY-NC-SA PDF 0.56
7 22 Single-Variable Calculus I CC BY HTML/Text 0.50
8 25 Single-Variable Calculus II CC BY HTML/Text 0.50
9 15 Highlights of Calculus CC BY-NC-SA Video 0.42
10 21 Calculus I CC BY HTML/Text 0.38
Top 10 results when D-index is applied to the results returned by MERLOT

56. Benefits of the D-index
The application of the D-index
greatly improves the effectiveness of
the search with respect to locating
the most useful resources for
use and reuse.

58. OERScout Technology Framework

59. • What is OERScout?
• Why use it?
Abeywardena, I.S., Chan, C.S., &
Tham, C.Y. (2013). OERScout
Technology Framework: A Novel
Approach to Open Educational
Resources Search. International
Review of Research in Open and
Distance Learning, In press.
(ISI-indexed publication)

60. i. Facilitate finding, retrieving
and sharing of OER.
Encourage the
development of user-
friendly tools to locate and
retrieve OER that are
specific and relevant to
particular needs.
(UNESCO Paris OER Declaration, 2012)
The Declaration
UNESCO. (2012). Paris OER Declaration, Retrieved September18, 2012 from
http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/CI/CI/pdf/Events/Paris%20OER%20Declaration_01.pdf

61. Specific & Relevant
• Specific: the suitability of an OER for a particular
teaching need.
For example, an OER on physics from the final
year syllabus of a physics degree would not be
suitable for a high school physics class.
• Relevant: the match between the content of the
OER and the content needed for a particular
teaching need.
For example, physical chemistry is not relevant for
a teaching need in organic chemistry.

62. What is OERScout

63.  READS
 LEARNS
 RECOMMENDS
A Text Mining Algorithm

64. The ‘Bag of Words’

66. Getting Smarter

67. Term Dictionary

68. Making the Connection

69. Keyword-Document Matrix (KDM)

70. KDM

72. Web Server hosting
OERScout KDM
Server Tools creating KDM
OERScout
Web Service
OERScout Client
Deployment

73. The Cluster Map

74. User Interface

75. Advantages of OERScout
• Generates ranked lists of
relevant OER
• Incorporates the
Desirability framework
• Uses faceted search

78. About…
Ishan Abeywardena
Deputy Dean and Senior Lecturer, School of Science and Technology, Wawasan Open University, Penang, Malaysia
• MSc in Wireless Enterprise Business Systems, Brunel University, UK.
• MSc in Engineering Management, Brunel University, UK.
• BSc in Computer Science, Bangalore University, India.
• PhD Candidate in Computer Science, University Malaya, Malaysia. Areas of specialisation: text mining,
metadata, faceted search
Professional Member of
– Institute of Electrical and Electronic Engineers (MIEEE)
– British Computer Society (MBCS)
– Institution of Engineering and Technology (MIET)
– Microsoft Technology Associate (MTA)
Official Profile: http://www.wou.edu.my/IshanAbeywardena.html
Professional Profile: http://www.linkedin.com/in/ishansa
Research Profile: http://www.researchgate.net/profile/Ishan_Abeywardena/
Tech Blog: http://www.ishantalks.com
E-mail: ishansa@wou.edu.my

79. Media Sources
• http://upload.wikimedia.org/wikipedia/commons/thumb/5/58/Multimedia_icon.png/120px-Multimedia_icon.png
• http://upload.wikimedia.org/wikipedia/commons/thumb/2/25/Nuvola_multimedia.png/120px-Nuvola_multimedia.png
• http://upload.wikimedia.org/wikipedia/commons/thumb/b/b5/Open_Clip_Art_Library_Piggy_Bank.svg/120px-
Open_Clip_Art_Library_Piggy_Bank.svg.png
• http://commons.wikimedia.org/wiki/File:Active_learning_-_jigsaw_map_of_Southeast_Asia.jpg
• http://commons.wikimedia.org/wiki/File:%D0%9A%D0%BE%D0%BF%D0%B8%D1%8F_800px-Ua-er9m-558-20070728.jpg
• http://commons.wikimedia.org/wiki/File:Public_Bus,_Cairo_02.JPG
• http://commons.wikimedia.org/wiki/File:Polski_Fiat_126p_in_Krak%C3%B3w_(MWM_2010).jpg
• http://commons.wikimedia.org/wiki/File:PLM_2011_64_Jetalliance_Lotus.jpg
• http://www.youtube.com/watch?v=8YkbeycRa2A
• hydrodictyon.eeb.uconn.edu