Opportunities to facilitate learning on the Internet are widely recognized across subject matters, levels of education and situations ranging from extending one’s hobbies to life-long learning relating to workers’ changing roles in the workplace. However, information available in the Internet, even in formal academic courses, is rarely presented using empirically proven findings from the learning sciences. Often, learners are left “on their own” to figure out which tactics work best for them in seeking and understanding information, and studying to learn it. Given that most learners have weak skills in these areas and in self-regulating learning, this sets a stage for major failures in sensemaking and learning that can have dire societal consequences. On the other hand, there are open issues with the existing (a) tools that are typically designed for a hypothetical but factually non-existent “average” user; and (b) methods that are too often based on self-reports (e.g., questionnaires) that are insufficient to advance research on sensemaking and complex learning processes that involve dynamic feedback loops. This talk (i) discusses results of several studies, in which we have addressed the above challenges, and (ii) outlines promising research topics that spans across the three main research cornerstones – computational, socio-cognitive, and user-centered design.

1. https://semtech.athabascau.ca
Tools and Methods to
Enhance Information Seeking,
Sensemaking, and Learning
Dragan Gaševid
@dgasevic

3. Exciting times
MOOC-mania

4. Scientific American, March 13, 2013
http://www.scientificamerican.com/article.cfm?id=massive-open-online-courses-transform-
higher-education-and-science

5. Aren’t they great?!
They offer good resources, but…

6. CHALLENGES

7. What skills to promote?
How about – critical thinking, creativity,
research-intensive learning,
information seeking, sensemaking,
self-directed and self-regulated learning,
…

8. “We teach what we can measure.
If we don't measure what we care about,
it will never be taught.”
Paulo Blikinsein, LAK 2013

9. Three Generation of
Distance Education Pedagogies
Anderson, T. & Dron, J. (2011). Three Generations of Distance Education Pedagogy, International
Review of Research in Open and Distance Learning 12(3), 80-97, http://goo.gl/j3mRF

10. Why does it matter?!
Challenge
Information seeking skills
Judd, T., & Kennedy, G. (2011). Expediency-based practice? Medical students’ reliance on Google and
Wikipedia for biomedical inquiries. British Journal of Educational Technology, 42 (2), 351-360.
doi:10.1111/j.1467-8535.2009.01019.x

11. Why does it matter?!
Challenge
Sensemaking paradox
Butcher, K. R., & Sumner, R. (2011). Self-Directed Learning and the Sensemaking Paradox. Human–
Computer Interaction, 26(1-2), 123-159. doi:10.1080/07370024.2011.556552

12. Why does it matter?!
Challenge
Metacognitive skills
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-Regulated Learning: Beliefs, Techniques, and
Illusions. Annual Review of Psychology, 64, 417-444. doi:10.1146/annurev-psych-113011-143823

13. How can we help?
Information seeking, sensemaking, & learning
Complex processes w/ dynamic feedback loops

14. APPROACH

15. Evidence-based
Discipline
Integration of best research evidence with
practitioner expertise and stakeholder values
The goal made up based on

16. Evidence-based discipline
Practice
Information seeking,
sensemaking, Research
Real
learning
world

17. Evidence-based discipline
Self-reports,
laboratory,
intrusive,
causality,…
Practice
Information seeking,
sensemaking, Research
Real
learning
world

18. Evidence-based discipline
Users “on Intervene/instrument Self-reports,
average”,
laboratory,
GIGO, ..
intrusive,
causality,…
Practice
Information seeking,
sensemaking, Research
Real
learning
world
GIGO – Garbage In Garbage Out

19. Evidence-based discipline
Users “on Intervene/instrument Self-reports,
average”,
laboratory,
GIGO, …
intrusive,
causality,…
Practice
Information seeking,
sensemaking, Research
Real
learning
world!
Collect/Analyze
GIGO – Garbage In Garbage Out

20. Learning Analytics – What?
Measurement, collection,
analysis, and reporting of data
about learners and their contexts

21. Learning Analytics – Why?
Understanding and optimising
learning and the environments
in which learning occurs

22. TOOLS

23. Data Collection
Importance of context
Tool and format independent
Aggregates and integrates

24. Learning Context Ontology:
LOCO
Jovanovic, J., Knight, C., Gasevic, D., Richards, G. (2007). Ontologies for Effective Use of Context in
e-Learning Settings. Educational Technology & Society, 10(3), 47-59.

25. http://intelleo.eu

26. LOCO-Analyst
 OAST and LOCO-Analyst
Learning Environment
Ali, L., Hatala, M. Gaševid, D., Jovanovid, J., (2012). A Qualitative Evaluation of Evolution of a Learning
Analytics Tool," Computers & Education, 58(1), 470-489

27. LOCO-Analyst

28. Learning Analytics
Visual analytics requested
(77.8%)
Ali, L., Hatala, M. Gaševid, D., Jovanovid, J. (2012). A Qualitative Evaluation of Evolution of
a Learning Analytics Tool. Computers & Education, 58(1) 470-489.

29. LOCO-Analyst

30. LOCO-Analyst

31. LOCO-Analyst

32. LOCO-Analyst

33. Student comprehension

34. Student comprehension
Gaševid, D., et al. (2011). An Approach to Folksonomy-based Ontology Maintenance for Learning
Environments. IEEE Transactions on Learning Technologies, 4(4), pp. 301-314.

35. METHODS

36. Learning Analytics Acceptance Model
Ali, L., Asadi, M., Gaševid, D., Jovanovid, J., Hatala, M. (2013). Factors Influencing Beliefs for Adoption of a
Learning Analytics Tool: An Empirical Study," Computers & Education, 62, 130–148.

37. Learning Analytics
What to measure?
We don’t need page access counts!
Wilson, T.D. (1999). Models in information behaviour research.
Journal of Documentation, 55(3), 249 - 270, doi:10.1108/EUM0000000007145

39. Cognitive Presence in Online
Discussions – Moderator Role
Cognitive presence Non-moderator Moderator t(df)
Triggering event 4.29 (4.51) 1.05 (0.23) 4.18(37)*
Control group
Exploration 4.92 (4.12) 5.26 (3.43) -0.46(37)
Integration 1.42 (2.07) 2.68 (1.76) -3.05(37)**
Resolution 0.24 (0.49) 0.74 (1.01) -3.15(37)**
Other 0.89 (1.45) 0.89 (1.03) 0.00(37)
Intervention group
Triggering event 1.43 (1.98) 1.02 (0.15) 1.35(43)
Exploration 3.64 (2.47) 2.82 (2.09) 1.83(43)
Integration 3.86 (3.00) 3.93 (2.32) -0.15(43)
Resolution 0.43 (0.85) 1.07 (1.50) -2.74(43)***
Other 0.86 (1.36) 0.61 (0.90) 1.21(43)
*p < 0.001; ** p < 0.005; *** p < 0.01

40. Cognitive Presence in Online
Discussions – Association w/ Grades
Cognitive presence TMA1 TMA2 TMA3 TMA4 Final
Triggering event -.226 .005 -.046 -.050 -.010
Control group
Exploration -.001 .141 .009 -.037 .048
Integration .128 .060 .034 .043 .113
Resolution .201 .027 -.023 -.054 .074
Other -.028 .078 .113 .106 .154
Triggering event .149 -.077 -.070 .000 .016
Intervention
Exploration .216 .197 .163 .223 .243
group
Integration .156 .396** .417** .338* .454**
Resolution -.041 .060 .154 .083 .129
Other .219 .046 .050 .075 .088
** p < 0.01; * p < 0.05

41. Analytics for Social Learning
Environments
Automated content analysis and role mining
Open learner modeling
Recommendations

42. Self-regulated (Workplace) Learning

43. Trace-based Measurement
Protocol
Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning,
PhD Thesis, Simon Fraser University.

44. Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD
Thesis, Simon Fraser University.

45. Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD
Thesis, Simon Fraser University.

46. Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD
Thesis, Simon Fraser University.

47. Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD
Thesis, Simon Fraser University.

48. Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD
Thesis, Simon Fraser University.

49. Self-regulated (Workplace) Learning

50. Self-regulated (Workplace) Learning

51. http://learningworksforkids.com/EF/metacognition.html

52. Information Interaction
Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced
goal orientation. Learning and Instruction, 22(6), 413–419. doi:10.1016/j.learninstruc.2012.03.004

53. Information Interaction
Achievement goal
orientation (2x2)
Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced
goal orientation. Learning and Instruction, 22(6), 413–419. doi:10.1016/j.learninstruc.2012.03.004

54. Concept and relation filtering
Zouaq, A., Gaševid, D., Hatala, M., "Towards Open Ontology Learning and Filtering,"
Information Systems, Vol. 36, No. 7, 2011, pp. 1064-1081.

55. Concept and relation filtering
Zouaq, A., Gaševid, D., Hatala, M., "Towards Open Ontology Learning and Filtering,"
Information Systems, Vol. 36, No. 7, 2011, pp. 1064-1081.

56. FORWARD-LOOKING

57. Analyzing dynamic processes

58. http://www.processmining.org

59. Enough text obsession!
Forward-looking
George Siemens, LAK 2013

60. CLAS – Collaborative Lecture
Annotation System

61. Multimodal analytics
Forward-looking

62. http://www.solaresearch.org

65. SUMMARY

72. Thank you!