Presented at the First Workshop on Intelligent Textbooks (Chicago, IL, US; June 25, 2019)

1. Scaffolded, Scrutable Open Learner
Model (SOLM) as a Foundation for
Personalised e-Textbooks
Judy Kay and Bob Kummerfeld
Human Centred technology
Sydney University
hct:.human-centred technology

2. E-textbook centric view

3. The actual
situation
Canvas
Peerwise
Colab
edTrello Slack ...
Student
Evidence from student
e-text as one source of evidence about learner

4. Case Study: Top level of HILDA model
– Knowledge components for Human-in-the-loop Data Analytics (HILDA)
– Big picture and HILDA links
– Ethics, privacy, anonymisation, provenance, security and access control
– Problem formulation and stakeholder analysis
– Data: Raw and Inferred, cleaned and transformed data
– Model building
– Interfaces and tools for exploration and interpretation
– Interfaces for reporting and decision making
– Managing data and models, literate programming, collaborative HILDA
– Engagement
– Group work
– Self-regulated learning and meta-cognition

5. What might the learner model look like?
Self-monitoring:
• Am I meeting my personal goals?
• Am I meeting the teacher’s goals for a bare pass?
• And for a high grade?
• How do I compare with other students in my class? (needs other students’ data)
Reflection
• What is the trend in my performance this semester?
Planning
• What is the priority for me to do next?

6. Overview example from C course MOOC
Black = “Known”
White = “Unknown”
Cook, R., Kay, J., & Kummerfeld,B. (2015).
MOOClm: user modelling for MOOCs. In
International Conferenceon User Modeling,
Adaptation, and Personalization (pp. 80-91).
Springer..

7. Student progress against teacher’s ideal
model for Week 4
Compared with things the teacher
expects students to know by Week 4:
Black = Alice meets targets
Green = Alice is ahead
Yellow = Alice is behind
White = Alice has not mastered this.
Teacher did not expect students to
know it at Week 4 either.

8. Adding scrutability

9. Forms of evidence
Map learning objectives to:
Learning Objects used: Components of the
course designed to teach eg lectures,
Assessments: Testing achievement in
learning objectives eg. self-test MCQ,
summative quiz, auto-graded assignment …
Data from these provide evidence of learning

10. Viewing Evidence
Description of model component
Student can see exactly what activity
was recorded for them - playing video
indicates “familiarity” level of knowledge
Student can make manual corrections.
eg. Tell Model this item is KNOWN
Also links to the relevant videos & exercises
… to support learning!

11. Image included with the permission of the author from the following paper:
Guerra-Hollstein, J., Barria-Pineda, J., Schunn, C.D., Bull, S., Brusilovsky, P.: Fine-grained open learner models:
complexity versus support. In: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization.
pp. 41–49. ACM (2017)

12. Scaffolding for the student
Overview and details on demand for learning progress monitoring
Selection of standards for comparison
Allowing the student to add their own evidence

13. What scaffolding do teachers need?
• Defining the learner model
• Linking the learner model to the available evidence
• Providing standards for the pl

15. Reflection scaffolding

19. Why Runestone – wishlist of e-textbook
Authoring
• Bibliographic data management
• Good support for co-authorship, versioning
• Supports embedded code that can be run
• Self-test exercises
Students
• All the affordances of paper textbooks
• Single place with all materials and searchable for content with links to external content
• See progress: mark as mastered for text only, completed with tests
• Mark up for self: highlight, bookmarks, add notes
• Communication: annotate problematic parts, connect with other students who are reading current material
Modelling learning
• Supports tagging on content with Knowledge Components and level
• Learning analytics data can be customised

20. Key take-aways
• Independent OLM that gives learner ownership of their own data
• Integrates data from e-book and other learning data
• Scaffolding the student
• Scaffolding the teacher
• Scrutability
• Standards for self-monitoring