1. Learning Dispositions and
Transferable Competencies:
Pedagogy, Modelling and Learning Analytics
Simon Buckingham Shum and Ruth Deakin Crick
The Open University & University of Bristol, UK
@sbskmi @ruthdeakincrick / LearningEmergence.net
1

2. The story in 1 slide…
Learning dispositions matter
They can be modelled as “Learning Power”
A platform hosts apps, pools data,
generates real time analytics
and manages permissions
Criteria for validating Learning Analytics
— context
Ongoing/future directions
2

3. learning
dispositions
matter
3

4. Why do dispositions matter?
“Knowledge of methods alone
will not suffice: there must be
the desire, the will, to employ
them. This desire is an affair
of personal disposition.”
John Dewey
Dewey, J. How We Think: A Restatement of the Relation of Reflective Thinking to the
Educative Process. Heath and Co, Boston, 1933
4

5. Why do dispositions matter?
“educational philosophy and
theory face the unfamiliar
and challenging task of
theorising a formative
process which is not guided
form the start by the target
form designed in advance”
Zygmunt Bauman
Zygmunt Bauman, 2001: The Individualised Society, Cambridge, Polity Press.
5

6. Why do dispositions matter?
§ Widening disconnect between what engages many
young people, and their experience of schooling:
§ Canadian Education Association: intellectual
engagement falls during the middle school years and
remains at a low level throughout secondary school
§ English Department for Education: 10% of students
“hate” school, with disproportionate levels amongst less
privileged learners
§ US study across 27 states:
49% students felt bored every day, 17% in every class
6

7. C21 competencies (we’ve all got a list)
Reflects the urgent need to build learners’ capacity to
cope with unprecedented uncertainty and challenge
Sensemaking
Authentic purpose
Learning for teaching as designing
Curriculum as narration rather than script
Learning how to learn
Creativity
Entrepreneurialism
Team work
Problem solving
Citizenship 7

8. What do we mean by “disposition”?
a relatively enduring tendency to behave in a certain way
there are varying conceptions as to how fixed or
malleable dispositions are
Our focus is on malleable dispositions
that are important for developing
intentional learners, and which,
critically, learners can recognise and
develop in themselves
8

9. From transmission to learning design
The Knowledge-Agency Window
Expert
led
enquiry
Student
led-­‐enquiry
co-generation
Teaching
as
Authenticity,
knowledge
agency, learning
identity
and use
design
Repetition,
Pre-scribed
Knowledge
abstraction,
acquisition
Expert
led
teaching
Student
led
revision
Teacher
agency
Student
agency

10. Learning is complex: moving between personal
+ public, via different ways of knowing
Public Practical
Competence
in the world
Knowledge
skills & Propositional
understanding
Learning
power
values attitudes
& dispositions
Presentational
Identity
story, desire,
motivation,
relationships
Personal Experiential
DEAKIN CRICK, R. 2012. Student Engagement: Identity, Learning Power and Enquiry - a complex systems approach. In:
CHRISTENSON, S., RESCHLY, A. & WYLIE, C. (eds.) The Handbook of Research on Student Engagement New York: Springer
HERON, J. & REASON, P. 1997. A Participatory Inquiry Paradigm. Qualitative Inquiry, 3, 274-294.

11. dispositions can be
modelled:
“learning power”
11

12. Competencies and skills
for employability
Personal
Development: Learning Knowledge, Skills and
Values, Attitudes,
Dispositions,
Power Understanding
Identity, Story

13. Where did this construct come from?
3 year project to identify the most important qualities shown by
effective learners, and then devise a valid assessment tool
Experts & Practitioners consulted on overall process
Meta-analysis of the literature (empirical + theoretical)
Expert Workshops (policymakers + scholars)
Leading Practitioner input to survey questions
Survey design iterations and refinement
Factor analysis on survey data (N=2000)
Seven factors identified
13

14. In your experience, what are the qualities shown
by the most effective learners?
Think about the most effective learners you’ve met/
mentored/taught
Not necessarily the highest grade scorers, but the ones
who went on to do well in real world learning
What qualities/dispositions/attitudes did they bring?
Tweet them now on
#lak12 #dispositions
14

15. Learning to Learn: 7 Dimensions of Learning Power
Being Stuck & Static Changing & Learning
Data Accumulation Meaning Making
Passivity Critical Curiosity
Being Rule Bound Creativity
Isolation & Dependence Learning Relationships
Being Robotic Strategic Awareness
Fragility & Dependence Resilience

16. Learning to Learn: 7 Dimensions of Learning Power
Factor analysis of the literature plus expert interviews: identified seven
dimensions of effective learning power , since validated empirically with
learners at many levels. (Deakin Crick, Broadfoot and Claxton, 2004)

17. Learning to Learn: 7 Dimensions of Learning Power
Factor analysis of the literature plus expert interviews: identified seven
dimensions of effective learning power , since validated empirically with
learners at many levels. (Deakin Crick, Broadfoot and Claxton, 2004)

18. Learning
Warehouse:
analytics platform
18

19. Learning Warehouse 2.0
User experience:
Research-validated assessment tools
Researcher interface
Learning Communities
Analytics:
Real time ELLI Analytics reports
Bespoke research reports
Datasets:
>40,000 ELLI profiles
(data from other hosted apps)

20. ELLI: Effective Lifelong Learning Inventory
See the paper for examples of questions loading onto each dimension of Learning Power
20

21. Immediate feedback enabling timely reflection and
interventions, via a mentored discussion
ELLI profile showing pre/post change
21

22. Cohort analytics for
educators and
organizational leaders
22

23. The Learning Warehouse manages permissions
for different stakeholders
§ Learners sign in to complete the right version of the
ELLI questionnaire (e.g. Child or Adult) and receive
their personal ELLI visual analytic
§ Administrators can upload additional learner metadata
or datasets
§ Educators/organisational leaders access individual and
cohort analytics, scaling to the organisation as a whole
if required
§ Researchers can see all of the above, together with
other datasets and institutions (depending on
permissions)
23

24. Additional Analytics services currently provided
through Vital Partnerships (social enterprise spinout from
U. Bristol)
www.vitalpartnerships.com
§ Bespoke organisational analyses to inform leadership
§ a gender cohort in a school; a marketing department in a bank;
underachieving students; or measures of change over time in a school
§ Analysis of data across a cohort of organisations
§ the impact of Learning Futures pedagogies on student engagement in
learning
§ Collaborative research data analysis service
§ researchers who wish to use the instruments in their research
projects or to avail themselves of secondary datasets
§ Secondary analysis of large-scale datasets
§ see paper for details of correlations found between ELLI and other
cross-dataset measures of attainment, teacher attitudes and student
engagement
24

25. validating
learning analytics
context context context
25

26. When we use the same data for different purposes, we
need different ‘truth paradigms’/forms of rationality (Habermas)
Individual
Learners
(Emancipatory)
Teachers Group/Community
Administrators (Hermeneutical)
Leaders
System Wide
Researchers (Empirical/Analytical)
Policymakers
…so Learning Analytics MUST be
interdisciplinary + methodologically plural = painful + challenging!

27. Quality Criteria – how can we be confident in our
measurement model?
Empirical analytical criteria
§ Reliability – are the results repeatable?
§ Validity – does it measure what it says it does?
§ Internal validity – do the research results mean what
they appear to?
§ External validity – can the results be generalised to
other settings (ecological validity) and to other
populations (population validity)?
Deakin Crick, R., Broadfoot, P. and Claxton, G. Developing an Effective Lifelong Learning Inventory: The ELLI
Project. Lifelong Learning Foundation, Bristol, 2002.
Deakin Crick, R. and Yu, G. The Effective Lifelong Learning Inventory (ELLI): is it valid and reliable as a
measurement tool? Education Research, 50, 4, (2008), 387–402.

28. Interpretive/Hermeneutic Criteria
§ Reliability and validity are replaced with
trustworthiness.
§ Extrinsic trustworthiness: credibility,
transferability, contextual transparency,
verifiability.
§ Intrinsic trustworthiness: fairness,
authenticity, internal ethics.
Ren, K. Could Do Better! So Why Not? An Exploration of the Learning Dispositions of Underachieving Students
Doctoral Dissertation, Graduate School of Education, University of Bristol, Bristol, UK, 2010.
Deakin Crick, R., Jelfs, H., Ren, K. and Symonds, J. Learning Futures. Paul Hamlyn Foundation, London, 2010.
Students, W. I. Taronga Zoo Break Out. Singleton High School Ka Wul Centre, Singleton, 2010.

29. Emancipatory Criteria
§ Positionality – researcher declares a standpoint
§ Attention to voice – who speaks for whom?
§ Critical reflexivity – researcher self-awareness
§ Reciprocity – trust and mutuality - dialogue
§ Potential for emancipation and action
§ Undistorted communication
§ Substantive contribution
§ Persuasiveness of discursive critique
§ Participatory ethics
§ Narrative evidence – aesthetic merit / impacts on
reader / communicated ‘lived experience’
Millner, N., Small, T. and Deakin Crick, R. Learning by Accident. Report No. 1,
ViTaL Development and Research Programme, University of Bristol, 2006.
Deakin Crick, R. and Bond, T. 'It's like a Gift: How to get a long life easier': Narratives of Personalisation,
2012. Submitted to Education Other

30. ongoing/future
trajectories
30

31. Learning Warehouse 2.0 (3.0)
Learning App Store
Learning Communities
Collective Intelligence
Learning Analytics
System Simulations
Recommendation Engines
Data Sets + Data Streams

32. EnquiryBlogger: ELLI Wordpress plugins
Ferguson, R., Buckingham Shum, S. and Deakin Crick, R.(2011). EnquiryBlogger: using widgets to
support awareness and reflection in a PLE Setting. In: 1st Workshop on Awareness and Reflection
in Personal Learning Environments. PLE Conference 2011, 11-13 July 2011, Southampton, UK.
Eprint: http://oro.open.ac.uk/30598 32

33. ELLIment: scaling up ELLI mentoring
Thomas Ullman, KMi: http://people.kmi.open.ac.uk/ullmann/projects/ELLIMent 33

34. ELLI-based analytics for a social learning
platform
http://www.open.ac.uk/blogs/SocialLearnResearch

35. Using ELLI dimensions to classify online activity
§ Time-spent in different parts of applications
§ Might repeated attempts to pass an online test load onto
Resilience?
§ Might the sharing of good resources from eclectic
domains load onto Meaning Making?
§ Discourse analytics to detect ‘exploratory talk’ and
argumentation in online spaces
§ Might the presence of questioning and challenging in
discourse load onto Critical Curiosity?
§ Social network analytics
§ Might different SNA patterns in different contexts load
onto Learning Relationships?
35

36. ELLI-based Recommendation Engine?
Could we connect learners to each other, and to
educational resources, based on similar or
complementary strengths on different dimensions?
However, there are many reasons
why a profile may be as it is,
so beware simplistic
interpretation and intervention
— ethical criteria paramount
36

37. To join the global community…
LearningEmergence.net
37