Presentation of Suthers, D. D., & Rosen, D. (2011). A unified framework for multi-level analysis of distributed learning Proceedings of the First International Conference on Learning Analytics & Knowledge, Banff, Alberta, February 27-March 1, 2011.
Abstract: Learning and knowledge creation is often distributed across multiple media and sites in networked environments. Traces of such activity may be fragmented across multiple logs and may not match analytic needs. As a result, the coherence of distributed interaction and emergent phenomena are analytically cloaked. Understanding distributed learning and knowledge creation requires multi-level analysis of the situated accomplishments of individuals and small groups and of how this local activity gives rise to larger phenomena in a network. We have developed an abstract transcript representation that provides a unified analytic artifact of distributed activity, and an analytic hierarchy that supports multiple levels of analysis. Log files are abstracted to directed graphs that record observed relationships (contingencies) between events, which may be interpreted as evidence of interaction and other influences between actors. Contingency graphs are further abstracted to twomode directed graphs that record how associations between actors are mediated by digital artifacts and summarize sequential patterns of interaction. Transitive closure of these associograms yields sociograms, to which existing network analytic techniques may be applied, yielding aggregate results that can then be interpreted by reference to the other levels of analysis. We discuss how the analytic hierarchy bridges between levels of analysis and theory.
Suthers & Rosen, Learning Analytics and Knowledge 2011
1. A Unified Framework for Multi-Level
Analysis of Distributed Learning
Dan Suthers
Department of Information and Computer Sciences
and
Communication and Information Sciences Program
Devan Rosen
Department of Speech
University of Hawaii
Funded by NSF VOSS
2. Learning in Social Settings
Multiple theories of how learning takes place in social settings
▪ From social as stimulus to social entity as learning agent
▪ From "networked individualism" to "maintaining a joint
conception of a problem"
▪ From "diffusion of innovations" to "knowledge building"
3. Learning in Social Settings
Multiple theories of how learning takes place in social settings
▪ From social as stimulus to social entity as learning agent
▪ From "networked individualism" to "maintaining a joint
conception of a problem"
▪ From "diffusion of innovations" to "knowledge building"
All involve uptake: when an actor takes (a trace of) another
actor's activity as being relevant in some way for his or her
current activity
▪ See Suthers (ijCSCL 2006) for discussion of learning
epistemologies; and Suthers et al. (ijCSCL 2010) for uptake
4. Learning in Social Settings
Multiple theories of how learning takes place in social settings
▪ From social as stimulus to social entity as learning agent
▪ From "networked individualism" to "maintaining a joint
conception of a problem"
▪ From "diffusion of innovations" to "knowledge building"
All involve uptake: when an actor takes (a trace of) another
actor's activity as being relevant in some way for his or her
current activity
▪ See Suthers (ijCSCL 2006) for discussion of learning
epistemologies; and Suthers et al. (ijCSCL 2010) for uptake
Uptake is evidenced by how individual actions are observably
contingent on the actions of others in their socio-technical
network contexts
5. Analytic Challenges
Fundamental question: how learning takes place through the
interplay between individual and collective agency
▪ Situated accomplishments of individuals and small groups
▪ How these local accomplishments give rise to larger
phenomena in networks
Requires coordinated multi-level analysis
6. Analytic Challenges
Fundamental question: how learning takes place through the
interplay between individual and collective agency
▪ Situated accomplishments of individuals and small groups
▪ How these local accomplishments give rise to larger
phenomena in networks
Requires coordinated multi-level analysis
Activity can be distributed across multiple media and sites
▪ Traces of activity may be fragmented across multiple logs
▪ Logs may record activity in the wrong ontology for analysis
(e.g., media-level events rather than interaction or ties)
Distributed activity may be analytically cloaked
7. Analytic Challenges
Fundamental question: how learning takes place through the
interplay between individual and collective agency
▪ Situated accomplishments of individuals and small groups
▪ How these local accomplishments give rise to larger
phenomena in networks
Requires coordinated multi-level analysis
Activity can be distributed across multiple media and sites
▪ Traces of activity may be fragmented across multiple logs
▪ Logs may record activity in the wrong ontology for analysis
(e.g., media-level events rather than interaction or ties)
Distributed activity may be analytically cloaked
✓ Abstract transcript representation that collects relevant
events into a single analytic artifact
✓ Analytic hierarchy that supports multiple levels of analysis
15. Event Model as Abstract Transcript
▪ Log files no longer needed (except as we discover new
information needs)
Entity-Relations
(Domain Model)
w1
Discussion 1
r1 w2
r2 w3 Containment
P1
r1 m1
r2 P2
m2
r3
m3 P3
Events w4
(Event Model) Threading m4
16. Event Model as Abstract Transcript
▪ Log files no longer needed (except as we discover new
information needs)
Entity-Relations
(Domain Model)
w1
Discussion 1
r1 w2
r2 w3 Containment
P1
r1 m1
r2 P2
m2
r3
m3 P3
Events w4
(Event Model) Threading m4
▪ Straightforward extension to include events from other
media
▪ Sequence of events serves as unified transcript of
distributed events
36. Multi-Media Associations
▪ Characterize pairwise relationships in terms of
distribution across media
▪ Compare roles of various media in supporting
associations (Suthers & Chu, Networked Learning
2010)
w1
m2
P2 P1
m3 m4
P3
m1
w2
37. Ties
▪ Straightforward to collapse into sociogram by transitive
closure or similar computations
P2
P3
Associograms Pairwise Associations (Relationship Model)
(Mediation Model)
m2 m3
m2
P2 P1 P2 P1 P2 Producer/ P3
Dialogue
m3 m4 Consumer
m1 m1
P3
m1
38. Ties
▪ SNA methods can now be applied
Sociogram (Tie Model)
P2 P1
P3
Associograms Pairwise Associations (Relationship Model)
(Mediation Model)
m2 m3
m2
P2 P1 P2 P1 P2 Producer/ P3
Dialogue
m3 m4 Consumer
m1 m1
P3
m1
39. Prior Research
Contingency graphs used for ...
▪ Microanalysis of process through which learners achieved
an insight
▪ Semi-automated analyses of graph manipulations to find
pivotal moments
40. Current Research
Tapped In (SRI International)
▪ Network of educators: professional development and peer
support (longest running educational online community)
▪ 8 years of data; focusing on 2 year peak
▪ 20K educators, 8K user-created spaces; 50 tenant
organizations
▪ Chats, threaded discussions, wikis, resource sharing ...
41. Current Research
Tapped In (SRI International)
▪ Network of educators: professional development and peer
support (longest running educational online community)
▪ 8 years of data; focusing on 2 year peak
▪ 20K educators, 8K user-created spaces; 50 tenant
organizations
▪ Chats, threaded discussions, wikis, resource sharing ...
Current Focus
▪ Identifying where significant activity takes place and
characterizing the nature of that activity (talk tomorrow
afternoon)
▪ Nonlocal consequences of local activities, e.g., trace
contingencies to find whether actors move ideas and other
actors to new settings
42. Advantages
As a data representation
▪ Integration of distributed data: uncloak distributed interaction
▪ Common format for reuse of algorithms
43. Advantages
As a data representation
▪ Integration of distributed data: uncloak distributed interaction
▪ Common format for reuse of algorithms
As an analytic framework
▪ Multi-Level Multi-Theoretical analysis possible
▪ Multiple ontologies allow for mapping between interaction,
mediated affiliation and tie levels of analysis
44. Advantages
As a data representation
▪ Integration of distributed data: uncloak distributed interaction
▪ Common format for reuse of algorithms
As an analytic framework
▪ Multi-Level Multi-Theoretical analysis possible
▪ Multiple ontologies allow for mapping between interaction,
mediated affiliation and tie levels of analysis
Workshop: Connecting Levels of Learning in Networked
Communities
▪ July 5th @ CSCL in Hong Kong
▪ http://www.isls.org/cscl2011/ or http://engaged.hnlc.org/
