The ways in which people seek and process information are fundamentally epistemic in nature. Existing epistemic cognition research has tended towards characterizing this fundamental relationship as cognitive or belief-based in nature. This paper builds on recent calls for a shift towards activity-oriented perspectives on epistemic cognition and proposes a new theory of ‘epistemic commitments’. An additional contribution of this paper comes from an analytic approach to this recast construct of epistemic commitments through the use of Epistemic Network Analysis (ENA) to explore connections between particular modes of epistemic commitment. Illustrative examples are drawn from existing research data on children’s epistemic talk when engaged in collaborative information seeking tasks. A brief description of earlier analysis of this data is given alongside a newly conducted ENA to demonstrate the potential for such an approach. Paper at: http://oro.open.ac.uk/39254/

1. Analytics insights into Epistemic
Commitments
Simon Knight
http://sjgknight.com
@sjgknight
These slides are available at:
http://www.slideshare.net/sjgknight/
And the video at:
https://www.youtube.com/user/sjgknight

2. Presentation
• Background/motivation
• Epistemic commitments as behavioural
trace
• Epistemic Network Analysis as analytic
tool
• Pilot analysis on existing data

3. “epistemological beliefs are a lens for a learner's views
on what is to be learnt” (Bromme, 2009)
Certainty, simplicity, source, justification – for knowing
The Lens of Epistemic Beliefs
Commitments

4. “exploring students’ thought processes during online
searching allows examination of personal epistemology
not as a decontextualized set of beliefs, but as an
activated, situated aspect of cognition that influences the
knowledge construction process” (Hofer, 2004, p. 43).
The Lens of Epistemic Beliefs
Commitments

5. • What do students do, what do they make?
• Analytics give unprecedented(?) access
• Marks a shift, from standardised assessments
to knowledge in action
• Psychometrics to situations
Performance Assessment

6. Explicitly or implicitly learners:
• select sources of information
• have standards by which to judge credibility
• collate information in meaningful ways
• build arguments and make claims
• decide when to start and stop looking for
information
• decide (implicitly or explicitly) the breadth and
depth of information required
Examples of epistemic behaviour

7. Epistemic commitments
Operationalisation centered on
connections between:
• Source selection
• Links between concepts
• Use of information to state
claims, or reason points
• Tokens of information
Within a search session
By Pko (Own work) [Public domain], via Wikimedia Commons
https://commons.wikimedia.org/wiki/File%3AAdditive_color_mixing_simulated.png

8. Epistemic commitments
BBC
• “that’s a good
site”
Answe
rs.com
• “let’s see
what another
site says”
Nobelp
rize.or
g
It’s
importa
nt
because
• It tells us about
her
• It answers the
question
She won
2 prizes

9. Epistemic commitments
BBC
• “that’s a good
site”
Answe
rs.com
• “let’s see
what another
site says”
Nobelp
rize.or
g
It’s
importa
nt
because
• It tells us about
her
• It answers the
question
She won
2 prizes

10. Epistemic commitments
BBC
• “that’s a good
site”
Answe
rs.com
• “let’s see
what another
site says”
Nobelp
rize.or
g
It’s
importa
nt
because
• It tells us about
her
• It answers the
question
She won
2 prizes

11. Epistemic Frames for Epistemic
Commitments

12. © David Shaffer (2005,
2008)
Epistemic Frames
Discourse Oriented. Connections between
(Community of Practice) elements usually
described as:
Skills, knowledge, values, identities, and
epistemological rules.

13. Epistemic Frames and Epistemic
Commitments
we should try
looking on
Wikipedia for
that

19. Use/justificationConnectionSourcingSearchClaims

20. Use/justificationConnectionSourcingSearchClaims

21. Epistemic Network Analysis
(ENA)

22. ©2005, 2008 David
Williamson Shaffer
A
B
D
F
C

23. Use/justificationConnectionSourcingSearchClaims

24. Use/justificationConnectionSourcingSearchClaims
Model frames – nodes are frame
elements (keywords/codes),
connections represent co-occurrence
Novice v Expert network diagrams

29. Characterising groups
• For any stanza, code co-occurrence of frame
elements coded ‘1’
• Singular Value Decomposition (like Principal
Component Analysis) run across stanzas
• Creates a space to explore distances between
groups of networks

31. Player 1: Good Morning
Player 2: Hello everyone, is this
chat with our new
group now?
Player 1: I think so
Player 3: Probably
Player 1: Good morning new
team. I excited to work
with with everyone.
Player 4: hey
Player 1: We are meeting and
currently discussing
our various materials.

32. Player 1: Good Morning
Player 2: Hello everyone, is this
chat with our new
group now?
Player 1: I think so
Player 3: Probably
Player 1: Good morning new
team. I excited to work
with with everyone.
Player 4: hey
Player 1: We are meeting and
currently discussing
our various materials.

34. Player 1: We ranked the different
materials based on the
device specifications and
our experimental data.
Player 2: And we just need to
decide on the 5th team
experimental device.
Player 3: We looked at… how the
cost changed with each
matieral while controling
everything else
Player 4: hey team, I am going to
post the rankings of the
membranes as a public
document so everyone
can see it.

38. Product

39. Product
Professionalism Design
Materials

40. Product
Professionalism Design
Materials

41. Product
Professionalism Design
Materials

42. Product
Professionalism Design
Materials

43. Professionalism
Product
Design
Materials

44. Data used for this paper

45. The Data
• Previously analyzed
dataset
• Applied ENA for
epistemic commitments

46. The Collaboration
Pair and two trios of
female 11 year old English
secondary school students
Three reasons:
– We need the data
– Suggestion it is related to
better seeking
– Suggestion certain modes
of dialogue (exploratory)
may be epistemic in
nature

47. The Analysis
• 1 hour of assigned tasks on ‘role models’
• Closed (“How many women have won the
Nobel Prize?”) & more open (“Why do
some people think Nelson Mandela is a
good role model?”).
• Asked to ‘justify’ choices & state sources
• Analysis for epistemic & exploratory
dialogue w/reference to screencast

48. Group differences
Group 1 – Quite successful, characterised:
“it’s got all the important information” “it’s
a good site” (quality)
Group 2 – Quite successful, characterised:
“I didn’t know that” (novelty), authority
Group 3 – Least successful, characterised:
“There’s loads there” (quantity)

49. Applying ENA to prior data

50. Code Definition
Source – Authority Make reference to authorities
when selecting information
Source – Corroboration Make reference to repetition of
information when selecting
information
Justification – Matching Little attempt to sensemake in
question; targeted matching of
source information to questions.
Justification –Understanding Attempt to make sense of
information, use more
accountable/exploratory dialogue
key phrases
Simplicity – Simple Make few connections between
information, look for individual
tokens of knowledge
Simplicity – Complex Make connections between tokens
of information, possibly across
questions, contextualize
information (e.g. account for
temporal aspect)
Search Referring to search or webpage
specific aspects of the task
General References to general knowledge
required
References to specific pieces of

51. ENA
Group 1
Group 2
Group 3
(nodes have been combined
to aid interpretation)
67% - x axis accounts, groups with
low values making stronger
connections with Simplicity Complex,
and groups with high x values
making stronger connections with
Simplicity Simple and Source
Corroboration.
32% - y axis largely distinguishes
between Justification Match, and
Source Authority with the use of
question specific terminology.

52. ENA
Group 1
Group 2
Group 3
(nodes have been combined
to aid interpretation)
y, representing stronger connections to matching, connecting to: understanding,
making of Question General and Specific claims, and Simplicity Complex.
X, focused on task requirements & linking information to these
“we have to say why it matters, so [fact] is important because…”
Group 1

53. ENA
Group 1
Group 2
Group 3
(nodes have been combined
to aid interpretation)
y, representing greater use of Authority – making the same
connections otherwise (the cluster in the middle).
X, take info from authorities relevant to them without link to q.s
“the answer’s [fact], because it’s a good site”.
Group 2

54. ENA
Group 1
Group 2
Group 3
(nodes have been combined
to aid interpretation)
X, perspective that simple knowledge obtained through
corroboration is used to match general aims (and justified as such)
although not actually targeting question specific knowledge claims
Group 3

55. Epistemic Commitment Conclusions
• Epistemic commitments are seen in
collaborative web search
• Connections can be seen in discourse and
interactions with sources of information

56. ENA Conclusions
• ENA may provide methodological-conceptual
resource for analysis
• Scope for expert v novice comparison
• Manual analysis comparison

57. Pre-empting a criticism
• Circularity?
• Preliminary validation that ENA offers a
representational tool for scalable
interpretation of epistemic commitment

58. Potential approach
Hypotheses:
– Analysis of sets of connections will yield more
insight than particular modes of working (e.g.
looking for ‘authority’)
– E.g. Connection of sourcing-
authority/corroboration to ‘understanding’ will
give greater insight

59. Thank you
Simon Knight
http://sjgknight.com
@sjgknight
Knight, S., Arastoopour, G., Williamson Shaffer, D., Buckingham
Shum, S., & Littleton, K. (2014). Epistemic Networks for Epistemic
Commitments. In International Conference of the Learning Sciences.
Boulder, CO: International Society of the Learning Sciences.
http://oro.open.ac.uk/39254/
See also Knight, S., & Mercer, N. (Forthcoming). The role of exploratory talk in
classroom search engine tasks. Technology, Pedagogy and Education.
http://oro.open.ac.uk/39181/

60. Code Definition Example
Source –
Authority
Make reference to authorities when
selecting information
"use BBC, that's a good site",
“just use the top site”, “it’s a reliable site”
Source –
Corroboration
Make reference to repetition of
information when selecting information
"we said x cuz it was on lots of sites", “well
everyone agreed”
Justification –
Matching
Little attempt to sensemake in question;
targeted matching of source information
to questions.
“it’s an answer”, “look, it says it there”, “what’s it
asking?”
Justification –
Understanding
Attempt to make sense of information, use
more accountable/exploratory dialogue
key phrases
“because”, “so”, “I think”
Simplicity –
Simple
Make few connections between
information, look for individual tokens of
knowledge
“all the information”, “need more facts”, listing of
claims without connections
Simplicity –
Complex
Make connections between tokens of
information, possibly across questions,
contextualize information (e.g. account for
temporal aspect)
“important information”, making connections
between claims (co-occurrence)
Search Referring to search or webpage specific
aspects of the task
“google that”, “click there”, “try searching for…”
General References to general knowledge required Keywords selected for general task relevance; e.g.
“role models”
Specific References to specific pieces of
information in each question
Keywords selected for relevance to the specific
questions asked e.g. “43 women”

61. Some maths happens
• For the purposes of this analysis, I just
presented the final ENA representation,
without the trajectories
• There is interesting scope for the future to
look at the trajectories of ‘experts’ and
‘novices’ in information seeking – for example,
do experts evaluate more throughout? Do
novices emphasise authoritativeness only at
the search stage?

62. Some maths happens
• For any stanza, if a connection exists ->1 – we
don’t care how many times a connection is
made, just that learners are making it
• PCA across stanzas