A framework to measure conversation based on social psychological and sociological perspectives.

1. Web 2.0 4.1.09
Beyond Buzz: On
measuring a conversation
Kate Niederhoffer, Ph.D Marc A. Smith, Ph.D
Dachis Corporation Telligent Systems
Proprietary & Confidential — © 2008 DachisCorporation

2. Why us?
Marc Smith
Kate Niederhoffer
• Ph.D UCLA Sociology
• Ph.D UT Social Psychology
• Microsoft Research, Community
• BuzzMetrics/Nielsen Online,
Technologies Group
Measurement Science
• Telligent Systems – “Harvest” reporting
• Dachis Corporation - Methodology,
and analysis tools for social media platforms
Social Business Design
and systems
Note: This is a conceptual address. We’re talking about ideas; each of our
companies have distinct methodologies in place related to these concepts.
2

3. Why are we here?
1. Demonstrating the depth
of buzz; ways to think
about signal within vast
universe.
2. Going beyond buzz;
learning more about
individuals.
3

4. Why are we here?
3. Highlighting the unique
roles individuals play in
communities that afford
the conversation.
4. Illustrating that
aggregated relationships
are network structures.
4

5. Why now?
5

6. Blogs were all the rage
In 2005, clients attracted by novelty:
Simple question: What’s my buzz?
- How much?
- Good or bad?
Incremental improvement: How “important” is it?
- Are “Influencers” talking?
- How many eyeballs exposed?
- Engagement?
However, all superficially measured;
limited scope of what’s important: what kind of influence?
6

7. Blogs are now features
• Today’s “media” enable richer social interaction-- and, leave a path
of data with more opportunities to capture depth
• Buzz levels, page views, followers, in isolation miss big picture
• Must take advantage context to tell whole story and capture value
7

8. Social networks are all the rage, but
rarely do we think about social metrics
We need to stop blackboxing:
quot;When a machine runs efficiently, when a matter of fact is settled, one need focus
only on its inputs and outputs and not on its internal complexity. Thus, paradoxically, the
more science and technology succeed, the more opaque and obscure they become.quot;
- Bruno Latour
Even if a conversation is running smoothly, we must figure out what makes it tick.
8

9. Social Network
Theory
Central tenet:
Social structure emerges from
the aggregate of relationships (ties)
among members of a population
Phenomena of interest:
Emergence of cliques and clusters Source: Richards, W. (1986). The NEGOPY network
from patterns of relationships analysis program. Burnaby, BC: Department of
Communication, Simon Fraser University. pp.7-16
Centrality (core), periphery (isolates),
betweenness
Methods:
Surveys, interviews, observations, log file analysis,
computational analysis of matrices
(Hampton &Wellman, 1999; Paolillo, 2001; Wellman, 2001)

10. Context of a conversation
Relevance
Signal
Where’s the signal in the noise?
Mindset
Person
What else do we know about the individuals?
Role
Persona
What is the pattern of connections?
Ecosystem
Environment
What is the dynamic, en masse?
10

11. 11

12. Context of a conversation
Relevance
Where’s the signal in the noise?
Mindset
Role
Ecosystem
12

13. Relevance today
• As a user, easy to relate to issues with pre-determined filters.
• As an enterprise, complexity increases.
We don’t always know what we want to know!
13

14. Relevance:
Which filters are in place to strengthen the signal?
• Identifying your filters can
be inductive:
• What are people really saying?
• Which concepts differentiate the
posts that mention you vs. posts
that don't?
* Source: Nielsen Online, 2008
• All terms on your map have a correlation to the central concept; the closer a word appears to the center, the stronger the
association.The groupings of terms indicate the dimensions of discussion: micro-conversations within a broader discussion.
14

15. Relevance is multi-faceted
• Rather than looking at
associations with, as
compared to without,
consider discussion this
week as compared to
discussion over the past
year.
• Not what’s being said about
her in a more recent
timeframe, but instead
when you control for what’s
said about her in general,
what pops?
* Source: Nielsen Online, 2008
15

16. Relevance - Summary
• Information can be visualized in so many different
ways; don’t take it for granted.
• Listening can be limited if you’re exclusively looking for
something in particular; broaden your net. Be inductive.
Let the data speak for itself.
16

17. 17

18. Context of a conversation
Relevance
Mindset
What else can we know about the individuals?
Role
Ecosystem
18

19. Says Who?
19

20. Mindset
What else can we know about the person in conversation?
• By measuring the types of words used, we can
Findings
Linguistic Cues
tap into how people ‘slice’ their worlds.
• Linguistic style is closely tied to:
Are you self-oriented?
Pronoun use: I and We
• Demographics (e.g. age, sex, class)
Are you living in ‘the
Past, Present, Future tense
• Emotion (e.g. depression, deception) now’?
• Cognitive style (e.g. complex thinking) What is your emotional
Positive vs. Negative
tone?
• Personality (e.g. Neuroticism)
Are you abstract or Articles: “a” vs. “the”
concrete?
Nouns vs. verbs
e.g. Pennebaker, Mehl, Niederhoffer, 2003
20

21. When people make
recommendations on blogs, is
there something deeper going on?
“Got the next three PW/GS games for my
birthday. And I am one happy gal, there was
some stuff that I absolutely LOVED and I
would definitely recommend
the game to anyone who owns a PS3 regardless
of its flaws -- which really were at their heart
personal quibbles of mine so your mileage may
vary. Plus, I cried like a b*$$ at the end. That's
got to be saying something.”
21

22. Getting into the Engaged Mind
• Recommendations have:
• More pronouns: intimacy with both the brand/product/ service being
recommended, and those to whom they’re recommending.
• More verbs: sharing experience more than discussion of concrete features.
* all differences significant at p<.01 level
22

23. “Invisible” language gives us clues
about individuals, and groups
23

24. Changes in work atmosphere,
captured in words
Engineers, economists
programmers collaborating on
economic simulations of disasters
• Complexity of thought (-)
• Cohesion (-)
• Work information (-)
• Negative emotion (+)
• Funding lost
Tausczik, Scholand, and Pennebaker, 2009
24

25. “Connected Age”: relationships are
groundwork of work
Social: niceties (lol), affirmations (cool), Work: economic (production, supply),
coordination (call), broad communication analytic (results, problem)
(http, thinking)
25

26. Mindset- Summary
• Language is a good way to go beyond the surface
and better understand constituents without self-
report biases (or effort).
• Metrics in the hands of users (yourselves) are helpful:
know thyself, know how you’re perceived.
26

27. Beyond thoughts and feelings, who comes to roost?
27

28. Context of a conversation
Relevance
Mindset
Role
What is the pattern of connections?
Ecosystem
28

29. Social
Network
Analysis
with
NodeXL:
Identify
different
roles in
social media
spaces

30. Identify core groups in the
network

31. Distinguishing attributes:
• Answer person
– Outward ties to local isolates
– Relative absence of triangles
– Few intense ties
• Reply Magnet
– Ties from local isolates often inward
only
– Sparse, few triangles
– Few intense ties
31

32. Distinguishing attributes:
• Answer person
– Outward ties to local isolates
– Relative absence of triangles
– Few intense ties
• Discussion person
– Ties from local isolates often inward
only
– Dense, many triangles
– Numerous intense ties
32

33. Answer
Person
Signatures
Discussion
People
33

34. Discussion
Starter
Spammer
Reply oriented
Discussion Flame
Warrior
34

35. Role – Summary
• Network awareness, like court vision enables strategic play. Know which positions/
players are on your team.
• Social media behavior is differentiated. Rare (~.5-2%) roles are critical and must
be cultivated.
• E.g. Clear and consistent signatures of an “Answer Person
• Light touch to numerous threads initiated by someone else
• Most ties are outward to local isolates
• Many more ties to small fish than big fish
35

36. What is the mix in the
neighborhood?
36

37. Context of a conversation
Relevance
Mindset
Role
Ecosystem
What is the dynamic, en masse?
37

38. 38

39. The Ties that Blind?
Pajek without modifica>on can
some>mes reveal structures of great interest.

40. Darwin Bell
40

41. 41

42. 42

43. Mapping
Newsgroup
Social
Ties
Microsoft.public.windowsxp.server.general
43
Two “answer people” with an emerging 3rd.

44. Research shows social media
spaces vary and roles are present
Adamic et al. WWW
2008

45. Ecosystem- Summary
• Social media is about collective action.
• A balance of roles and strategies is critical for a
healthy/ successful collective good.
• Harvesting the common good takes many forms,
and is the ultimate goal of social media.
45

46. Why does this matter?
• This is not measurement for the sake of
measurement; we need to measure conversations in
order to manage social business.
• Measuring conversations is about measuring the
context in which those conversations arise.
• Value is an intermediate step in calculating ROI. Moot
to bypass it.
• Techniques from social science help capture “the
immeasurable” in social media and the enterprise.
• The future of conversations- the enterprise being
one-- is about cultivating ecologies of the right balance
of relationships.
46

47. Thank You
k.niederhoffer@gmail.com
marc.smith@telligent.com
Questions?
47

48. Additional Resources
48

49. 49

50. Small Groups
Individuals Uniform Large Groups
Heterogeneous
Variable Contribu>on
Variable Contribu>on
Large Groups
How uniform are social Large Groups
media producing groups?

51. Social Science Theory and
Method
Interactionist Sociology Collective Action Dilemmas
Central tenet
Central tenet
Focus on the active effort of Individual rationality leads to collective
accomplishing interaction disaster
Phenomena of interest
Phenomena of interest
Presentation of self
Provision and/or sustainable
consumption of collective resources
Claims to membership
Juggling multiple (conflicting) roles Public Goods, Common Property, quot;Free
Rider” Problems, Tragedies
Frontstage/Backstage
Strategic interaction
Methods
Managing one’s own and others’ “face”
Surveys, interviews, participant
observation, log file analysis, computer
Methods modeling
Ethnography and participant observation
(Axelrod, 1984; Hess, 1995; Kollock &
Smith, 1996)
(Goffman, 1959; Hall, 1990)