Presented at the Web Observatory Workshop at the World Wide Web Conference in Rio De Janiero, Brazil.

1. Interoperability of
a Social Media Observatory
Karissa McKelvey
And Filippo Menczer
Center for Complex Networks and Systems Research
School of Informatics and Computing
Indiana University, Bloomington
1

2. truthy.indiana.edu 2

3. Can we use social media as
laboratories for social
science?
truthy.indiana.edu 3

4. Political Polarization on Twitter Michael Conover, Jacob Ratkiewicz, Bruno Gonçalves, Alessandro Flammini
& Filippo Menczer International Conference on Weblogs and Social Media 2011
truthy.indiana.edu 4

5. Competition for attention
Competitions among topics in a world with limited attention Nature Scientific
Report, 2012. Lilian Weng, Alessandro Flammini, Alessandro Vespignani and Filippo
Menczer.
#jan25
truthy.indiana.edu 5

6. truthy.indiana.edu 6
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
0 20 40 60 80 100
−2e+050e+002e+05
a. 2010
Tweet Share
VoteMargin
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
0 20 40 60 80 100
−2e+050e+002e+05
b. 2012
Tweet Share
VoteMargin
More Tweets, More Votes
DiGrazia, Joseph, McKelvey, Karissa, Bollen, Johan and Rojas, Fabio. More Tweets, More Votes: Social
Media as a Quantitative Indicator of Political Behavior (February 21, 2013). Available at SSRN:
http://ssrn.com/abstract=2235423 or http://dx.doi.org/10.2139/ssrn.2235423

7. astroturf
truthy.indiana.edu 7

8. Data “mine” ing
truthy.indiana.edu 8

9. Data “ours” ing
truthy.indiana.edu 9

10. truthy.indiana.edu
truthy.indiana.edu 10

11. truthy.indiana.edu 11

12. Infrastructure
nosql& mysql&hbase&
Storage&
Social&
Media&
Model& APIs&
Toolkits/
Website&
truthy.indiana.edu 12

13. Infrastructure
nosql& mysql&hbase&
Storage&
Social&
Media&
Model& APIs&
Toolkits/
Website&
truthy.indiana.edu 13

14. Data Collection
• Twitter Streaming API
• August, 2010 – present
• 5TB Compressed
truthy.indiana.edu 14

15. Real-Time Filtering
• Past 90 days
• US Politics 2012, Social Movements, News
• MySQL database + website
truthy.indiana.edu 15

16. Infrastructure
nosql& mysql&hbase&
Storage&
Social&
Media&
Model& APIs&
Toolkits/
Website&
truthy.indiana.edu 16

17. Model
• Events
– Post on a social media site
• Users
– Actors in the post
– Sender, receiver, forwarder, etc
• Meme
– Discernable unit of information transfer
truthy.indiana.edu 17

18. Meme
Hashtag
User
URL
Phrase
truthy.indiana.edu 18

19. Building the Network
truthy.indiana.edu 19
Retweet (forward) Mention (conversation)

20. celebrities
spam
astroturf
politics
20

21. politics
truthy.indiana.edu 21

22. #tcot
truthy.indiana.edu 22

23. celebrities
politics
truthy.indiana.edu 23

24. @barackobama
truthy.indiana.edu 24

25. celebrities
spam
astroturf
politics
25

26. @bek_rt
truthy.indiana.edu 26

27. Infrastructure
nosql& mysql&hbase&
Storage&
Social&
Media&
Model& APIs&
Toolkits/
Website&
truthy.indiana.edu 27

28. truthy.indiana.edu
truthy.indiana.edu 28

29. truthy.indiana.edu 29

30. Infrastructure
nosql& mysql&hbase&
Storage&
Social&
Media&
Model& APIs&
Toolkits/
Website&
truthy.indiana.edu 30

31. API & Website
• Filterable and searchable interface to find
memes of interest
• Endpoints for users to access data
programmatically
• Visualizations (saw earlier)
truthy.indiana.edu 31

32. Further Thoughts
• Potential uses:
– Pedagogy
– Prototyping
– Research
• Terms of Service
– Tweet text
– Derived Data
truthy.indiana.edu 32

33. Thanks
Papers at cnets.indiana.edu/groups/nan/truthy
Sandro Flammini
Bruno Conçalves
Jacob Ratkiewicz
Lilian Weng
Mike Conover
Johan Bollen
Przemek Grabowicz
Mark Meiss
Alex Vespignani
Alex Rudnick
Luca Aiello
Filippo Menczer
truthy.indiana.edu 33