presented at ACM Web Science 2013, Paris

1. Introduction Socialbot Challenge Success Measures Results Conclusions
Understanding The Impact Of Socialbot Attacks In
Online Social Networks
Silvia Mitter, Claudia Wagner, Markus Strohmaier
Knowledge Technologies Institute
Graz University of Technology
ACM Web Science
May 4, 2013
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 1 / 13

2. Introduction Socialbot Challenge Success Measures Results Conclusions
Introduction
What is a socialbot?
• A socialbot is a piece of software that controls a user account in an
online social network and passes itself of as a human being
The danger of socialbots
• Harvest private user data
• Spread misinformation and influence users
• Boshmaf et al. [2] show that Facebook can be infiltrated by social
bots sending friend requests. Average reported acceptance rate:
35,7% up to 80% depending on how many mutual friends the social
bots had with the infiltrated users.
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 2 / 13

3. Introduction Socialbot Challenge Success Measures Results Conclusions
Research Question
To what extent and how can socialbots manipulate the link creation
behavior of users in OSN?
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 3 / 13

4. Introduction Socialbot Challenge Success Measures Results Conclusions
Research Question
Can socialbots animate previously unconnected users to connect?
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 4 / 13

5. Introduction Socialbot Challenge Success Measures Results Conclusions
The Experiment: Socialbot Challenge
Socialbot challenge on Twitter organized by the Pacific Social Architecture
Group [3].
• Bots: 9 bots using same strategies with some variance
• Aim: manipulate users’s link creation behavior – i.e., animate
previously unconnected users to connect
• Target groups: 9 groups, each consisting of 300 partially socially
interlinked users
• Control Phase: how many new links are usually created between
target users?
• Experimental Phase: how many new links are created between target
users if socialbots are active?
• Success: PacSocial reported a link creation increase of 43% [3]
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 5 / 13

6. Introduction Socialbot Challenge Success Measures Results Conclusions
Dataset
• Tweets of targets and bots during control (33 days) and experimental
phase (21 days)
• Social relations between targets and bots at different points in time
during control and experimental phase
Socialbots 9
Targets 2,700
Number of Tweets 1,006,351
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 6 / 13

7. Introduction Socialbot Challenge Success Measures Results Conclusions
Control and Experimental Phases
19.09 22.10. 12.11.control phase
33 days
experimental phase 1
21 days
control phase
33 days19.09 24.11.
experimental phase 2
33 days22.10.
PacSocial Exp
Modified Exp
Oct 26 2011 Nov 02 2011 Nov 09 2011 Nov 16 2011 Nov 23 20110
100
200
300
400
500
TweetCount
Tweets authored by bots
Figure: Tweets authored by bots over time.
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 7 / 13

8. Introduction Socialbot Challenge Success Measures Results Conclusions
Measuring the Success of Socialbots
PacSocial found link creation increase of 43% during experimental
phase 1, which they attributed to the socialbots.
Measure the impact of socialbots while controlling the impact of some
obvious confounding variables
Success measures describe preceding situations of new link creation events,
along two dimensions:
• Recommendation Types: How is the link creation recommended?
• Mediators: Who recommends the link creation?
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 8 / 13

9. Introduction Socialbot Challenge Success Measures Results Conclusions
Some ways in which new links can be recommended
(Recommendation Types)
• RT 1 – Direct User Recommendation via Tweet
• RT 2 – Indirect User Recommendation via Follow
• RT 3 – Indirect User Recommendation via Tweet
User A User B
Mediator
starts following
creates Tweet
includes
includes
Figure: RT 1
Figure: RT 2 Figure: RT 3
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 9 / 13

10. Introduction Socialbot Challenge Success Measures Results Conclusions
Some ways in which new links can be recommended
(Recommendation Types)
• RT 1 – Direct User Recommendation via Tweet
• RT 2 – Indirect User Recommendation via Follow
• RT 3 – Indirect User Recommendation via Tweet
User A User B
Mediator
starts following
creates Tweet
includes
includes
Figure: RT 1
User A User B
Mediator
follow
s
starts following
follow
s
Figure: RT 2
Figure: RT 3
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 9 / 13

11. Introduction Socialbot Challenge Success Measures Results Conclusions
Some ways in which new links can be recommended
(Recommendation Types)
• RT 1 – Direct User Recommendation via Tweet
• RT 2 – Indirect User Recommendation via Follow
• RT 3 – Indirect User Recommendation via Tweet
User A User B
Mediator
starts following
creates Tweet
includes
includes
Figure: RT 1
User A User B
Mediator
follow
s
starts following
follow
s
Figure: RT 2
User A User B
Mediator
follows
starts following
includes
creates Tweet
Figure: RT 3
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 9 / 13

12. Introduction Socialbot Challenge Success Measures Results Conclusions
Who recommends a link (Mediator Types)
• Human Mediator: Every user from the target group can act as human
mediator.
• Socialbot Mediator: Every socialbot can act as socialbot mediator.
• Human AND Socialbot Mediator: Preceding human and socialbot
mediator actions can be measured before link creation.
• No Measurable Mediator: No potential mediator can be identified
from the data explaining the link creation.
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 10 / 13

13. Introduction Socialbot Challenge Success Measures Results Conclusions
Who recommends a link (Mediator Types)
• Human Mediator: Every user from the target group can act as human
mediator.
• Socialbot Mediator: Every socialbot can act as socialbot mediator.
• Human AND Socialbot Mediator: Preceding human and socialbot
mediator actions can be measured before link creation.
• No Measurable Mediator: No potential mediator can be identified
from the data explaining the link creation.
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 10 / 13

14. Introduction Socialbot Challenge Success Measures Results Conclusions
Who recommends a link (Mediator Types)
• Human Mediator: Every user from the target group can act as human
mediator.
• Socialbot Mediator: Every socialbot can act as socialbot mediator.
• Human AND Socialbot Mediator: Preceding human and socialbot
mediator actions can be measured before link creation.
• No Measurable Mediator: No potential mediator can be identified
from the data explaining the link creation.
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 10 / 13

15. Introduction Socialbot Challenge Success Measures Results Conclusions
Who recommends a link (Mediator Types)
• Human Mediator: Every user from the target group can act as human
mediator.
• Socialbot Mediator: Every socialbot can act as socialbot mediator.
• Human AND Socialbot Mediator: Preceding human and socialbot
mediator actions can be measured before link creation.
• No Measurable Mediator: No potential mediator can be identified
from the data explaining the link creation.
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 10 / 13

16. Introduction Socialbot Challenge Success Measures Results Conclusions
Who recommends a link (Mediator Types)
• Human Mediator: Every user from the target group can act as human
mediator.
• Socialbot Mediator: Every socialbot can act as socialbot mediator.
• Human AND Socialbot Mediator: Preceding human and socialbot
mediator actions can be measured before link creation.
• No Measurable Mediator: No potential mediator can be identified
from the data explaining the link creation.
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 10 / 13

17. Introduction Socialbot Challenge Success Measures Results Conclusions
Link Creation
Link Creation ctr exp1 exp2
Total 5.49 7.62 6.76
-Direct User Interaction -2.12 -2.71 -2.55
Basis for Calculations 3.36 4.91 4.21
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 11 / 13

18. Introduction Socialbot Challenge Success Measures Results Conclusions
Link Creation
Link Creation ctr exp1 exp2
Total 5.49 7.62 6.76
-Direct User Interaction -2.12 -2.71 -2.55
Basis for Calculations 3.36 4.91 4.21
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 11 / 13

19. Introduction Socialbot Challenge Success Measures Results Conclusions
Link Creation
Link Creation ctr exp1 exp2
Total 5.49 7.62 6.76
-Direct User Interaction -2.12 -2.71 -2.55
Basis for Calculations 3.36 4.91 4.21
follows
User A User B
starts following
User A User B
starts following
communicate
(a)
(b)
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 11 / 13

20. Introduction Socialbot Challenge Success Measures Results Conclusions
Link Creation
Link Creation ctr exp1 exp2
Total 5.49 7.62 6.76
-Direct User Interaction -2.12 -2.71 -2.55
Basis for Calculations 3.36 4.91 4.21
follows
User A User B
starts following
User A User B
starts following
communicate
(a)
(b)
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 11 / 13

21. Introduction Socialbot Challenge Success Measures Results Conclusions
Results
Link Creation RT 123 human
mediated
socialbot
mediated
human &
socialbot
mediated
undefined
mediated
abs % abs % abs % abs %
control phase 1.49 44.14 0.00 0.00 0.00 0.00 1.88 55.86
exp. phase 1 1.81 36.90 0.33 6.79 0.29 5.83 2.48 50.48
exp. phase 2 1.46 34.54 0.49 11.51 0.49 11.51 1.79 42.45
• Total increase of links in exp1 around 40% and around 20% in exp2.
That means human’s link creation behavior increased. But why?
• Around 50% of links can not be explained by preceding situation
(external factors?)
• Around 1/3 of all links have been recommended by human and only
6-12% have been recommended by bots
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 12 / 13

22. Introduction Socialbot Challenge Success Measures Results Conclusions
Results
Link Creation RT 123 human
mediated
socialbot
mediated
human &
socialbot
mediated
undefined
mediated
abs % abs % abs % abs %
control phase 1.49 44.14 0.00 0.00 0.00 0.00 1.88 55.86
exp. phase 1 1.81 36.90 0.33 6.79 0.29 5.83 2.48 50.48
exp. phase 2 1.46 34.54 0.49 11.51 0.49 11.51 1.79 42.45
• Total increase of links in exp1 around 40% and around 20% in exp2.
That means human’s link creation behavior increased. But why?
• Around 50% of links can not be explained by preceding situation
(external factors?)
• Around 1/3 of all links have been recommended by human and only
6-12% have been recommended by bots
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 12 / 13

23. Introduction Socialbot Challenge Success Measures Results Conclusions
Results
Link Creation RT 123 human
mediated
socialbot
mediated
human &
socialbot
mediated
undefined
mediated
abs % abs % abs % abs %
control phase 1.49 44.14 0.00 0.00 0.00 0.00 1.88 55.86
exp. phase 1 1.81 36.90 0.33 6.79 0.29 5.83 2.48 50.48
exp. phase 2 1.46 34.54 0.49 11.51 0.49 11.51 1.79 42.45
• Total increase of links in exp1 around 40% and around 20% in exp2.
That means human’s link creation behavior increased. But why?
• Around 50% of links can not be explained by preceding situation
(external factors?)
• Around 1/3 of all links have been recommended by human and only
6-12% have been recommended by bots
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 12 / 13

24. Introduction Socialbot Challenge Success Measures Results Conclusions
Results
Link Creation RT 123 human
mediated
socialbot
mediated
human &
socialbot
mediated
undefined
mediated
abs % abs % abs % abs %
control phase 1.49 44.14 0.00 0.00 0.00 0.00 1.88 55.86
exp. phase 1 1.81 36.90 0.33 6.79 0.29 5.83 2.48 50.48
exp. phase 2 1.46 34.54 0.49 11.51 0.49 11.51 1.79 42.45
• Total increase of links in exp1 around 40% and around 20% in exp2.
That means human’s link creation behavior increased. But why?
• Around 50% of links can not be explained by preceding situation
(external factors?)
• Around 1/3 of all links have been recommended by human and only
6-12% have been recommended by bots
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 12 / 13

25. Introduction Socialbot Challenge Success Measures Results Conclusions
Results
Link Creation RT 123 human
mediated
socialbot
mediated
human &
socialbot
mediated
undefined
mediated
abs % abs % abs % abs %
control phase 1.49 44.14 0.00 0.00 0.00 0.00 1.88 55.86
exp. phase 1 1.81 36.90 0.33 6.79 0.29 5.83 2.48 50.48
exp. phase 2 1.46 34.54 0.49 11.51 0.49 11.51 1.79 42.45
• Total increase of links in exp1 around 40% and around 20% in exp2.
That means human’s link creation behavior increased. But why?
• Around 50% of links can not be explained by preceding situation
(external factors?)
• Around 1/3 of all links have been recommended by human and only
6-12% have been recommended by bots
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 12 / 13

26. Introduction Socialbot Challenge Success Measures Results Conclusions
Conclusions
• It is unlikely that a causal relation between the 40% increase of social
link creation and socialbot interaction exists.
• But we observed few new links which are likely to be caused by
socialbots though the short experimental period and the cold start
problem of social bots.
• Socialbots may indeed manipulate the social graph of OSN but they
are not yet as powerful as human.
• Our results also highlight the role of external factors in link creation,
which is partly in line with Backstrom et al. [1] and Rowe et al. [4].
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 13 / 13

27. Introduction Socialbot Challenge Success Measures Results Conclusions
Conclusions
• It is unlikely that a causal relation between the 40% increase of social
link creation and socialbot interaction exists.
• But we observed few new links which are likely to be caused by
socialbots though the short experimental period and the cold start
problem of social bots.
• Socialbots may indeed manipulate the social graph of OSN but they
are not yet as powerful as human.
• Our results also highlight the role of external factors in link creation,
which is partly in line with Backstrom et al. [1] and Rowe et al. [4].
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 13 / 13

28. Introduction Socialbot Challenge Success Measures Results Conclusions
Conclusions
• It is unlikely that a causal relation between the 40% increase of social
link creation and socialbot interaction exists.
• But we observed few new links which are likely to be caused by
socialbots though the short experimental period and the cold start
problem of social bots.
• Socialbots may indeed manipulate the social graph of OSN but they
are not yet as powerful as human.
• Our results also highlight the role of external factors in link creation,
which is partly in line with Backstrom et al. [1] and Rowe et al. [4].
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 13 / 13

29. Introduction Socialbot Challenge Success Measures Results Conclusions
Conclusions
• It is unlikely that a causal relation between the 40% increase of social
link creation and socialbot interaction exists.
• But we observed few new links which are likely to be caused by
socialbots though the short experimental period and the cold start
problem of social bots.
• Socialbots may indeed manipulate the social graph of OSN but they
are not yet as powerful as human.
• Our results also highlight the role of external factors in link creation,
which is partly in line with Backstrom et al. [1] and Rowe et al. [4].
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 13 / 13

30. Introduction Socialbot Challenge Success Measures Results Conclusions
[1] Lars Backstrom and Jure Leskovec. “Supervised random walks: predicting and
recommending links in social networks”. In:
Proceedings of the fourth ACM international conference on Web search and data mining.
WSDM ’11. Hong Kong, China: ACM, 2011, pp. 635–644. doi:
10.1145/1935826.1935914 (cit. on pp. 26–29).
[2] Yazan Boshmaf, Ildar Muslukhov, Konstantin Beznosov, and Matei Ripeanu. “The
socialbot network: when bots socialize for fame and money”. In:
Proceedings of the 27th Annual Computer Security Applications Conference. ACSAC ’11.
Orlando, Florida: ACM, 2011, pp. 93–102. doi: 10.1145/2076732.2076746 (cit. on p. 2).
[3] Max Nanis, Ian Pearce, and Tim Hwang. PacSocial: Field Test Report.
http://www.pacsocial.com. 2011 (cit. on p. 5).
[4] Matthew Rowe, Milan Stankovic, and Harith Alani. “Who will follow whom? Exploiting
semantics for link prediction in attention-information networks”. In:
11th International Semantic Web Conference (ISWC 2012). 2012 (cit. on pp. 26–29).
Mitter, Wagner, Strohmaier (TU Graz) Impact of Socialbots in OSNs May 4, 2013 13 / 13