Abul-Fottouh, D., Song, M. Y., & Gruzd, A. (2020). Examining algorithmic biases in YouTube’s recommendations of vaccine videos. International Journal of Medical Informatics, 104175. Read our follow-up study at https://doi.org/10.1016/j.ijmedinf.2020.104175 ================================= Song, M.Y. & Gruzd, A. (2017). Examining Sentiments and Popularity of Pro- and Anti-Vaccination Videos on YouTube. In Proceedings of the 8th International Conference on Social Media & Society (#SMSociety17). ACM, New York, NY, USA, Article 17, 8 pages. DOI: https://doi.org/10.1145/3097286.3097303

1. Examining Sentiments
and Popularity of Pro- and
Anti-vaccination Videos
on YouTube
Melodie Yun-Ju Song, Health Policy PhD candidate, McMaster University
Anatoliy Gruzd, Director, Social Media Lab, Professor, Ryerson University

2. Have you seen these videos?
@MelodieYJSong @gruzd
?
2
Blackbox
Vaccine hesitancy Anti-vaccine

3. The State of Vaccine Confidence/ Sentiment in 2016:
Global Insights Through a 67-Country Survey
3
Larson et al, 2016

4. 4
What can networks tell us?
“We use network science to discover how the whole comes to be greater than sum of
its parts. The study of networks enriches our understanding of phenomena such as
riots, violence, and social and biological epidemics — enabling us to develop more
effective interventions.” – Nicholas Christakis

5. 1. What is the sentiment/opinions towards vaccine on YouTube?
2. What are the network properties of vaccine-related videos on YouTube?
3. How are vaccine-related videos categorized?
4. Are there any pronounced differences in vaccine sentiment in relation to
video attributes (i.e., video category, dislike/like count, view count)?
Research Question
@MelodieYJSong @gruzd 5

6. Data Collection
• Software: Netvizz for collecting YouTube related videos (Reinhardt, 2015)
• Search terms: vaccine, immunization, and other vaccine-related keywords for
5 iterations (N= 9489 videos, including video attributes: view count, comment
count, dislike/like ratio, video categories).
• Inclusion criteria: (1) Titles containing vaccination-related terms, (2) pro- and
anti-vaccine videos in English (97.9% of all videos). (N=1984)
@MelodieYJSong @gruzd 6

7. Data Analysis
@MelodieYJSong @gruzd
Data analysis
❖ Sentiment analysis/ visual content analysis: N=1984
❖ Social Network analysis: Gephi and UCINet 6
Statistical
analysis
❖ T-test to compare the means of node-level anti-vaccine and
pro-vaccine video’s centrality measures on UCINet.
❖ Logistic regression for video properties and vaccine sentiment
❖ Association between sentiment and video categories.
7

8. 8
Using video network analysis to understand relations of videos
Centrality: The connections one video has with other videos, including degree centrality, closeness centrality,
and betweenness centrality.
Out-degree centrality: The number of videos one video links outwards to.
In-degree centrality: The number of videos are linked towards one particular video. Videos with higher in-
degree are considered prominent.
Out-closeness centrality: The shortest distance (e.g., number of videos to click through) to reach other videos
from a certain video.
In-closeness centrality: The shortest average distance (e.g., avg. number of videos to click through) to reach
a certain video.
Betweenness centrality: The extent a video is positioned on the shortest path between other pairs of videos in
the vaccine video network.

9. 9
Vaccine-related videos network on Youtube
@MelodieYJSong @gruzd
“Vaccines -The truth behind vaccinations”
“No Vaccines Necessary, that's the truth.”
“Worst Nightmare for Mother of 6
Unvaxxed Children”

10. 65.02% of vaccine-related videos
are anti-vaccine (N=1984)
1. What is the sentiment towards vaccines on YouTube?
@MelodieYJSong @gruzd 10

11. 2. What are the network properties of vaccine-related videos on YouTube?
Anti-vaccine videos are easier to reach than pro-vaccine videos if you started
with one.
11
Centrality measures Mean of Anti-vaccine
related videos
Mean of Pro-vaccine
related videos
Difference in means Significance
Out-degree 0.004 0.004 0.000 0.0009**
In-degree 0.003 0.003 0.000 0.9735
Out-closeness 0.240 0.232 0.008 0.0001***
In-closeness 0.183 0.179 0.004 0.0096**
Betweenness 0.001 0.002 -0.001 0.0021**

12. News & Politics (50.8%)
Education (49.0%)
People & Blogs (40.0%)
Non-profit & Activism (22.8%)
Science & Technology (19.5%)
Comedy
Entertainment
Music
@MelodieYJSong @gruzd
3. How are vaccine-related videos categorized?
12
Anti-vaccine
videos is
significantly over-
represented in the
‘News and Politics’
and ‘People and
blogs’ category.
Pro-vaccine videos
are over-
represented in the
‘Education’ and
‘Science and
technology’
category.

13. Videos with higher
dislike/like ratio have
3.912 higher odds of
being pro-vaccine.
Pro-vaccine videos are
disliked, and more
controversial than anti-
vaccine videos?
@MelodieYJSong @gruzd
4. What are the differences in vaccine sentiment in relation to video attributes?
13
Sentiment
Dislike/like ratio (OR 3.912)**
Dislike count (OR 0.996)**
Like count (OR 1.000)**
View count (OR 1.000)*
Comment count (OR 0.338)
R = 0.09

14. • Starting a YouTube search with an anti-vaccine keyword search will more likely
introduce viewers to other anti-vaccine videos.
• YouTube vaccine video categories may reflect why uploaders of vaccine-related
videos think vaccine-issues are important.
• How might interdisciplinary research contribute to rethinking anti-vaccine
movements?
Discussion
@MelodieYJSong @gruzd 14

15. 1. Knowledge deficit paradigm
2. Risk-aversion paradigm
o Rational choice
o Semi-rational choice
3. Influenced by others
o Media, social media, news
o Friends and family
o Personal experiences
o Influence by default (e.g., algorithms, bots, etc)
A convergence of paradigms to approach anti-vaccine sentiment
@MelodieYJSong @gruzd 15
Influence of
others
Knowledge
deficit
Risk-aversion

16. • More understanding and collaboration between developers of social media platforms
and public health policy researchers.
• The rise of botnets and anti-vaccine astroturfers pose potential implications to the
centrality of videos.
• Longitudinal observations to trace the trends of migration from sentiment- neutral to
anti-vaccine sentiment are needed (Mitra et al, 2016)
Recommendations
@MelodieYJSong @gruzd 16

17. Acknowledgement
• Director of Social Media Lab, Professor Anatoliy Gruzd
• Director of Business and Communications, Philip Mai
• Professor Julia Abelson, McMaster University
• The entire Social Media Lab
17

18. "The paranoid spokesman sees the fate of
conspiracy in apocalyptic terms- he traffics in the
birth and death of whole worlds, whole political
orders, whole systems of human values."
- Richard Hofstadter, 1964.
18