Lefkothea Spiliotopoulou, Researcher, University of the Aegean, Greece

1. Empowering Digital Direct
Democracy:
Policy Making via Stance
Classification
Lefkothea Spiliotopoulou, Yannis Charalabidis, Euripidis Loukis,
Dimitrios Damopoulos

2. Meet the team 
Lefkothea Spiliotopoulou, Dr. Yannis Charalabidis, Dr. Euripidis Loukis, Dr. Dimitrios Damopoulos
2

3. Introduction
• Ideal democracy allows citizens to directly be involved in the
decision making process
• Traditional direct policy making mechanisms (e.g.
referendums, elections, gallops, polls) cannot be utilized
frequently due to high cost
• Innovation in ICT & social media can be the key to facilitate
collaborative governance with citizens & online participation
Aim:
• Governmental policy making mechanism can utilize public’s
stance to empower direct democracy through text mining
exploitation
3

4. Today we focus on…
• How a stance classification system can be utilized in digital
government for direct democracy empowerment
• Building our model through the use of text mining techniques
(topic modeling, sentiment analysis, stance classification)
• Empirical studies on:
• Public’s stance on 4 EU Referendums
• Political candidates' opinions during
2016 U.S. Presidential Elections
• 2016 U.S. President’s stance towards
societal issues & their affects
4

5. Policy Making Process
• Decisions taken by governments solving problems & improving
citizens’ quality of life
• Stages of policy cycle
• Agenda Setting
• Policy Formulation
• Decision Making
• Policy Implementation
• Policy Evaluation
5

6. Text Mining Techniques
• Topic modeling
• statistical modeling to discover abstract "topics" that occur in a
collection of documents
• Sentiment analysis
• natural language processing to determine positive, negative or
neutral opinions expressed in a text
• Stance classification
• categorize an author’s personal position towards a topic of
discussion as for or against
6

7. System Architecture
7

8. • Experiments based on 10-fold cross validation
• Data analysis on the antsle one pro server with a 2.40GHz Intel
8 Core, 32GB ECC DDR3, 12TB internal storage
8
System Evaluation

9. Empirical Studies
EU Referendums
9

10. Portrait of EU Referendums
• Utilization of online news sites with articles’ topics based on
the 4 EU Referendums
• Sequence of daily political occurrences (events)
• events with high rate of articles & users’ posts  critical
• We extract from each article:
• event’s title
• article
• user’s comments
• timestamp
10

11. Policy making cycle
for end of policy
cycleagainst 11
• Visually represent citizens' stance
• Political parties can learn from real-
time feedback during the process
• Relate positive or negative stance
towards a specific discussion topic
of the agenda
• Re-evaluate policies before decision
making stage

12. Duration of a policy making cycle
12
• Not all policy making cycles are the same when it comes to their
duration
• Represent an event macroscopic or microscopic over the years
• Still something that needs further analysis
• -
for end of policy cycleagainst

13. Early awareness leads to a better
policy
for end of policy cycleagainst 13
• Continues evaluation of policy making process guides to happier
citizens
• Mitigating issues of the policy in early stages can be a route to
successful policy evaluation (or not)
Agenda Setting

14. Empirical Studies
2016 U.S. Presidential Elections
14

15. 2016 U.S. Presidential Elections
• Track public’s stance for Donald Trump & Hilary Clinton
• 2016 U.S. presidential elections public feeling
• online user’s stance towards Donald Trump & Hilary Clinton
• Compare Donald Trump’ & Hilary Clinton’s Stance on societal
issues
• Utilization of Twitter API & online news sites for data
collection
• Events Timeline:
• starting point: Spring of the year before an election (12/04/2015)
• ending point: Inauguration Day (20/01/2017)
15

16. U.S. Presidential Elections Public
Feeling
• Determine public’s sentiment for Donald Trump & Hilary
Clinton in each stage of policy life cycle
• Timeline: 2016 U.S. Presidential elections
16

17. Trump vs Clinton Twitter Stance
• Classify the overall stance for Donald Trump & Hilary Clinton in
Twitter
17

18. Empirical Studies
2016 U.S. Political Candidates Stance on Societal Issues
18

19. Trump vs Clinton Twitter Topic
Stance
• Compare the overall stance of
Donald Trump & Hilary Clinton
in societal issues
• Selection of topics with the
highest popularity for the US
Community
Topics
Guns
Clinton Trump
Abortion
Taxes
Immigration
Health
For
Against
19

20. U.S. President Trump’s Feeling &
Stock Market
• U.S. President Donald Trump’s tweets stance towards policies
formulation (e.g. immigration law)
• Stock market value measurement: correlation of political
presidential statements with market capitalization
20
Marketcap
tech companies

21. Conclusions
• Investigate the societal impact of strong political events
• Introduce a stance classification architecture with
linguistic features
• Analyze citizens’ opinions and their stance towards critical
political topics through our 3 empirical studies
• Online news sites require users’ subscription
• APIs from Twitter or third party services frequently change
or upgrade
• Need to constantly update a platform, to maintain its
functionalities, is a challenging task 21
Summary
Limitations

22. Future Steps
• Explore additional features containing tolerance or irony
• Examine more sophisticated classifiers (e.g. Deep Learning) to
discover hidden features
• Analyze similar political events to gain further insights 
• Need of an ideal framework for real-time policy making to
empower users’ participation in the modern era of Digital
Direct Democracy
• Policy making is not just a mechanism. Is a viable system
22

23. Stance Classification Machine in
Numbers
23

24. ThankYou!
24

25. System Architecture
25

26. Back-End Module
• Twitter & Online News
Sites
• Collect event title,
article, users’
comments, username &
timestamp
• News Sites selection
based on high
popularity measured by
Alexa
26

27. Policy Making Module
• Statistical Analysis to
distinguish days with
high rate of posts &
comments in articles
• Tokenization
• POS-tagging
• Sentence Splitting
• Stanford Parser to
calculate shortest
distance between words
• Porter Stemming
Algorithm to find root of
each word
• NLTK for n-gram
generation
• Create uni-grams & bi-
grams utilizing nouns,
adjectives & verbs
• Utilize them as features
for stance classification
27

28. Policy Making Module
• Mallet to extract topics
with top-words
• SentenceLDA algo with
input posts' sentence &
output topics in each
sentence with top-
words
• Diffchecker to find &
compare whether topics
& top-words of each
posts' sentence are the
same with topic
modeling results
• if yes, sentence is
relative
• Split dataset in training
& testing sets
• Training dataset is by
learning from the 20%
of the daily topics of
each study containing
top-words
• Manual annotation to
label training dataset
• 2 human annotators, via
Mechanical Turk, label
each post's sentence
stance towards topics as
for or against
• Annotate each sentence
based on the topic to
which it was most
related (topic
classification)
• Annotate post ’s overall
position towards the
topic (stance
classification)
28

29. Policy Making Module
• Keep top-words from
topic modeling that are
same with those
identified as uni- or bi-
grams in a sentence
• Calculate top-word score
with tf—idf assigning the
score at the specific POS-
tag as a weight
• MPQA subjectivity
lexicon to assign to POS--
tags sentiment polarities
• Classification Features:
uni- & bi-grams with
weights & sentiment
polarities
• Weka library to build
Stance Classification
module
• Selection of Random
Forest classifier as
engine after cross-
evaluation
• Topic Stance as
predicted class with
values for or against
• Classify topics' stance of
each sentence in each
datasets as for or
against
• Identify overall topic
stance of each post of
the datasets by
summing up the for-
stances and the against-
stances
• Summarization to
determine the overall
stance across all posts of
each empirical study
29

30. Greek Bailout Referendum
for end of policy
cycleagainst 30

31. Dutch Ukraine-EU Association
Agreement
for end of policy
cycleagainst 31

32. UK EU Membership Referendum
for end of policy
cycleagainst 32
Agenda Setting

33. Hungarian Migrant Quota
Referendum
for end of policy
cycleagainst 33