1. Online Forums vs. Social Networks: Two
Case Studies to support eGovernment
with Topic Opinion Analysis
Timo Wandhöfer, Beccy Allen, Steve
Taylor, Paul Walland, Sergej Sizov
IFIP EGOV13, University of Koblenz

2. Contents
• Background & Context
• Applied Process Model
• Online Forums: The HeadsUp Case
• Social Networks: The Facebook/Twitter Case
• General Findings
• Conclusions & Further Work

3. Background

4. Context:
WeGov EU-Project
• Allow policy makers to interact directly with
citizens using Social Network Sites
– Use the tools the citizens already use
• Find and understand
people’s opinions
• Become part of the
discussion
• Open dialog
• Respect privacy
• Encourage trust

5. Problem Statement
• In many cases, discussion tracks in social
media become long and complex
• Stakeholders (WeGov) are often interested in
gaining a quick overview of a discussion
• However, completely reading hundreds of
posts is too time-consuming to be practical
• There is thus a huge need to summarize the
discussion tracks…

6. WeGov Toolbox: Topic
Opinion Analysis

7. Is this really useful within
everyday life?

8. Applied Process Model

9. Online Forums:
The HeadsUp Case

10. Online Forums
• Civil society groups run forums and blogs to
connect with their members and supporters
• Analysing the themes of discussions is often
beyond the organizations resources

11. HeadsUp
• Online forums for connecting young people with
politics and politicians
• Forums ran for a decade (2003-2013) by the
Hansard Society
• They racked up nearly 10,000 comments from
students in 50 forums
• 330 parliamentarians & decision-makers have
taken part in debates with 11-18 year olds
• HeadsUp users' views have been mentioned in
debates in both the House of Commons and the
House of Lords

12. Methodology
• Evaluating the usefulness (accuracy, reliability) with
regard to forum data using a real world data set
• Independent data (created beforehand)
• Each forum was analyzed manually within a report
• Three forums of different sizes:
– Sex Education – Do you get enough? (36 posts)
– Youth Citizenship Commission: are young peopled
allergic to politics? (317 posts)
– How equal is Britain? (1186 posts)
• Toolbox output was compared to the forum reports
– Themes of the debate, sentiment, user interface

13. Findings
• The toolbox is best at dealing with large quantities of
data, amounts that could not be analyzed effectively
by a human.
• The toolkit performs well on relatively in-depth data -
this lends itself to blogs and forums that encourage
more considered and less immediate responses.
• The toolbox also performed well in showing the
nuances between different elements of a wider
debate. E.g. the subtopics for “How equal is Britain?”
• The toolkit works best when analyzing medium
length comments that focus on one issue and when
spelling and vocabulary are good.

14. Social Networks:
Facebook/Twitter Case

15. Methodology:
Intention
• The intention of this case study was the
validation of usefulness of topic opinion
analysis of social media for politics.
• Therefore we designed use cases how this
technology may support politicians’ everyday
life.

16. Methodology:
Real Life Applications
1. Local Facebook topics:
– Monitoring a sample of at least ten Facebook pages
represent a geographical area like an MP’s constituency.
– Apply topic opinion analysis to extract the topics that people
discuss on the pages.
– Each topic is a combination of words that represents a
theme of the discussion, and comes with key users, and key
comments.
2. Monitoring topics on Twitter:
– Identify subtopics e.g. the general debate on climate change
covers subtopics like green energy, new kinds of
technologies.
– Because the results are already filtered by the search, the
analysis produce subtopics.

17. Methodology:
Stakeholders
• Conducted with a number of governmental
representatives as end-user stakeholders:
– two members of the German Bundestag
– four employees that work directly for a member of
the German Bundestag
– two members of the State Parliament North Rhine-
Westphalia
– one small German city (Kempten)
– one big German city (Cologne)
– and with a German state chancellery (Saarland)

18. Methodology:
Qualitative Workflow
• Input: Crawling of individual data (themes
and constituency)
• Analysis: Topic Opinion Analysis
• Report: Reports covering an individual
analysis
• Questionnaire: Using individual examples
from the report
• Follow-up interview: Conducted to receive
more in-depth assessments about the results

19. Findings
• Sensible and expected topics
– All topics from local Facebook pages that were assessed as
understandable were known beforehand; the reasons being:
WeGov stakeholders are ‘aware of the (online) public area’
– Equal with Twitter
• Quality of topics
– Better results with Twitter
– Less clear topics with local Facebook pages. The reason why
42% of the 110 topics were assessed as understandable topics
is due to the fact that policy makers know what’s happening in
the area of their electorate
• Different meanings for topics
– the combination of five words for one topic could have
multiple meanings

20. General Findings

21. Potential end-user
groups
• The toolbox could play an important role in
helping to understand feedback across a range
of communication channels
– small not-for-profit organizations
– larger media organizations
– as well as politicians and policy
(parties, parliaments, bigger city administrations)
• Anyway: Experiences with social media and as
well a social media strategy in general are
necessary!

22. Improvements
• A plain explanation of how the algorithm understands and
processes data -> ensure users trust the results. An explanation
of irregularities. E.g.
– Why the same data sometimes yields different results?
– Why keywords appear in the order and frequency that they do?
• Showing the hidden workings of the toolkit. E.g.
– The relative influence of a greater number of key words e.g. via a tag
cloud.
– Highlighting positive/negative words that contribute to sentiment
scores.
• Implementing more options for the users to refine the data and
customize it to their situation and needs. E.g.
– The ability to exclude certain posts or words from analysis.
– Splitting up long posts into sections that can be analyzed separately
to avoid the conflicting analysis of longer posts.

23. Conclusions
• Two case studies how to apply and validate topic opinion analysis
– HeadsUp focused on accuracy/reliability
– Social network case focused on the usability within the decision-
makers’ everyday life
• Follow the same process model and show added value / boarder-
lines
• Within both cases there was already an understanding of what
the data were about – we call it the „control group“
• Toolbox performed well in showing the nuances between
different elements of a wider debate
• With local Facebook pages the quality of results worsen due to
the fact of the quality of input data and less political
conversations
• Qualitative validating process was very effective but time
consuming
• The Cases protect the rights and privacy of citizens