Detecting subtle text manipulations: A cross-article analysis chasing the signals of media framing. This talk is part of the KMi seminars series http://kmi.open.ac.uk/seminars/3505 Watch the presentation here: from minute 43 https://www.facebook.com/1218423921581149/videos/205112747432649/

1. Detecting subtle text manipulations
A cross-article analysis chasing the signals of media framing
Date: 25 March 2020
Author: Martino Mensio
1

2. Information disorder
https://firstdraftnews.org/latest/fake-news-complicated/ 2

3. Media framing and bias
• Selection of details (Agenda setting [1]):
– Omitting details
– Rescale importance
• Specific word choices
• Mix of factual and subjective
• Argument distortion
3[1] Cohen, B.C., 2015. Press and foreign policy. Princeton University Press.

4. The need for cross-article analysis
Acknowledge: framing cannot be avoided
Research questions:
• Can we help the reader being aware of the framing
around the piece of news that is being consumed?
• Can we enrich the study of media framing with a
comparative analysis?
4

5. Two disjoint areas of research
5
+ Document clustering (e.g., news aggregators)
+ Corroboration and omissions of information [3]
+ Plagiarism detection
+ Semantic frames [2]
+ Structural role [4]
+ Sentiment and subjectivity [5]
- Analysis of differences is left to the reader - One article at a time
Document relationships Media framing analysis
Cross-article framing analysis [7]
+ Main focus change
+ Ordering
+ Selection of details
+ Framing differences
[2] Charles J Fillmore. Frame semantics. Cognitive linguistics: Basic readings, 34:373–400, 2006.
[3] Bountouridis, D., Marrero, M., Tintarev, N. and Hauff, C., 2018. Explaining credibility in news articles using cross-referencing. In SIGIR
workshop on ExplainAble Recommendation and Search (EARS).
[4] Zahid, I., Zhang, H., Boons, F. and Batista-Navarro, R., 2019. Towards the Automatic Analysis of the Structure of News Stories. In Text2Story@
ECIR (pp. 71-79).
[5] Liu, B., 2010. Sentiment analysis and subjectivity. Handbook of natural language processing, 2(2010), pp.627-666.
[6] Mensio, M.; Alani, H. and Willis, A., 2020. Towards a Cross-article Narrative Comparison of News. In: Proceedings of the Text2Story’20
Workshop, CEUR WS.
http://oro.open.ac.uk/69887/

6. Similarity
6

7. Similarity - Objective
Similarity applied at different levels:
• Article: same events (e.g., News aggregators)
• Sentence: the same detail (e.g. [3])
• Word: find specific words
Resistant to:
• Changes in the linguistic surface
• Changes in framing
7
[3] Bountouridis, D., Marrero, M., Tintarev, N. and Hauff, C., 2018. Explaining credibility in news articles using cross-referencing. In SIGIR
workshop on ExplainAble Recommendation and Search (EARS).

8. Similarity applied
Documents
Document
vectors
Documents
adjacency
matrix
Documents
Graph
[7] Devlin, J., Chang, M.W., Lee, K. and Toutanova, K., 2018. Bert: Pre-training of deep bidirectional transformers for language understanding.
arXiv preprint arXiv:1810.04805.
[8] Cer, D., Yang, Y., Kong, S.Y., Hua, N., Limtiaco, N., John, R.S., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C. and Sung, Y.H., 2018.
Universal sentence encoder. arXiv preprint arXiv:1803.11175.
[9] Johnson, J., Douze, M. and Jégou, H., 2019. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data. 8
Sentences Sentence
vectors
Sentences
adjacency
matrix
Sentences
Graph
Sentencisation
•similarity
•Indexing [9]
•Threshold
•Cliques
Embedding [7,8]

9. Example: sentence-sentence similarity
First article: https://www.bbc.co.uk/news/uk-northern-ireland-51478855
Second article: https://news.sky.com/story/lyra-mckee-man-charged-with-murder-of-journalist-in-northern-ireland-11932429
9

10. Example: word-word similarity
First article: https://www.bbc.co.uk/news/uk-england-hereford-worcester-51791346
Second article: https://www.dailymail.co.uk/news/article-8088805/Britons-facing-heavy-downpours-four-inches-rain-50mph-winds-set-batter-UK.html
10

11. Example: word-word similarity
1: “Coronavirus: Strict new curbs on life in UK announced by PM” https://www.bbc.co.uk/news/uk-52012432
2: “The moment a British prime minister put the whole nation under house arrest”
https://www.independent.co.uk/voices/lockdown-coronavirus-boris-johnson-address-statement-social-distancing-isolate-a9420131.html
• Shopping trips should be as infrequent as possible • One form of exercise a day such as a run,
walk, or cycle.
• We are allowed out to buy necessities – “as infrequently as possible” – and to do our state-
approved exercise – once a day.
11

12. Media Framing Signals
12

13. Framing indicators
• Semantic frames [2]
• Structural role [4]
• Subjectivity / sentiment strength [5]
13
[2] Charles J Fillmore. Frame semantics. Cognitive linguistics: Basic readings, 34:373–400, 2006.
[4] Zahid, I., Zhang, H., Boons, F. and Batista-Navarro, R., 2019. Towards the Automatic Analysis of the Structure of News Stories. In Text2Story@
ECIR (pp. 71-79).
[5] Liu, B., 2010. Sentiment analysis and subjectivity. Handbook of natural language processing, 2(2010), pp.627-666.

14. That’s all for now…
https://twitter.com/MartinoMensio
14
Similarity
Framing
signals
Evaluation