Sjterp ds_of_misinfo_feb_2019
•Descargar como PPTX, PDF•
1 recomendación•145 vistas
Data science of misinformation - CUNY gradschool class, 2019-02-28
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Sjterp ds_of_misinfo_feb_2019
Similar a Sjterp ds_of_misinfo_feb_2019 (20)
Más de bodaceacat
Más de bodaceacat (20)
Último
Último (20)
Sjterp ds_of_misinfo_feb_2019
- 1. Bodacea Light Industries, 2019 Data Science of Misinformation Sara-Jayne “SJ” Terp February 2019 1
- 2. Bodacea Light Industries, 2019 Yeah. Me. • Lifetime includes: Big data. Submarines. Robots. Cold War information systems. Large-scale data systems. Traffic. Human-robot cooperation. Crisismapping. Developing world community-driven data. Consultant to fortune100s. Pyrotechnics. Adtech. Some other stuff. • Has a ‘big ideas’ habit (last one was “change the way NGOs use data”). Current big idea is that misinformation control is very similar to infosec, it’ll develop in a similar way, and we’ll need to use similar (ongoing forever) patterns 2
- 3. Bodacea Light Industries, 2019 What I do all day • URL-based misinformation: • Global Disinformation Index: data science • Message-based misinformation: • Credibility Coalition: Misinfosec WG • Sofwerx/Arizona: misinfo red team exercise • Sofwerx: misinfo alerting design • Misinfosec: community • MLsec, book etc 3
- 4. Bodacea Light Industries, 2019 (my) Misinformation Hacking • Understand it • Try to stop it happening • Know when someone’s trying to do it • Know when someone’s done it • Respond to it • (try to) stop it happening again 4
- 5. Bodacea Light Industries, 2019 Misinformation It’s not all fake news 5
- 6. Bodacea Light Industries, 2019 Misinformation 6
- 7. Bodacea Light Industries, 2019 Individuals: fake events 7
- 8. Bodacea Light Industries, 2019 Communities: diverted crisis efforts 8
- 9. Bodacea Light Industries, 2019 Nationstates: Qanon campaigns 9 “Action: continuous barrage of memes. All SM platforms Hashtags: #HRCvideo #releasethevideo #maga #QAnon Use top trending hashtags along with your posts. Share and retweet as much as possible”
- 10. Bodacea Light Industries, 2019 Actors, Motivations • State/nonstate actors • Entrepreneurs • Grassroots groups • Private influencers 10
- 11. Bodacea Light Industries, 2019 Social Engineering 11
- 12. Bodacea Light Industries, 2019 At Scale 12 Facebook group total_shares interactions Facebook.com/Blacktivists 103,767,792 6,182,835 Facebook.com/Txrebels 102,950,151 3,453,143 Facebook.Com/MuslimAmerica 71,355,895 2,128,875 Facebook.Com/Patriototus 51,139,860 4,438,745 Facebook.Com/Secured.Borders 5,600,136 1,592,771 Facebook.Com/Lgtbun 5,187,494 1,262,386
- 13. Bodacea Light Industries, 2019 Targeting all social sites 13
- 14. Bodacea Light Industries, 2019 Targeting everyone 14
- 15. Bodacea Light Industries, 2019 (Maybe) Coordinated 15
- 16. Bodacea Light Industries, 2019 Defences “Isn’t it just like spam?” 16
- 17. Bodacea Light Industries, 2019 Sources 17
- 18. Bodacea Light Industries, 2019 Artefacts: Content 18 • Co-occurring hashtags • Correlated text • URLs • Stories
- 19. Bodacea Light Industries, 2019 Stories 19
- 20. Bodacea Light Industries, 2019 Artefacts: Context 20 • Known botnets/trolls • Previous rumours • friends/followers • retweets/likes • Metadata (e.g. DNS)
- 21. Bodacea Light Industries, 2019 Money 21
- 22. Bodacea Light Industries, 2019 Respond 22
- 23. Bodacea Light Industries, 2019 Individual: report trolls/botnets 23 “Twitter (reportedly) suspended over 70 million accounts” “Facebook created a human crisis team after algorithms failed it”
- 24. Bodacea Light Industries, 2019 Individual: report fraud 24
- 25. Bodacea Light Industries, 2019 Individual: block 25
- 26. Bodacea Light Industries, 2019 Platforms • Remove non-human traffic • Rate-limit / shadowban trolls • Remove pages from ad exchanges • Remove non-human traffic from ad exchanges 26
- 27. Bodacea Light Industries, 2019 Community: Engage 27
- 28. Bodacea Light Industries, 2019 Individual: Repair 28
- 29. Bodacea Light Industries, 2019 Community • Parody-based counter-campaigns (e.g. riffs on “Q”) • SEO-hack misinformation sites • Dogpile onto misinformation hashtags • Divert followers (typosquat trolls, spoof messaging etc) • Identify and engage with affected individuals • Educate, verify, bring into the light 29
- 30. Bodacea Light Industries, 2019 Adaptations The game is changing all the time 30
- 31. Bodacea Light Industries, 2019 Attacks are adapting all the time 31
- 32. Bodacea Light Industries, 2019 Offense: Potentials for Next • Algorithms + humans attack algorithms + humans • Shift from trolls to ‘nudging’ existing human communities (‘useful idiots’) • Subtle attacks, e.g. ’low-and-slows’, ‘pop-up’, etc • Massively multi-channel attacks • More commercial targets • A well-established part of hybrid warfare 32
- 33. Bodacea Light Industries, 2019 Defence: Potential for next • Strategic and tactical collaboration • Trusted third-party sharing on fake news sites / botnets • Misinformation version of ATT&CK, SANS20 frameworks • Algorithms + humans counter algorithms + humans • Thinking the unthinkable • “Countermeasures and self-defense actions” 33
- 34. Bodacea Light Industries, 2019 The Datascience part And now we go digging 34
- 35. Bodacea Light Industries, 2019 Master “trackers” • Kate Starbird • Ben Nimmo / #DigitalSherlocks • @Conspirator0 • Josh Russell (@Josh_Emerson) • Iwr • … 35
- 36. Bodacea Light Industries, 2019 @katestarbird 36
- 37. Bodacea Light Industries, 2019 #digitalsherlocks 37
- 38. Bodacea Light Industries, 2019 @josh_emerson 38
- 39. Bodacea Light Industries, 2019 @Conspirator0 39
- 40. Bodacea Light Industries, 2019 @r0zetta (Andy Patel) 40
- 41. Bodacea Light Industries, 2019 @fs0c131y (Elliot Alderson) 41
- 42. Bodacea Light Industries, 2019 www.iwr.ai 42
- 43. Bodacea Light Industries, 2019 Doing the thing Getting into misinformation data science 43
- 44. Bodacea Light Industries, 2019 Asking questions • Is there unusual activity on hashtag x, topic y, platform z? • What are ‘known’ bots talking about today? • What’s the chatter in 8chan/ 4chan/ r/thedonald RussiaToday etc • What are the misinformation creators trying to do? What artifacts are they likely to leave when they do it? • What are the other trackers getting excited about today? 44
- 45. Bodacea Light Industries, 2019 Getting your own data Trollbot lists: • https://botsentinel.com/ Tools: • https://github.com/IHJpc2V1cCAK/socint • https://labsblog.f-secure.com/2018/02/16/searching-twitter-with- twarc/ Existing datasets • https://github.com/bodacea/misinfolinks 45
- 46. Bodacea Light Industries, 2019 Trollbots 46
- 47. Bodacea Light Industries, 2019 Code 47
- 48. Bodacea Light Industries, 2019 Artifacts: Images 48
- 49. Bodacea Light Industries, 2019 Artefacts: text 49
- 50. Bodacea Light Industries, 2019 Artefacts: message times 50
- 51. Bodacea Light Industries, 2019 Artefacts: Relationships 51
- 52. Bodacea Light Industries, 2019 And then the rest of the DS cycle • Explore: Go play. Pull some troll data, and think about what you’d like to know about it. Look at the hours the trolls tweet at, the topics, the hashtags. Do they repeat each other at all? Are there patterns? Think about names, dates, followers/following, profiles. Are they on existing “naughty lists”. Etc • Model • Iterate • Explain (who to?) 52
- 53. Bodacea Light Industries, 2019 Thank you SJ Terp @bodaceacat 53
- 54. Bodacea Light Industries, 2019 Bonus: Misinfo + Infosec What’s already happening here? 54
- 55. Bodacea Light Industries, 2019 Infosec support to misinfo tracking 55
- 56. Bodacea Light Industries, 2019 Include misinfo in infosec definitions? 56 “Prevention of damage to, protection of, and restoration of computers, electronic communications systems, electronic communications services, wire communication, and electronic communication, including information contained therein, to ensure its availability, integrity, authentication, confidentiality, and nonrepudiation” - NSPD-54
- 57. Bodacea Light Industries, 2019 Mapping Parallels • As Information Security (Gordon, Grugq) • Via Information Operations / Influence Operations (Lin etc) • As a form of conflict 57
- 58. Bodacea Light Industries, 2019 Mapping frameworks 58
- 59. Bodacea Light Industries, 2019 Misinfo red team exercises 59
- 60. Bodacea Light Industries, 2019 Frameworks How do we adapt these standards? 60
- 61. Bodacea Light Industries, 2019 Infosec already includes cognitive 61
- 62. Bodacea Light Industries, 2019 Stage-based models are useful 62
- 63. Bodacea Light Industries, 2019 Mapping Phases to Techniques is really useful 63
- 64. Bodacea Light Industries, 2019 Each Tactic stage has a pick-list of Techniques 64
- 65. Bodacea Light Industries, 2019 Zooming out (aka naming things is hard) • Campaigns : Advanced persistent threats • e.g. Internet Research Agency, 2016 elections • Incidents • e.g. Columbia chemicals • Failed attempts • ? 65
- 66. Bodacea Light Industries, 2019 2014 Columbian Chemicals incident 66 • Summary: Early Russian (IRA) “fake news” stories. Completely fabricated; very short lifespan. • Actor: probably IRA (source: recordedfuture) • Timeframe: Sept 11 2014 (1 day) • Presumed goals: test deployment • Artifacts: text messages, images, video • Method: • 1. Create messages. e.g. “A powerful explosion heard from miles away happened at a chemical plant in Centerville, Louisiana #ColumbianChemicals” • 2. Post messages from fake twitter accounts; include handles of local and global influencers (journalists, media, politicians, e.g. @senjeffmerkley) • 3. Amplify, by repeating messages on twitter via fake twitter accounts • Result: limited traction • Counters: None seen. Fake stories were debunked very quickly. • Related attacks: These were all well-produced fake news stories, promoted on Twitter to influencers through a single dominant hashtag -- #BPoilspilltsunami, #shockingmurderinatlanta,
- 67. Bodacea Light Industries, 2019 Tactics (draft, very draft) 67
- 68. Bodacea Light Industries, 2019 Techniques (draft) 68
- 69. Bodacea Light Industries, 2019 Example Technique • Behavior • Requirements • Cause • Effects 69
- 70. Bodacea Light Industries, 2019 Why Build This? • Understand badguys: create red team exercises • Build an alert structure (cf US-CERT) • Defend/countermove against reused techniques • Transfer other infosec principles • Identify gaps in attacks • Assess defence tools & techniques • Plan for large-scale adaptive threats (hello, Machine Learning!) 70
- 71. Bodacea Light Industries, 2019 We also need to design response 71
- 72. Bodacea Light Industries, 2019 Your part: don’t fight the last war 72
Notas del editor
- Misinformation is deliberately false information. One example is the “fake news” sites above, containing misinformation that’s used to gain advertising money, with clickbait tweets that bring people to them. Some of these currently contain the typical aliens and healthcure material, but many are political and trading on strong emotions like fear and useful divisions in society. Image: screenshot of http://www.sawthis.one/ 2018-07-08
- Misinformation is also moving from online to offline. Several times now, misinformation actors have sent invites to opposing groups to demonstrate at the same time in the same place. https://twitter.com/JuliaDavisNews/status/994704834577215495 https://twitter.com/donie/status/957246815056908288
- Misinformation is information that’s deliberately false (actually that’s disinformation, but “misinformation” as a term won). The smallest form of online misinformation is ‘joke’ viral content, for example in every disaster there’s someone who puts up an image of a shark in the street. Image: http://www.politifact.com/truth-o-meter/statements/2017/aug/28/blog-posting/there-are-no-sharks-swimming-streets-houston-or-an/ and pretty much any major US disaster
- And then, if you look, you can find organising pages for campaigns. Here are two Qanon “meme war organising page”. Qanon is a major group, but is just one of many. Note that this is from March/April, and has a specific date on it, targetting a specific event.
- Familiarity backfire effect Memory traces Emotions = stronger traces Here are some common brain vulnerabilities. My favourites are the familiarity backfire effect, where if you repeat a message with a negative in it, people remember the message without the negative, and that when people read, they take false information in as true before rejecting it - and in that fraction of a second, build other assertions off the false information, even if they *know* the original information is false.
- Online misinformation is huge. A few hundred trolls and thousands of bots can affect millions of people at a time. This is the scale that nationstate-run groups and pages, dedicated to creating division and confusion, typically work at. Here are some of the Russian-owned Facebook groups shown to Congress: these high volumes of shares and interactions might include a lot of botnet activity, but are still not insignificant.
- This stuff is everywhere online: the expected places (FB, twitter, reddit, eventbrite, medium etc) but also comment streams, payment and event sites.
- Social media buys reach and scale. 100 good bots = long game; 10000 ba ones = short but effective You can also use other advertising techniques, and things like that familiarity backfire. Botnets are very useful for this, and very cheap, at about $150 for a difficult-to-find “aged” set, to a few dollars per thousand for Russian recent bots. Buy the bots, use any of the handy online guides to set them up messaging or retweeting etc, or use some simple pattern matching or AI to make them harder to find.
- One big weakness for attackers is that they have to tell you about themselves. They leave a lot of “artefacts” - ways to find them. botsentinal.com
- Here are some of them, including hashtags, URLs, adverts. A simple media search with twitter, tweetdeck etc will find a lot of these. On the right are the artifacts tracked as part of the Canadian elections.
- There’s also a lot of content in fact check sites(Snopes etc); if you have the resources, then it’s also possible to pay someone to go look at an area being discussed. Sometimes misinformation propagation is more subtle. These are a good place to look for that too.
- Here are some of them, including hashtags, URLs, adverts. A simple media search with twitter, tweetdeck etc will find a lot of these. On the right are the artifacts tracked as part of the Canadian elections.
- You *can* report to platforms. So far this has been pretty underwhelming, but if we did it at scale, it could be interesting. What would be good in an ideal system includes: Realtime botnet removal Realtime troll dampening Etc But that’s not where we are, so here’s some others.
- Two things: advertising works by putting adverts into slots on pages. We can track unlabelled political ads, we can see the fakenews pages and pages associated with them, and we can see botnets going to pages to drive up their ad revenue. For communities, you can report ads on fake pages to brands.
- And as an individual, there are still things you can do. One of these is to work with other people to block misinformation sources and channels. Many anti-harassment apps can be repurposed for this.
- My favourite communities are the Lithuanian elves. Formed as an anonymous online group. They fight back every day against Russian misinformation, using a combination of humour and facts. It seems to be working. Other cool things to do include overwhelming misinformation hashtags with other content, and hacking search terms to make disambiguation pages appear above misinformation sites. Another group that’s got some traction is VOST (Virtual Operation Support Team), a team that supports responders in disasters: VOST Panama also used humour and “fake stamps” to counter misinformation, and helped me run a deployment on this during Hurricane Irma (when people also reported misinformation to Fema and Buzzfeed).
- You can also help in rebuilding damaged communities: this is The Commons Project, that uses a combination of bots, humans and peace techniques for this.
- https://medium.com/s/story/the-trolls-within-how-russian-information-operations-infiltrated-online-communities-691fb969b9e4
- https://medium.com/dfrlab/digitalsherlocks/home
- https://medium.com/@josh_emerson/ira-midterms-part-two-collection-of-russian-troll-factory-instagram-memes-5b3492108aa6 Josh is good pointer to other people: Meet the Indiana dad who hunts Russian trolls - CNNPolitics - also https://twitter.com/josh_emerson medium.com/@josh_emerson and eye_josh (u/eye_josh) - Reddit
- https://twitter.com/conspirator0/status/1098987451320659968
- https://www.iwr.ai/voterfraud/index.html
- https://botsentinel.com/top-100
- Andy Patel’s twitter-fetching code
- Image: FireEye tracking Iranian misinformation artefacts
- https://www.lawfareblog.com/cybersecurity-time-new-definition https://fas.org/irp/offdocs/nspd/nspd-54.pdf
- cyber attack lifecycle, with ATT&CK phases * Persistence – Any access, action, or configuration change to a system that gives an adversary a persistent presence on that system. Adversaries will often need to maintain access to systems through interruptions such as system restarts, loss of credentials, or other failures. • Privilege Escalation – The result of techniques that cause an adversary to obtain a higher level of permissions on a system or network. Certain tools or actions require a higher level of privilege to work and are likely necessary at many points throughout a remote operation. • Defense Evasion – Techniques an adversary may use for the purpose of evading detection or avoiding other defenses. • Credential Access – Techniques resulting in the access of, or control over, system, domain, or service credentials that are used within an enterprise environment. • Discovery – Techniques that allow an adversary to gain knowledge about a system and its internal network. • Lateral Movement – Techniques that enable an adversary to access and control remote systems on a network. Often the next step for lateral movement is remote execution of tools introduced by an adversary. • Execution – Techniques that result in execution of adversary-controlled code on a local or remote system. • Collection – Techniques used to identify and gather information, such as sensitive files, from a target network prior to exfiltration. • Exfiltration – Techniques and attributes that result or aid in an adversary removing files and information from a target network. This category also covers locations on a system or network where an adversary may look for information to exfiltrate. • Command and Control – Techniques and attributes of how adversaries communicate with systems under their control within a target network. Examples include using legitimate protocols such as HTTP to carry C2 information.
- I’m leading a team working on writing a misinformation equivalent to the ATT&CK TTP framework.
- <Add zoomed-in part of ATT&CK>
- … and we have to start filling these out…
- Image: SANS sliding scale of cyber security https://www.newyorker.com/magazine/2018/05/07/the-digital-vigilantes-who-hack-back
- There are still a lot of bots out there, but tactics, techniques and procedures are changing rapidly: we’re starting to see an early-infosec-style split into script-kiddie style crude botnets and more carefully crafted responsive bots. image: https://medium.com/@MediaManipulation/tracking-disinformation-by-reading-metadata-320ece1ae79b