Talk to the NPCC information officers' conference, June 2018

1. @PaulBradshaw, Birmingham City University, BBC
Author: Scraping for Journalists, Finding Stories in Spreadsheets, Data
Journalism Heist, Online Journalism Handbook
The ghost of FOI
yet-to-come

2. What I do, what data journalists do,
what other journalists do with FOI
Where FOI is (and isn’t) now
What next: APIs, big data, AI and
other acronyms
The ghosts of FOI past,
present and yet-to-come

3. https://leanpub.com/u/paulbradshaw
https://onlinejournalismblog.com/

4. FOI, data
journalism — and
other journalisms
.

5. Paul Bradshaw, Data Journalism Handbook
http://datajournalismhandbook.org/1.0/en/introduction_0.html

7. Why? How?
● Increasing data availability: FOI, open
data, big data
● Lower barriers to production: free
tools, learning communities
● Biggest stories of the age: MP
expenses, Wikileaks, Panama Papers
● Cultural change: Hacks/Hackers, civic
coding, open data movement

8. https://newslab.withgoogle.com/assets/docs/data-journalism-in-2017.pdf

9. “The BBC has a plan to develop
data journalism as a cornerstone of
its reporting and analysis in all
areas of News coverage.”
BBC

10. Sense checkers
“[I had] a reputation for being someone
who sort of poured cold water on what
looked like really promising stories …
with a week’s distance they were
probably relieved because it could
have ended up being a bit of a
damaging story [to the publisher]”.
Craig Butt

11. Transparenc
y
https://github.com/washingtonpost/data-homicides

12. BBC England data unit
https://github.com/bbc-data-unit

13. https://github.com/bbc-data-unit

14. (Other data units are also
available)

15. https://projects.propublica.org/graphics/hatecrime-map
Holding (data) to account

16. https://www.bloomberg.com/view/articles/2017-03-08/how-data-can-turn-immigrants-into-criminals

17. https://www.thebureauinvestigates.com/stories/2017-02-27/the-future-of-fake-news

18. https://www.bbc.co.uk/news/uk-england-43703454 https://www.bbc.co.uk/news/uk-england-lancashire-42142799

19. Journalists, FOI and data

20. Things that journalists
get wrong
• Reporting absolute numbers instead of
percentages
• Getting a partial picture due to
non-compliance

22. Things that journalists
get wrong
• Reporting absolute numbers instead of
percentages
• Getting a partial picture due to
non-compliance
• Asking for figures already out there
• Asking for aggregate figures, not
granular
• Not understanding system

32. How to make sure
journalists get it right
• Send info in XLS format (PDF
increases chance of human error)
• Include contextual information, e.g.
population
• List terminology & database fields to
help simplify request
• Don’t publish statistics with hidden
sheets & drop-down menus

33. Is this boilerplate? Reporter will
assume it is.

34. • Criticism of ‘he said, she said’
reporting
• If the figures are wrong, explain
why
• Don’t wait until after publication
(and the right of reply) to say
they’re wrong
The ‘right of reply’ phase

38. The ghost of FOI
yet to come
.

39. Is FOI broken?
• Press officers routinely redirect basic
questions towards FOI
• Skews journalistic scrutiny towards
public bodies (not public money)
• Exemptions applied broadly &
inconsistently
• Law routinely ignored on response
limits

41. “Meh.”

42. A story of an FOI request
“One came back to me 5 months later
saying I needed to narrow the request
and then finally sent the data 2 months
after that. Another made no contact for
9 months when they said they were
having “difficulty extracting the data”
and then emailed again after 11 months
saying they had a backlog”

43. “Some forces have a backlog, staffing
problems etc. - that’s understandable.
But if they know at the time you submit
an FOI request that there’s no chance of
them getting you the data for 6 months
then an automatic note just to say
"we’re massively behind and we won’t
be able to get this to you within 20
days" would be so helpful.”

44. “All of my positive experiences have
involved people being helpful and I can’t
stress enough how great
communication is. One of the forces
who had a backlog and also had
difficulty answering my specific request
because of the way their data was
recorded was actually really helpful
because they explained this.
Talk makes it easier

45. “It meant I was able to adapt the request
to make life easier for the FOI officer
and also it meant there were no hard
feelings when it came through months
late, as I knew it was coming so it was
still useful.”

46. Alternatives: API
• No need for compilation
• Can focus on speaking to
authority, experts etc.
• Information can be queried and
presented live
• Caveats need to be included in
results

47. Hackdays

48. Hack days
• Cross-industry: Bring industry
and journalists together
• Cross-discipline: Journalists,
developers, statisticians, experts
• Constructive: build skills,
knowledge, understanding,
relationships

49. Here come the
robots
.

50. “Each weekday, my computer
program goes to the Chicago
Police Department's website
and gathers all crimes
reported in Chicago.”
Adrian Holovaty
Automation

51. Big data?
• Problem of multiple
comparisons
• Not regular source for
journalists, but sensors could
change this

52. https://www.bbc.co.uk/news/uk-england-40131277

53. AI and machine learning
• Increasing automation in
journalism: ‘robot journalism’ vs
‘augmented journalism’
• It’s already here: Facebook etc.
‘learns’ what you like

54. More at pinboard.in/u:paulbradshaw/t:robotjournalism

55. “.”
Tow Center, Guide to Automated Journalism

56. Tow Center, Computational Campaign Coverage
Yellow: raw data inserted into the text
Purple: calculations with the raw data.
Green: synonyms, used to add variety
“exponentially increases the possible variants for the whole text”

57. Journalists much more data-literate,
increasingly seek context
Work with journalists: FOI is a crude
instrument both for reporters — and
press officers
APIs and open data provide more time
for journalists to speak to authorities
3 things…

58. Questions?
@paulbradshaw
paul.bradshaw@bcu.ac.uk