Presentation to NCRM event DIGITAL METHODS AS MAINSTREAM METHODOLOGY

1. Welcome and introduction to
British Library digital resources
for social scientists
John Kaye – Lead Curator Digital Social Science
Peter Webster - Web Archiving Engagement and Liaison
Manager
7th December 2012
www.slideshare.net/johnkayebl

2. What kind of library are we?
 “We exist for everyone who wants to
do research – for academic, personal
or commercial purposes”
 Our collections cover all known
subject areas; sciences, technology,
medicine, arts & humanities, social
sciences…
 We have a copy of every item
published in the UK
 Our collections cover all formats;
sound, images, video, newspapers,
maps, manuscripts, databases,
books and journals, much more…
2

3. News, newspapers and magazines
3

4. News and current events
Broadcast news, recording from May 2010
Political change in Middle East
Olympic Games
Occupy movement
4

5. Images and photographs
 Images online
 Online gallery
 Photographically illustrated books
5

6. Online Services
6

7. Social Science online resources for researchers
 ESRC online resource
 Management and Business Studies Portal
 Social Welfare Portal
 www.bl.uk/oralhistory
 Social Science blog
7

8. 8

9. Management and Business Studies portal
9

10. 10

11. http://britishlibrary.typepad.co.uk/socialscience/
11

12. Oral history at a glance
www.bl.uk/oralhistory
 370 collections from 1 tape to 5,500 (Millennium Memory Bank)
 100-150 hours of new digital fieldwork recordings per month
 2200 catalogue records added or updated per year
 4000 public enquiries per year
 40 talks and lectures per year
 60 training sessions per year with OHS (500+ people)
12

13. Guides and support
 Reference services: reading room, telephone, email
 Help for Researchers web pages
 Collection guides, eg for government publications:
http://www.bl.uk/reshelp/findhelprestype/offpubs/guides/govtgu
 Topical bibliographies, eg Globalisation and
employment, Gang culture and knife crime, Corporate
Social Responsibility, Far Right in Britain …
 Welfare Reform on the Web
13

14. Exhibitions and events
www.bl.uk/whatson
14

15. Doctoral Open Days 2013
11 February – Social Sciences
18 February – Media, Cultural Studies and Journalism
http://www.bl.uk/whatson/events/docopendays/index.html
15

16. Web archives and digital method
Dr Peter Webster
Web Archiving Engagement and Liaison Officer
@UKWebArchive / @pj_webster
Peter.Webster@bl.uk
http://www.webarchive.org.uk
December 7th 2012

17. The lost web: people
[votedavidcameron.com, (archived 24/5/05)]
17

18. The lost web: people
[robincook.org.uk (archived 8/8/05)]
18

19. The lost web: organisations
[tvpa.police.uk (archived 21/11/12)] 19

20. The lost web: organisations
[woolworthsgroupplc.com (archived 12/12/08)] 20

21. Our mission:
Collect, preserve, and
make accessible
web sites of
cultural and scholarly
importance
from the UK domain

22. UK Web Archive http://www.webarchive.org.uk
 Selective Web Archive
 over 11,000 websites collected since
2004
 over 50,000 instances
 Over 16TB of compressed data
 British Library, National Library
of Wales, JISC
 Also National Library of Scotland,
the National Archives, Wellcome
Library
 Many collaborators
 eg Women’s Library, Live Arts
Development Agency, Quakers in
Britain
22

23. A typical event-based special collection
Collect, preserve, and
make accessible
eb sites of
cultural and scholarly
importance
from the UK domain

24. The orphaned web
24

25. A comprehensive special collection
Collect, preserve, and
make accessible
eb sites of
cultural and scholarly
importance
from the UK domain

26. Web archiving: the basics
 What
 Selecting, capturing, storing, preserving and managing access to snapshots of websites over time
 How
 Use crawler software to download websites automatically
 Selective or domain archiving
 Provide access in a Web Archive
When
 Since mid 1990s
Who
 Heritage and memory organisations, eg BL, The National Archives
 University libraries
 Not-for-profit and commercial organisations, eg Internet Archive
 Individual researchers
 Why
 Global information resource
 Artefact of cultural and technology change
 Representative sample of the web: historical and sociological data that may not be found
elsewhere
 Part of national digital heritage - legal requirements
26

27. Selective versus domain archiving
 Two complementary approaches: selective and domain archiving
Width
D
e Domain harvesting:
p
- Typically once/twice a year
t - Domain wide snapshot
h - Supported by national legislative
framework
Selective archiving: -- automated & cost-effective
- More frequent gathers; manual QA
- Guided by collection policy
- Can be based on events or themes
e.g. credit crunch
-- manual & expensive
27

28. Non-print Legal Deposit 2013: what will we collect ?
A deposit library is entitled to copy UK publications from the
open web.
A deposit library is entitled to collect other password-protected
material by harvesting, subject to giving at least 1 month’s
written notice for the publisher to provide a password or
access credentials.
28

29. What will we be collecting ?
Includes resources:
• that are issued from a .uk or other UK geographic top-level
domain, or
• where part of the publishing process takes place in the UK;
• but excluding any which are only accessible to audiences
outside the UK.
29

30. What will we NOT be collecting ?
Film and recorded sound where the audio-visual content
predominates
Private intranets and emails
Personal data in social networking sites or that are only
available to restricted groups.
30

31. What will users be able to do with it ?
Users may:
• access deposited material while on “library premises
controlled by a deposit library”.
• print one copy of a restricted amount of any deposited
material, for non-commercial research or other defined ‘fair
dealing’ purposes such as court proceedings, statutory
enquiry, criticism and review or journalism.
31

32. What will users NOT be able to do with it ?
Users may NOT:
• use an item simultaneously with another user;
• make any digital copies, except by specific and explicit
licence of the publisher.
32

33. A web archiving strategy based on prioritisation
Domain Crawl
Event Event Event
Domain Events: Special
harvesting: •Political, Collection:
•Broad sweep cultural, social •Focused,
of .uk domain and economic thematic
•Survey and events of collections
discovery national •Support
•Implement interest, eg priority
Legal Deposit Olympics subjects
2012
33

34. JISC UK Web Domain Dataset (1996-2010)
Funded by JISC to create a research collection of UK
websites
 Collaboration between the Internet Archive, JISC and the
British Library
 Copy of subset of the Internet Archive’s web collection that
relates to the UK
 470466 files, mostly arc.gz, with 4494 warc.gz.
 Total size: 32TB
 No local access – possible through the Internet Archive
 Can be used to generate secondary datasets and make
these available
 Analytical access the main route
34

35. Historical Archive – HTML Version Analysis

36. N-Gram Search: Prime Ministers

37. N-Gram Search: Social Media

38. Questions ?
John.Kaye@bl.uk
Twitter: @johnkayeBL
Peter.Webster@bl.uk
Twitter: @UKWebArchive / @pj_webster
UK Web Archive: http://www.webarchive.org.uk
38