The presentation at Nick Ruest (York University) and I gave at the Association of Canadian Archivists' annual meeting in Victoria, BC on 26 June 2014.

1. The Great WARC
Adventure
Nick Ruest (York University)
Ian Milligan (University of Waterloo)

2. Today’s Talk
* A brief historical overview of web archiving
* How to capture and preserve
* Discoverability and usability (with open source
software)
* Interplay of the archivist and the historian
* Piecing the story together
* Internet Archive isn’t the only way

3. Historical overview of
web archiving

4. Going back to 1995/1996
We do not know today what Mozart sounded
like on the keyboard ... What will future
generations know of our history? ... But digital
technology seemed to come to the rescue,
allowing indefinite storage without loss. Now we
find that digital information too, has its dark
side. (Michael Lesk, 1995)

7. The Internet Archive

8. But that’s not all...

9. and… [http://freedaleaskey.plggta.
org/]

10. How to capture and
preserve

13. https://digital.library.yorku.ca/yul-92640/biafra
WARC?
https://digital.library.yorku.ca/yul-92640/biafra

14. PROVENANCE!

18. wget --mirror --page-requisites --warc-
file=THIS_AWESOME_SITE http:
//thisawesomesite.ca
Light weight

19. Heritrix
Industrial strength

20. using open source software
Discoverable and usable

22. $$$

23. Can we do this with Open
Source software?

24. Drupal
Fedora Commons
Islandora
Heritrix
wkhtmltopdf
wkhtmltoimage
Use the tools you know

25. Can we talk OAIS?

26. Heritrix
Wget
wkhtmltopdf
wkhtmltoimage
...and Bash!
SIP - Submission Information Package

27. Drupal
Islandora
Fedora Commons
Web ARChive SP
Checksum
Checksum checker
Premis
FITS
AIP - Archival Information Package

28. Drupal
Islandora
Fedora Commons
Web ARChive SP
DIP - Dissemination Information Package

29. Interplay of the archivist
and historian

30. An Ideal Case Study?

31. A Historian in the Archive
* WARC-Tools (https:
//code.google.
com/p/warc-tools/)
* By Date (awesome!)

32. A Historian in the Archive
* Don’t read the
comments
* Disqus

33. A Historian in the Archive

34. Learning from Word
Frequency

36. A Historian in the Archive (distant
reading)

37. A Historian in the Archive (distant
reading)

38. A Historian in the Archive (distant
reading)

39. Keywords = Gotta Know
What You’re Looking For

41. A Historian in the Archive (distant
reading) - search ‘edwin mellen’

42. A Historian in the Archive (distant
reading) - search ‘librarians’

43. A Historian in the Archive (distant
reading) - search ‘dale askey’

44. Problem: still need to
know what you’re looking
for!

45. A Historian in the Archive (distant
reading)

46. A Historian in the Archive (distant
reading)

47. Helps to piece the story
together from massive
web archives

48. Internet Archive isn’t the
only way!

49. …but they created the
Web Archiving Lifecycle

52. Thanks!
Nick Ruest: ruestn@yorku.ca / @ruebot
Ian Milligan: i2milligan@uwaterloo.ca / @ianmilligan1