Keynote talk on "Music in the Archives: Digital Musicology as a case study in Computational Archival Science" by David De Roure, for the workshop on "Computational Archival Science: digital records in the age of big data" at IEEE Big Data 2020, 11 December 2020.

1. David De Roure
Music in the Archives:
Digital Musicology as a case study in
Computational Archival Science
Oxford e-Research Centre
Alan Turing Institute
@dder

3. Setting the Scene in Scholarship:
Social Machines & Citizen Science
Signals & Salami
Automation, Apparatus & Agency
From Box 170 to the Barbican
Computational Archival Science

4. Edwards,P.N.,etal.(2013)KnowledgeInfrastructures:IntellectualFrameworksand
ResearchChallenges.AnnArbor:DeepBlue.http://hdl.handle.net/2027.42/97552
Knowledge Infrastructures:
Intellectual Frameworks and Research Challenges

5. Digital
Scholarship
Social
Networks
More people
Moremachines
Distributed
Computation
Conventional
Computation
Science 2.0
Data Science
Citizen Science
Scientific Computing
Cyberphysical
Web 2.0
@dder

6. More people
Moremachines Automation
& Machine
Learning
@dder
Digital
Scholarship

7. Social Machines
“Real life is and must be full of all kinds of social
constraint – the very processes from which society
arises. Computers can help if we use them to create
abstract social machines on the Web: processes in
which the people do the creative work and the
machine does the administration... The stage is set
for an evolutionary growth of new social engines.
The ability to create new forms of social process
would be given to the world at large, and
development would be rapid.”
Berners-Lee, Weaving the Web, 1999 (pp. 172–175)

9. Scientists
Talk ForumImage Classification
data reduction
Citizen Scientists

10. In an effort to speed up classifications to cope with the large number of galaxies
we expect to receive from new surveys, we've been working on ways to combine
your classifications with those of machines, inspired by the idea that the
combination of both automatic and human classification may be more powerful
than either alone. If you choose the 'Enhanced' work flow, you will be much more
likely to see the top 100 galaxies our galaxy-classifying robot thinks it needs help
with in order to improve. All galaxies will be seen by at least a few volunteers to
make sure we aren't missing anything. If you'd rather just see a random selection
of available galaxies, choose 'Classic’.
If you choose the 'Enhanced' work flow, you will be much more
likely to see the top 100 galaxies our galaxy-classifying robot thinks it needs help
with in order to improve. All galaxies will be seen by at least a few volunteers to
make sure we aren't missing anything. If you'd rather just see a random selection
of available galaxies, choose 'Classic’.
https://www.zooniverse.org/projects/zookeeper/galaxy-zoo/about/research

11. Setting the Scene in Scholarship:
Social Machines & Citizen Science
Signals & Salami
Automation, Apparatus & Agency
From Box 170 to the Barbican
Computational Archival Science

12. StephenDownie

13. INT. VERSE VERSE VERSE VERSEBRIDGEBRIDGE OUT.
ê
signal
understanding

14. Music Information Retrieval Evaluation eXchange
• Began as MIREX in 2005
• Tasks defined by community debate
• Data sets collected and/or donated
• Participants submit code
• Non-consumptive research
• Sessions convened at the annual International
Conference on Music Information Retrieval (ISMIR)
StephenDownie

15. https://ccrma.stanford.edu/~benwill1/arcs/about.html

16. Community
Software
Supercomputer
Digital Music
Collections
Student-sourced
ground truth
Community
Software
Linked Data
Repositories
Supercomputer
23,000 hours of
recorded music
Music Information
Retrieval Community
SALAMI
2009
DavidDeRoureJ.StephenDownieIchirofujinaga

17. Jordan B. L. Smith, J. Ashley Burgoyne, Ichiro Fujinaga, David De Roure,
and J. Stephen Downie. 2011. Design and creation of a large-scale
database of structural annotations. In Proceedings of the International
Society for Music Information Retrieval Conference, Miami, FL, 555–60.

18. https://github.com/DDMAL/salami-data-public

19. StephenDownie

20. • The world of music has changed for good in the digital age. This revolution
must be matched by a transformation of the means by which music is studied.
• While preserving the best traditional values and practices of musicology we
must take advantage of the immense opportunities offered by music
information retrieval
• Three parallel musicological investigations: 16th-century vocal and lute music,
Wagner's leitmotifs, Musicology of the social media
• Ensure sustainability and repeatability by embedding the above research
activities in a framework enabling data, methods and results to be shared
permanently as Linked Data
• Enhance Semantic Web workflow description methods for musicology

21. TerhiNurmikko-Fuller

22. CarolinRindfleisch

23. Setting the Scene in Scholarship:
Social Machines & Citizen Science
Signals & Salami
Automation, Apparatus & Agency
From Box 170 to the Barbican
Computational Archival Science

24. • A workflow commons for workflow sharing,
designed using Web 2.0 principles
• Launched open beta in November 2007, still
actively used
• Largest public collection of workflows
(3000+), for multiple workflow systems
• 1300+ entries in Google Scholar refer to
myexperiment.org
• Open source, REST API, part of Open Linked
Data cloud (66k triples) - lod-cloud.net
• Introduced “packs” which led to Research
Objects – www.researchobject.org
• Workflow collection studied in scientific
workflow and e-Science communities
CaroleGoble
www.myexperiment.org
De Roure, D., Goble, C. Stevens, R. (2009) The Design and Realisation of
the myExperiment Virtual Research Environment for Social Sharing of
Workflows. Future Generation Computer Systems 25, pp. 561-7.

25. Notifications and automatic re-runs
Machines are users too
Autonomic
Curation
Self-repair
New research?
JCDL 2013

27. Fusing Audio and
Semantic Technologies
www.semanticaudio.ac.uk

28. The Vision: Digital Music Objects and Music Flows
DMOs are rich computational objects that wrap up multiple structured
digital music content types with rich metadata. This ‘musical essence’
is structured in a way that allows it to be dynamically edited, mixed
and combined in the workflows of the music production landscape.
KevinPageJohnPybusGrahamklyneDavidWeigl

29. Compose
Perform
Record
Mix
Distribute
Discover
Listen
Share
Pros
Pro-Ams
Amateurs
Fans
Audiences
Compose
Perform
Record
Mix
Distribute
Discover
Listen
Share
Pros
Pro-Ams
Amateurs
Fans
Audiences
Steve Benford

30. Composer
Emily Howard
creates
‘Numbers into
Notes’ event
format
Numbers into
Notes Web
Software
Audience
Fan
Amateur
Semi-pro
Professional
Com
pose
Perform
Record
M
ix
Distribute
Discover
Listen
Share
Produce Consume
Pro
Casual
Actor
Activity
Learner
Recordings
available on
Soundcloud
Live performance
in Bethnal Green
People
generate
music
fragments
Film available on
YouTube
Fragments
used by
professional
composers
Music for film
soundtrack
Composition for
Tate Modern video
Published on
YouTube
‘Numbers into Notes’
event using software
Automated
reuse in SOFA
remixer
Spatial demo
in Florence

31. numbersintonotes.net

32. GrahamKlyne

33. This piece brings together, participation, algorithmic composition
andaugmentation (as a mechanism by which people can work together to
augment and support a composer’s workflow). It expands upon Chamberlain’s
work into compositional practices that explore autonomy and control, and
builds upon the Numbers into Notes system as developed by De Roure.
The piece uses the symbolism of the gift to frame parts of the interactions
that have occurred in the development of the piece. Individuals are given the
chance to create an algorithm. This is made into a physical entity, which is
then gifted to the composer; these together are combined and used to
compose a piece. The piece is then performed and given back to the audience
(live), of which some members have created the original algorithms.
https://soundcloud.com/alain_du_norde/the-gift-the-algorithm-beyond-autonomy-and-control
The performance creates a gift, a souvenir, a memento of the experience which some of the audience
members can take away. The performance also acts as a way in which we can also understand the
interplay between algorithms, art, performance, provenance and participation.
The gift of the algorithm: beyond autonomy and control
Alan Chamberlain, David De Roure
PhotoPipWillcox

34. Setting the Scene in Scholarship:
Social Machines & Citizen Science
Signals & Salami
Automation, Apparatus & Agency
From Box 170 to the Barbican
Computational Archival Science

35. Barbican,9Mar2019
The Eternal Golden Braid
RNCM,13June2019
The Voice of the Machine
RoyalInstitutionChristmas
Lectures2019
Secret & Lies
The Hidden Power of Maths
Marcus du Sautoy Robert Laidlow
Lecture 3 How can we all win? (at 57mins)

36. Milton Court Concert Hall, 2 Nov 2019
Alter | PRiSM led by Robert Laidlow
Alter was premiered by the Britten Sinfonia and Marta Fontanals-
Simmons, conducted by William Cole, at the Barbican.

37. Ada Lovelace
Charles Babbage
Bodleian Libraries
19th Century AI
21st Century AI
Audience
Human composer
Robert Laidlow
Alter Text – PRiSM team led by Robert Laidlow
Commissioned by Barbican, performed by Britten Sinfonia with
Marta Fontanals-Simmons, conductor William Cole, November 2019.
Mezzo-soprano
Marta Fontanals-Simmons
prompts
Images © Clay Mathematics Institute

38. Robert Laidlow interviewed by Andrew McGregor before the
performance of 'Alter', a piece by PRiSM led by Robert Laidlow.
Part of the event 'Ada Lovelace: Imagining the Analytical
Engine', curated by Emily Howard on 2 November 2019 at
Milton Hall, London.
https://www.robertlaidlow.co.uk/alter

39. Some questions about the AI:
If we train on copyright content, can
we make the model available?
Do we want to archive the training
data, the models, the generated
content? It’s computational archival
content.
Do we need to archive something if we
can reproduce it?

40. Setting the Scene in Scholarship:
Social Machines & Citizen Science
Signals & Salami
Automation, Apparatus & Agency
From Box 170 to the Barbican
Computational Archival Science

41. dcic.umd.edu/june-20-21-2019-uk-us-computational-archival-science-cas-datathon-in-london/
Eirini
Goudarouli
Richard
Marciano
Mark
Hedges
Dave
Beavan
Pip
Willcox
DDeRRyan
Cox
Greg
Jansen
Sonia
Ranade

42. Scholarly Primitives refer to some
basic functions common to
scholarly activity across disciplines,
over time, and independent of
theoretical orientation.
These ‘self-understood’ functions
form the basis for higher-level
scholarly projects, arguments,
statements, interpretations—in
terms of our original,
mathematical/philosophical
analogy, axioms.
John Unsworth
Discovering
Annotating
Comparing
Referring
Sampling
Illustrating
Representing
David De Roure and Pip Willcox. 2020. Scholarly Social Machines:
A Web Science Perspective on our Knowledge Infrastructure. In
12th ACM Conference on Web Science (WebSci '20). Association
for Computing Machinery, New York, NY, USA, 250–256.
DOI:https://doi.org/10.1145/3394231.3397915

43. We become what we behold. We shape our tools and then
our tools shape us.
We become what we behold. We shape our research tools
and then our tools shape our research.
We become what we behold. We shape our archives and
then our archives shape us.
What do we behold in Computational Archival Science?
Because that is what we will become.

44. Supported by:
Theory and Practice of Social Machines (SOCIAM)
EPSRC EP/J017728/2
Transforming Musicology, AHRC
Fusing Audio and Semantic Technologies (FAST)
EPSRC EP/L019981/1
The Alan Turing Institute ’Data Science of Music’
RNCM Centre for Practice & Research in Science &
Music (PRiSM), funded by Research England
Expanding Excellence in England (E3)
This talk is based on talks at:
‘Computational archival science: automating the
archive’, The National Archives, Sep 2018, organised
by Eirini Goudarouli
‘Discovering Collections, Discovering Communities’
(DCDC) conference, Birmingham UK 2019
ACM Web Science 2020, ‘Scholarly Social Machines’,
co-authored with Pip Willcox

45. RivkaGottliebperformsFib12atFASTIndustryDay
AbbeyRoadStudios,October2018
Fib12,composedusingNumbersintoNotes.
ReleasedonAdaLovelaceDay2018
https://soundcloud.com/david-de-roure/fib12-ada-lovelace-day-2018