AV Clash – Online Tool for Mixing and Visualizing Audio Retrieved from Freesound.org Database Presented at Doctor of Arts workshop, Aalto University, School of Art and Design, Media Lab, 12/11/2010

1. AV CLASH
Online Tool for Mixing and Visualizing Audio
Retrieved from Freesound.org Database
Nuno N. Correia
Aalto University, School of Art and Design, Media Lab
Helsinki, 11/2010

2. 1. Introduction
• Nuno N. Correia - Background:
– electronic musician / programmer / designer (and teacher);
– studies in international business / innovation management.
• Motivation for the research:
– lack of visual feedback in electronic music performances;
– lack of interactivity and visual feedback in Internet music.

3. 2. Framework

4. 2.1 Generic Title / Research Question
• Title of the dissertation:
Interactive Audiovisual Objects
• How to provide an engaging visual representation of
sound in an interactive Internet-based art project?

5. 2.2 Main Research Objective
• Build a series of engaging interactive audiovisual
projects for the web, with: increasing integration of
sound and graphics; variety of content (maintaining
cohesiveness); manipulation capabilities; and
customization.

6. 2.3 Secondary Research Objectives
• Produce
• Present
• Document
• Provide web access to
• Frame
• Explore interfaces of
• Map future uses of
• interactive audiovisual projects

7. 2.4 Research Methodology
• Practice-based artistic research

8. 2.5 Fields of Reference
• Media studies
• New media art / internet art
• Visual music / audiovisual art / music videos
• Experimental cinema / interactive narratives
• Film studies
• Sound and music computing
• Interaction design
• Internet and society
• Innovation management
• Synesthesia / perception
• Aesthetics / philosophy of art

9. Narra$ve
Abstract
3.
Master
and
Margarita
(2009)
2.
AVOL
(2007)
4.
AV
Clash
(2010)
1.
Heat
Seeker
(2006/8)
-­‐
Audio
reac$vity
/
diversity
+
0.
Idiot
Prince
(2006)
[founda$on
work]
2.6 Projects Flow Chart / Sandbox

10. 3. AV Clash

11. 3.1 Framing AV Clash
• AV Clash is the last project to be included in the scope of
the research plan.
• It reflects upon the main aspects of the research
question:
– integration of sound and graphics;
– variety of content (maintaining cohesiveness);
– manipulation capabilities;
– customization.
• It can be considered the culmination of the research.

12. 3.2 About AV Clash (1)
• A Web-based project by Video Jack (Nuno Correia, André
Carrilho and Gokce Taskan), which allows for the
creation of audiovisual compositions, consisting of
combinations of sound and animation loops.
• Composed of five audiovisual units, which enable
playback and manipulation of four main different loops
of sound and audio-reactive visuals.

13. 3.2 About AV Clash (2)
• Uses sounds retrieved from Freesound.org, an online
sound database (www.freesound.org).
• Aims to develop upon a previous Video Jack project, AVOL
(www.videojackstudios.com/avol), by allowing access to
more sounds and more manipulation possibilities.
• Allows for sound uploading, via Freesound.org, using the
“avclash” tag.
• URL : www.avclash.com

14. 3.3 IAVOs
• The five audiovisual units were named “Interactive
AudioVisual Objects” (“IAVOs”)
• They are composed of user interface (UI) elements that
trigger and manipulate sounds, together with
animations that react to those sounds.

15. 3.4 Tag Structure
• Each IAVO has a “tag” associated to it, which is retrieved
from Freesound.org.
• The tag is a filter to the possible sounds and animations
it may contain.
• There are eight animations and several dozens of
sounds available per tag.
• The user may change the four main sounds and
animations, and even the tag, of each IAVO, in its “back”.

16. 3.5 IAVO UI Ring (1)
• The UI is arranged around and inside the IAVO’s “ring”,
which allows for dragging and throwing the object
around the stage.
• The ring also incorporates two faders (for volume and
audio effect intensity), shaped as semi circles on each
of the sides.

17. 3.5 IAVO UI Ring (2)

18. 3.6 Additional Options (1)
• When the user presses the center button of the IAVO, the
four selection buttons expand into four larger
rectangles, allowing users to pick and adjust sounds and
animations for the respective buttons, from drop-down
menus.
• Sliders allow for trimming the sound loop and adjusting
the reactivity of the animation to the sounds.
• The tag can be changed.
• A cycle functionality is also available.

19. 3.6 Additional Options (2)

20. 4. Conclusions

21. 4.1 Conclusions
• The structure behind AVOL has been developed into a
more flexible and diverse project, in terms of :
– choice of source sounds (imported from Freesound.org);
– choice of animations (although less diverse than sounds);
– audiovisual manipulation (with effects);
– playfulness (with the “throw” and “clash” behaviors);
– randomness (reloading, “shuffle”);
– sound uploading (via the “avclash” tag).

22. 4.2 Conclusions – future developments
• However, AV Clash could benefit from:
– access to even more content;
– more content manipulation capabilities;
– implementation of collaboration functionalities;
– development of sharing and recording capabilities,
allowing users to record, and then share, their sessions.

23. 4.3 Conclusions - Questionnaire
• Feedback gathered from users, through questionnaires
(to be conducted soon), will enrich the conclusions.

24. 5. Related links
• Practice:
– www.videojackstudios.com
– www.avclash.com
• Theory:
– www.nunocorreia.com
• Technique:
– mlab.taik.fi/mediacode
• References:
– www.zotero.org/ncorreia/items
– www.mendeley.com/profiles/nuno-correia