AV Clash – Online Audiovisual Project A Case Study of Evaluation in New Media Art Presented at ACM ACE Conference, Lisbon 2011-11-11

1. AV CLASH – Online Audiovisual Project
A Case Study of Evaluation in New Media Art

2. What is AV Clash?
• AV Clash (http://www.avclash.com) is a Web-based artistic
project by Video Jack
– Nuno N. Correia and André Carrilho, with Gokce Taskan
• that allows the creation of audiovisual compositions,
consisting of combinations of sound and animation loops.
• The sounds are retrieved from Freesound.org.
• Visualization and UI are integrated into Interactive
AudioVisual Objects (IAVOs).
• AV Clash was released in November 2010.

3. IAVO – “Front”

4. IAVO – “Back”

5. The Questionnaire

6. The Questionnaire
• Soon after the project was released, an evaluation of the
project was planned, based on an online questionnaire.
• An online questionnaire seemed to be an adequate
format for evaluation, given that the project is Web-
based, and the audience of the project is global.

7. The Questionnaire
• The questionnaire was designed in order to assess if AV
Clash fulfills its objectives, as defined in the research
question it addresses:
• how to create a tool for integrated audiovisual creativity,
with customizable content, that is flexible, intuitive,
playful to use and engaging to observe?

8. The Questionnaire
• To address the issues of creativity, playfulness and
experience, and also the overall idea of engagement,
• recent texts regarding experience-focused approaches
to human-computer interaction (HCI) were important
references.
• The notion of aesthetic interaction also influenced the
design of the questionnaire.

9. The Questionnaire
• Different authors have identified gaps between HCI
methodologies and the methodologies used by new
media artists.
• These authors suggest that
– on the one hand, new media artists have ignored user studies;
– on the other hand, HCI methodologies are often task-focused,
and not experience-focused, which artists would find more
useful.

10. The Questionnaire
• The questionnaire was composed of 81 questions,
mostly close-ended, but also open-ended ones (13).
• The close-ended questions included mostly five-point
Likert scales.
• The questionnaire was answered by 22 anonymous
respondents.
• For this paper, sections of the questionnaire related to
the comparison of AV Clash with previous Video Jack
projects, and future developments, are excluded.

11. Conclusions

12. Integration of Media, Amount of Content
and Flexibility of Manipulation
• The integration of sound and visuals seems to be
successful, with the vast majority of the respondents
considering the visuals well suited to the sound.
• A large amount of sounds and visuals may have
remained undiscovered for users who only interacted
briefly with the system.
• More could be done to increase the amount of content,
but it might be equally important to focus on
mechanisms that facilitate the access to that content.

13. Integration of Media, Amount of Content
and Flexibility of Manipulation
• Users are split regarding the importance of customizing
sounds and visuals,
• which may reflect that there are more advanced users
who desire a larger diversity and deeper intervention on
the content side,
• and those who are satisfied with exploring and
manipulating a limited amount of existing materials.

14. Usability and Interactivity
• Usability should be substantially improved, particularly
in order to captivate casual users.
• Half of the users consider the interface to be unclear.
• The answers to open-ended questions also highlighted
this aspect—usability issues appeared even in answers
to unrelated questions.

15. Usability and Interactivity
• The approach of integrating the interface elements with
the animations seems positive for the vast majority of
test users.
• An equal percentage of participants also prefer the
interactive approach of AV Clash toward music and
visuals than linear sound or video.
• The appreciation for this interactivity was also recurrent
in the answers to the open-ended questions.

16. Creativity, Playfulness and Engagement
• Three quarters of the participants feel creative when
playing with AV Clash.
• The vast majority of the users stated that they had a
"fun" experience, with half answering that they had
"very much fun".
• An equal percentage of users declared they would use
the project again.

17. Creativity, Playfulness and Engagement
• When asked to describe the experience with the project
in an open-ended question, nine of the respondents used
the word "fun", with others using expressions such as
"playful", "immersive", "engaging", "surprising" and
"meditative".

18. Creativity, Playfulness and Engagement
• The positive results regarding the experience-related
questions seem to indicate that:
• despite the usability problems identified,
• a majority of users were willing to invest time exploring
the interface and the functionalities in order to be
rewarded in terms of playfulness and engagement.

19. Links
• www.avclash.com
• www.videojackstudios.com
Questions?