Presentation at Hochschule fuer Musik in Karlsruhe, Germany (16-March-2012). http://www.hfm-karlsruhe.de/

1. From
Networked
Music
to
Behavior-­‐Driven
Interac9on
Álvaro
Barbosa
-­‐
www.abarbosa.org
Research
Center
for
Science
and
Technology
of
the
Arts
(CITAR)
School
of
Arts,
Catholic
University
Porto
–
www.artes.ucp.pt

2. Álvaro
Barbosa
h=p://www.abarbosa.org/
Since
1998
–
EA,
UCP-­‐Porto,
Portugal
hIp://artes.ucp.pt/
Since
2004
–
CITAR,
UCP-­‐Porto,
Portugal
hIp://artes.ucp.pt/citar/
2000/2006
–
MTG,
UPF-­‐Barcelona,
Spain
hIp://mtg.upf.edu/
2010
–
CCRMA,
Stanford-­‐CA,
USA
hIp://ccrma.stanford.edu/
Since
2011
–
USJ,
Macau
–
China
hIp://www.usj.edu.mo/
Slide
#
2
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

3. Álvaro
Barbosa
Very
early
Web
Developer
in
the
industry
(Caleida
&
aeiou.pt,
1995-­‐1998)
Slide
#
3
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

4. Research
/
Book
• Displaced Soundscapes is a metaphorical description of the way lively
generated sounds can be perceived over the Internet."
• (Book) Barbosa, A. 2008. “Displaced Soundscapes”"
• VDM Verlag (ISBN: 978-3-8364-7154-1)"
- Extensive Survey of NM Applications "
- Research on Latency and Internet Acoustics"
- Models for Networked Music Systems"
- Implementation “Public Sound Objects” "
Slide
#
4
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

5. Background
on
Network
Music
Performance
Performing Music at a Distance
Synchronizing Berlin and Hong Kong (1/April/2011 – 8.753 Km)
Hong Kong New Music Ensemble & Ensemble Adapter
http://hknme.org/hongkongartsblog/?p=1525
Slide
#
5
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

6. Background
on
Network
Music
Performance
Performing Music at a Distance
Slide
#
6
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

7. Background
on
Network
Music
Performance
• Stanford-Mcgill 2002 (4.076 Km)
Slide
#
7
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

8. Background
on
Network
Music
Performance
• Stanford-Mcgill 2002 – Discovery Channel
Slide
#
8
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

9. Background
on
Network
Music
Performance
• Stanford-Mcgill 2002 (the real thing)
Slide
#
9
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

10. Background
on
Network
Music
Performance
Multiple Location – Same Performance
ARTECH 2008: Rambouillet (FR) / Casa da Música (PT) / SARC (IR)
http://artes.ucp.pt/artech2008/
Slide
#
10
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

11. Background
on
Network
Music
Performance
Latency and Networked Music
In a vocal conversation it is possible to maintain it even with one-way
delays of up to 500 ms.
Holub, J., Kastner, M., Tomiska, O. 2007
In order to maintain a synchronized and smooth musical interaction
reduces drastically to the order of tens of milliseconds.
Schuett, N. 2002
Chafe C., Gurevich M, 2004
Lago, N and Kon, F. 2004
Barbosa, A., Carôt, A. 2005
Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005
Bartlette, C., Headlam, D., Bocko, M., Velikic, G. 2006
Farner, S., Solvang, A., Sæbo, A., Svensson, U. P. 2009
Chafe, C., Cáceres, J-P., Gurevich, M., 2010
Slide
#
11
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

12. Background
on
Network
Music
Performance
Latency and Networked Music
For the Human ear to perceive the order of two simultaneous sounds,
they should not be displaced in time over 20ms (Hirsh, 1959)
A difficulty in discerning the order of sounds events
ê
hard to maintain a synchronous musical interaction.
Slide
#
12
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

13. Background
on
Network
Music
Performance
• Jitter and delay asymmetry
introduce further disruption in
Network Communication.
• Furthermore, Latency is not a
technological condition that can
be overcome in the near future.
Consider mobile or satellite
technology or a communication
setup that spans worldwide.
Moscow – Santiago (fiber-optic link):
Bidirectional Latency= 94,3 ms
Unidirectional Latency= 47,15 ms
Slide
#
13
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

14. Background
on
Network
Music
Performance
Latency and Networked Music
The ability to perform music synchronously is strongly dependent on:
• The music expressive qualities (Dynamics and Articulation)
• The music style (rhythm, melody, harmony)
• The music perceptual qualities (pitch, texture, timbre)
• The music structure/form
• The musician’s experience and practicing strategies
• Complementary feed-back modalities (visual, tactile)
• The listening conditions
Slide
#
14
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

15. Background
on
Network
Music
Performance
Barbosa,
A.
2003.
“Displaced
Soundscapes:
A
Survey
of
Network
Systems
for
Music
and
Sonic
Art
Crea9on”
Leonardo
Music
Journal
13,
pp.
53-­‐59
-­‐
MIT
Press,
Cambridge;
doi:10.1162/096112104322750791).
Slide
#
15
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

16. Background
on
Network
Music
Performance
Latency and Networked Music
Basic Principles - Remote Performance
(1) Ensemble Performance Threshold (EPT)
(2) Echo Feed-Back (Self Delay)
(3) Inverse Proportion to Tempo
(4) Reverb and Complementary Modalities
(5) Slow Attack Times
(6) Antiphony
Slide
#
16
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

17. Background
on
Network
Music
Performance
(1) Ensemble Performance Threshold (EPT)
(CCRMA 2004, USC 2005)
Comfort Zone for Expert Musicians EPT Interval for Expert Musicians
Comfort Zone for EPT Interval for
Untrained Users Untrained Users
Almost
Extremely Impossible to
Difficult to Compensate
Unnaturally Difficult to Compensate
Low Latency Compensate
DELAY (ms) 0 5 10 15 20 25 30 35 40 45 50 75 100 150
Slide
#
17
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

18. Background
on
Network
Music
Performance
(2) Echo Feed-Back (USC 2005)
Chew,
E.,
Sawchuk,
A.,
Tanoue,
C.,
and
Zimmermann,
R.
2005
Slide
#
18
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

19. Background
on
Network
Music
Performance
(2) Echo Feed-Back (UCP-Porto 2005)
Slide
#
19
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

20. Background
on
Network
Music
Performance
(3) Inverse Proportion to Tempo (UCP-Porto 2005)
Slide
#
20
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

21. Background
on
Network
Music
Performance
(3) Inverse Proportion to Tempo (CCRMA/Banff 2006)
St. Lawrence
String Quartet
(over 50ms Delay)
Slide
#
21
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

22. Background
on
Network
Music
Performance
(4) Reverb and Complementary Modalities
Studies on the effect of Reverberation
(Farner 2009)
The Influence of Delay and Various Acoustic Environments
Studies on the effect of Visual Feed-Back
Pilot experiments at CCRMA since 2006
Studies on the effect of Haptic Feedback ???
Slide
#
22
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

23. Background
on
Network
Music
Performance
(5) Slow Attack Times
Bregman (1990)
Auditory Scene Analysis
This
experiment
clearly
reveals
that
it
is
harder
to
perceive
the
order
of
overlapping
sound
events
when
their
rise
]me
(aIack)
is
slower.
In
other
words,
overlapping
sounds
with
slower
rise
]mes
are
beIer
perceived
as
synchronous
even
when
their
onsets
are
not
physically
simultaneous.
Slide
#
23
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

24. Background
on
Network
Music
Performance
• Pilot
Study
Hypothesis
• If
slow
a?ack
@mes
allow
a
be?er
percep@on
of
sound
simultaneity
this
might
also
lead
to
a
be?er
ability
to
perform
synchronous
musical
interac@on
• Experimental
Setup
Slide
#
24
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

25. Background
on
Network
Music
Performance
• Rhythmic
Performance
• 80
BPM
(to
avoid
possible
overlaying
of
subsequent
notes)
• Slow
Tempo
=>
Higher
EPT
• Range
from
25
ms
to
110
ms
(with
steps
of
10
ms)
• 28
trials
for
Slow
and
Fast
aIack
strokes
Slide
#
25
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

26. Background
on
Network
Music
Performance
Results
How
each
musician
performs
in
rela@on
to
the
other?
The
analysis
was
performed
using
the
soeware
tool
MATCH:
Music
Alignment
Tool
Chest,
developed
by
Simon
Dixon
(this
soeware
analyses
the
alignment
of
audio
files
using
the
OLTW
algorithm)
Slide
#
26
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

27. Background
on
Network
Music
Performance
(6)
ANTIPHONY
Technique
of
alternate
or
responsive
singing
by
a
choir
in
two
separate
sec@ons.
Each
sec]on
sings
alternate
musical
phrases
Or
POLYCHORAL
ANTIPHONY
(AKA
Vene]an
polychoral
style)
When
two
or
more
groups
of
singers
sing
in
alterna]on.
Origin
in
the
late
Renaissance
and
early
Baroque
Can
this
Technique
improve
network
latency
tolerance?
Slide
#
27
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

28. Background
on
Network
Music
Performance
Latency and Networked Music
Basic Principles - Shared Sonic Environments
(1) Digital Control Interface (software or tangible)
(2) Local Synthesis & Transmission of Control Data
(3) Peer-To-Peer Communication
(4) Latency Adaptive Dynamics
(5) Behavior Driven Interaction (Loose Coupling)
Slide
#
28
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

29. Background
on
Network
Music
Performance
Latency and Networked Music
Examples Shared Sonic Environments
PUBLIC
SOUND
OBJECTS
THE
HORGIE:
COLLABORATIVE
by
Álvaro
Barbosa
ONLINE
SYNTHESIZER
by
Jorge
Herrera
Slide
#
29
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

30. Public
Sound
Objects
Public Sound Objects
2004/2006, MTG (BCN)
Slide
#
30
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

31. Public
Sound
Objects
Local Network of PSOs – CITAR (Porto)
commissioned by “Casa da Musica”
http://en.wikipedia.org/wiki/Casa_da_musica
Slide
#
31
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

32. Public
Sound
Objects
Slide
#
32
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

33. Public
Sound
Objects
Slide
#
33
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

34. Public
Sound
Objects
Slide
#
34
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

35. Public
Sound
Objects
Permanent Installation
Casa da Música (Renaissance Hall)
Slide
#
35
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

36. Behavior-­‐Driven
Interac9on
Small Fish (Fujihata e Furukawa 1999)
Slide
#
36
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

37. Behavior-­‐Driven
Interac9on
Radial String Chimes
(1st prototype Developed at CCRMA – Stanford University)
Interactive musical device triggered by motion applied to a spinning vinyl record.
Users can spin the record, making hanging coffee straws to bounce and pluck
twelve radial guitar strings, applied to a round wood board.
Slide
#
37
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

38. Behavior-­‐Driven
Interac9on
Slide
#
38
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

39. Behavior-­‐Driven
Interac9on
Used
in
Live
Performances:
Live
Set
at
Galerija
Kapelica
in
Ljubljana,
Slovenia
During
The
EARZOM
Fes]val
2010
Slide
#
39
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

40. Behavior-­‐Driven
Interac9on
Jorney
to
the
Last
Fron9er
-­‐
with
Victor
Gama
–
B&W
2012
Slide
#
40
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

41. Behavior-­‐Driven
Interac9on
Live
Set:
2
Ipads
and
1
Iphone
runing
CS-­‐Grain,
Reactable
and
Cur]s;
1
Radio
String
Chimes;
2
Wind
Blowers;
1
E-­‐Bow
Slide
#
41
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

42. Behavior-­‐Driven
Interac9on
Victor
Gama:
www.victorgama.org
Performed
on
His
own
Instruments
–
Dino,
Acrux
and
Thoa
Slide
#
42
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

43. Behavior-­‐Driven
Interac9on
Audio
Materials:
Recorded
at
the
Antarc]ca
Expedi]onin
January
2012
by
Álvaro
Barbosa
and
Victor
Gama
Slide
#
43
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

44. Behavior-­‐Driven
Interac9on
Field
Recordings
at
the
Antarc9ca
Peninsula
Slide
#
44
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

45. Behavior-­‐Driven
Interac9on
Video
and
Photography
for
Mul9media
Perfoamances
Slide
#
45
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

46. Behavior-­‐Driven
Interac9on
Slide
#
46
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

47. Behavior-­‐Driven
Interac9on
Photography
Books:
Drieing
Ice;
Antarc]cans
Slide
#
47
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)

48. School
of
Arts
–
Porto,
Portugal
Thank
You!!!!
alvaro.barbosa@usj.edu.mo
abarbosa@porto.ucp.pt
|
hIp://www.abarbosa.org
Slide
#
48
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)