Notes on Audio-Vision, Synaesthetic Cinema, Audiovisual Composition and related concepts by Chion, Youngblood, Grierson and others

1. Notes on:
Visual Music,
Audio-Vision,
Synaesthetic Cinema,
Audiovisual Composition"
Nuno N. Correia, 2013
mail@nunocorreia.com

2. Visual Music"
• When defining visual music, Ox and Keefer (2008)
distinguish between four “differently formed visual
structures”:"
– A visualization of music, which translates sound into visuals,
“with the original syntax being emulated in the new visual
rendition”. According to Ox and Keefer, this can also be
defined as intermedia."
– A time-based visual composition, which is similar to the
structure of a kind or style of music – “as if it were an aural
piece”. It can have sound, or exist silent."
– A direct translation of image to sound – “literally, what you see
is also what you hear”. Some of Norman McLaren’s works
(where scratchings on film produce simultaneously image and
sound) fit in this category."
– A static visual composition, “as in Klee”."

3.
Michel Chion’s
Audio-Vision"

4. Audiovisual Contract"
• Sound does not correspond "naturally" to an image."
!
• Audiovisual contract:"
– "a kind of symbolic contract that the audio-viewer
enters into, agreeing to think of sound and image
as forming a single entity" (Chion 1994, p.216)."
"

5. Added Value"
• Added value: "
– "the expressive and informative value with which
a sound enriches a given image "
– so as to create the definitive impression, in the
immediate or remembered experience one has
of it, "
– that this information or expression 'naturally'
comes from what is seen, "
– and is already contained in the image
itself" (Chion 1994, p.5)."

6. Added Value"
• Added value works reciprocally: "
– on the one hand, "sound shows us the image differently
than what the image shows alone"; "
– on the other hand, image "makes us hear sound
differently than if the sound were ringing out in the
dark" (Chion 1994, p.21)."
• Added value is the most important of the relations
between sound and image (Chion 1994, p.5). "

7. Synchresis"
• Based on the idea of synchronization (and also
synthesis), Chion created the notion of synchresis:"
– "the forging of an immediate and necessary
relationship between something one sees and
something one hears at the same time" (Chion
1994, p.224)."

8. Synchresis"
• According to Chion, synchresis allows for numerous
combinations of possible sounds with possible
images: "for a shot of a hammer, any one of a
hundred sounds will do" (Chion 1994, p.63)."
• But random associations may not generate
synchresis: "play a stream of random audio and
visual events, and you will find that certain ones will
come together through synchresis and other
combinations will not" (Chion 1994, p.63)"

9. Senses as Channels"
• Chion states that there is no "sensory given" that is
isolated from the start: "the senses are channels,
highways more than territories or domains”."
• He clarifies this, stating that "when Kinetic
sensations organized into art are transmitted
through a single sensory channel", they can convey
all the other senses via that one channel. (Chion
1994, p.137). "
• He exemplifies with the inherent visuality of concrete
music, and the implied sound behind silent movies. "

10. Gene Youngblood’s
Expanded Cinema"

11. Expanded Cinema"
• Expanded cinema has been expanding for a long
time. "
• Since it left the underground and became a popular
avant-garde form in the late 1950's the new cinema
primarily has been an exercise in technique, "
– the gradual development of a truly cinematic language
with which to expand further man's communicative powers
and thus his awareness. "
• If expanded cinema has had anything to say, the
message has been the medium."
• Youngblood 1970, p. 75"

12. Expanded Cinema"
• Slavko Vorkapich: "
– "Most of the films made so far are examples not of
creative use of motion-picture devices and techniques, but
of their use as recording instruments only.” "
– “There are extremely few motion pictures that may be
cited as instances of creative use of the medium, "
– and from these only fragments and short passages may
be compared to the best achievements in the other arts”."
• Youngblood 1970, p. 75"

13. Synaesthetic Cinema"
• The new cinema has emerged as the only aesthetic
language to match the environment in which we live."
• Emerging with it is a major paradigm: a conception
of the nature of cinema so encompassing and
persuasive that it promises to dominate all image-
making in much the same way as the theory of
general relativity dominates all physics today. I call it
synaesthetic cinema."
• Youngblood 1970, p. 76"

14. Synaesthetic Cinema"
• The new artist and the new scientist recognize that
chaos is order on another level, and they set about
to find the rules of structuring by which nature has
achieved it. "
• That's why the scientist has abandoned absolutes
and the filmmaker has abandoned montage."
• Youngblood 1970, p. 76"

15. Synaesthetic Cinema"
• Synaesthetic cinema is the only aesthetic language
suited to the post-industrial, post-literate, man-made
environment with its multi- dimensional simulsensory
network of information sources. "
• It's the only aesthetic tool that even approaches the
reality continuum of conscious existence in the
nonuniform, nonlinear, nonconnected electronic
atmosphere of the Paleocybernetic Age."
• Youngblood 1970, p. 77"

16. Mick Grierson’s
Audiovisual
Composition"

17. Audiovisual Composition"
• Conventionally, audiovisual composition may be
taken to mean the process of putting sound and
music to visual material in order to create a
'soundtrack' to an existing film or animation. "
• The study of this type of practice is often referred to
as 'sound on film', 'audio for video' or 'sound design'. "
• The theoretical debates which surround this practice
are informed in particular by Michel Chion's Audio-
Vision:Sound on Screen."
• Grierson 2005, p. 9"

18. Audiovisual Composition"
• However, Chion also introduces concepts which
point to a more complex type of audiovisual practice.
With this in mind, it is illuminating that Chion takes
exception to the idea of the 'soundtrack'. According
to Chion, there is no such thing as a soundtrack."
• Grierson 2005, p. 9"
• Chion's justification for this view is bound up with his
notion of added value. This is, in essence, the idea
that through the combination of audio and visual
elements, a third audiovisual element is generated."
• Grierson 2005, p. 10"

19. Audiovisual Composition"
• Therefore, the audiovisual work is more than a
combination of its component parts. "
• As such, the practice of audiovisual composition is
not simply the production of audio with video. "
• It is the process of composing audiovisual works
which exploit added value."
• Grierson 2005, p. 10"
"

20. Audiovisual Composition"
• So in audiovisual composition and analysis, added
value shifts the focus from separate audio and visual
components, to the relationship between audio and
visual components. "
• This is particularly the case where a work has been
designed from the outset with great attention to
detail regarding the composition of both the sonic
and visual material and their effect together, "
• and even more so if the work attempts to exploit
ideas about the formal relationships between sonic
and visual material."
• Grierson 2005, p. 10"

21. Bibliography"
• Chion, M., 1994. Audio-Vision: Sound on Screen, New
York: Columbia University Press."
• Grierson, M., 2005. Audiovisual Composition. Kent:
University of Kent.
Available at: http://www.strangeloop.co.uk/Dr.
%20M.Grierson%20-%20Audiovisual%20Composition
%20Thesis.pdf."
• Ox, J. & Keefer, C., 2008. On Curating Recent Digital
Abstract Visual Music. In Abstract Visual Music. The
New York Digital Salon.
Available at: http://www.centerforvisualmusic.org/
Ox_Keefer_VM.htm."
• Youngblood, G., 1970. Expanded Cinema, New York: P.
Dutton & Co.
Available at: http://www.vasulka.org/Kitchen/
PDF_ExpandedCinema/ExpandedCinema.html."
!