Presentation for GECCO 2014: https://dl.acm.org/citation.cfm?id=2598303

1. E N G A G I N G T H E W O R L D
Philippe Pasquier
pasquier@sfu.ca
1
Matthieu Macret
mmacret@sfu.ca
Automatic design of sound synthesizers as Pure
Data patches using Coevolutionary Mixed-typed
Cartesian Genetic Programming
Wednesday, July 16, 2014

2. The Synthesis Calibration Problem
2
Target sound
Sound
Synthesizer
i1
i2
...
in
=
Synthesized
sound
Input
parameters
Wednesday, July 16, 2014

3. Automatic calibration of Sound synthesizers
3
Synthesis
technique Op/miza/on
technique Reference
Addi)ve HMM Heise
(AES
Conven)on
2009)
Substra)ve Par)cle
Swarm
Yoshimura
(Speech
Communica)on
and
Tech.
1999)
Substra)ve Neural
Network Roth
(AES
Conven)on
2011)
FM Cellular
Automata
Serquera
(Applica)ons
of
Evolu)onary
Computa)on
2010)
FM GA
Horner
(Computer
Music
Journal
1993)
Addi)ve GA
Horner
(Computer
Music
Journal
1996)
Granular GA Fujinaga
(ICMC
1994)
Wednesday, July 16, 2014

4. ModFM synthesizer (Macret SMC 2012)
4
Wednesday, July 16, 2014

5. ModFM synthesizer (Macret SMC 2012)
4
Target
Wednesday, July 16, 2014

6. ModFM synthesizer (Macret SMC 2012)
4
Target
Wednesday, July 16, 2014

7. ModFM synthesizer (Macret SMC 2012)
4
Wednesday, July 16, 2014

8. ModFM synthesizer (Macret SMC 2012)
4
Synthesized
Wednesday, July 16, 2014

9. ModFM synthesizer (Macret SMC 2012)
4
Synthesized
Wednesday, July 16, 2014

10. More complex synthesizer
• VST Instruments with GA (Yee-King ICMC 2011)
5
Wednesday, July 16, 2014

11. OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-­‐objec)ve
Gene)c
Algorithm
Wednesday, July 16, 2014

12. OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-­‐objec)ve
Gene)c
Algorithm
Target
Wednesday, July 16, 2014

13. OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-­‐objec)ve
Gene)c
Algorithm
Target
Wednesday, July 16, 2014

14. OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-­‐objec)ve
Gene)c
Algorithm
Wednesday, July 16, 2014

15. OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-­‐objec)ve
Gene)c
Algorithm
Synthesized
Wednesday, July 16, 2014

16. OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-­‐objec)ve
Gene)c
Algorithm
Synthesized
Wednesday, July 16, 2014

17. General Sound Synthesis Problem
7
Target sound
Sound
Synthesizer
i1
i2
...
in
=
Synthesized
sound
Input
parameters
Wednesday, July 16, 2014

18. Genetic Programming - Garcia (DAFx 2001)
8
Produc)on
rules
in
had
hoc
synthesis
system
Sound
Synthesizer
i1
i2
...
in
=
Synthesized
sound
Target sound
GP
GA
Wednesday, July 16, 2014

19. Coevolutionary Mixed-typed Cartesian GP
• Coevolution of:
–Mixed-typed Cartesian Genetic Programming
•Evolve synthesizer architecture
–Genetic Algorithm
•Optimize Input Parameters
9
Wednesday, July 16, 2014

20. Pure Data
• Graphical programming environment for
audio, video, and image processing
• http://puredata.info/
• Popular among musician, artist and
sound designers
10
Wednesday, July 16, 2014

21. 11
Wednesday, July 16, 2014

22. 11
Wednesday, July 16, 2014

23. 11
Wednesday, July 16, 2014

24. Mixed-typed Cartesian GP
12
• Harding (GECCO 2012)
• Characteristics:
–Directed Acyclic Graph
–Mixed-typed
Wednesday, July 16, 2014

25. 13
0 0 0 5 1 4 5 9 2 10 3 0 2 9 7 3 5 6 0 7 9 3 5 4 3 2 0 1
CGP
graph
Pd
Patch
Wednesday, July 16, 2014

26. 14
Gene Input
1 Input
2
Extra
Param.
Descrip/o
n
Output
Type
3 control control -­‐ Division control
4 signal control -­‐ Oscillator signal
Wednesday, July 16, 2014

27. 15
0 0 0 5 1 4 5 9 2 10 3 0 2 9 7 3 5 6 0 7 9 3 5 4 3 2 0 1
CGP
graph
Pd
Patch
Wednesday, July 16, 2014

28. Fitness function
• MFCC (Mel-Frequency Cepstral
Coefficients) in windows of 23 ms
16
Euclidean
distance
between
synthesized
sound
and
target
sound
Wednesday, July 16, 2014

29. Fitness function
• MFCC (Mel-Frequency Cepstral
Coefficients) in windows of 23 ms
16
Euclidean
distance
between
synthesized
sound
and
target
sound
Wednesday, July 16, 2014

30. Evaluation
17
i1
i2
...
in
i1
i2
...
in
i1
i2
...
in
i1
i2
...
in
GA
popula)on
(150
individuals) CGP
Popula)on
(5
individuals)
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Wednesday, July 16, 2014

31. Genetic Algorithm
18
• Binary encoding of chromosome
• Binary tournament selection
• 2-points crossover
• Bitflip mutation
Wednesday, July 16, 2014

32. Mixed-typed Cartesian GP
• Integer encoding of chromosome
• Evolutionary Strategy 1+4
• Mutation-only approach
19
Wednesday, July 16, 2014

33. 20
Initialize CGP /GA
population
Evaluate CGP / GA
population
Variate GA
population
Select best CGP
individual
Variate CGP
population
Termination
criteria ?End
No
Yes
Wednesday, July 16, 2014

34. Clarinet sound
21
Wednesday, July 16, 2014

35. Clarinet sound
21
Wednesday, July 16, 2014

36. Clarinet sound
21
Wednesday, July 16, 2014

37. Clarinet sound
21
Target
Wednesday, July 16, 2014

38. Clarinet sound
21
Target
Wednesday, July 16, 2014

39. Clarinet sound
21
Wednesday, July 16, 2014

40. Clarinet sound
21
Synthesized
Wednesday, July 16, 2014

41. Clarinet sound
21
Synthesized
Wednesday, July 16, 2014

42. Clarinet sound
22
Wednesday, July 16, 2014

43. 23
Pd
patch
used
to
generate
the
target
sound Evolved
Pd
Patch
Contrived sound
Wednesday, July 16, 2014

44. 23
Pd
patch
used
to
generate
the
target
sound Evolved
Pd
Patch
Contrived sound
Wednesday, July 16, 2014

45. 23
Pd
patch
used
to
generate
the
target
sound Evolved
Pd
Patch
Contrived sound
Wednesday, July 16, 2014

46. 23
Pd
patch
used
to
generate
the
target
sound Evolved
Pd
Patch
Contrived sound
Wednesday, July 16, 2014

47. 23
Pd
patch
used
to
generate
the
target
sound Evolved
Pd
Patch
Contrived sound
Wednesday, July 16, 2014

48. Stats
24
Wednesday, July 16, 2014

49. Conclusions
• New approach to automate the design of
sound synthesizers using Coevolutionary
Mixed-typed Cartesian Programming
• Synthesizers represented as reusable Pd
patches
25
Wednesday, July 16, 2014

50. Future works
• Our methods involves many parameters
– Parameter sensitivity analysis
• Extend our system to reproduce not only
one target sound but a set of target sounds
• Improve our fitness function
– Move to a multi-objective fitness function
• Quantitative/Qualitative comparison with
previous similar systems
26
Wednesday, July 16, 2014