Deep learning is becoming ubiquitous in Machine Learning (ML) research, and it's also finding its place in industry-related applications. Specifically, deep generative models have proven incredibly useful at generating and remixing realistic content from scratch, making themselves a very appealing technology in the field of AI-enhanced content authoring. As part of this year's Machine Learning Tutorial at the Game Developers Conference 2019 (GDC), Jorge Del Val from SEED will cover in an accessible manner the fundamentals of deep generative modeling, including some common algorithms and architectures. He will also discuss applications to game development and explore some recent advances in the field. The attendee will gain basic understanding of the fundamentals of generative models and how to implement them. Also, attendees will grasp potential applications in the field of game development to inspire their work and companies. This talk does not require a mathematical or machine learning background, although previous knowledge on either of those is beneficial.

1. Towards Deep Generative Models in Game
Development
Jorge del Val
Research Engineer / SEED (Electronic Arts)

3. Agenda
1. Motivation and fundamentals
2. Variational autoencoders (VAE)
3. Generative adversarial networks (GAN)
4. Conditional generative models
5. Some applications to game development

4. In a sentence…
Models that generate or remix stuff

5. In a better sentence…
Models that learn the data probability distribution and are able
to sample from it

6. But… why in games?

7. (Thanks A. Opara )

9. Photo Wake-Up: 3D Character Animation from a Single Photo. Weng et al. 2018

10. Which is real?
A Style-Based Generator Architecture for Generative Adversarial Networks. Karras et al. 2018 (NVIDIA)

11. Which is real?
FAKE FAKE
A Style-Based Generator Architecture for Generative Adversarial Networks. Karras et al. 2018 (NVIDIA)
REAL

12. https://thispersondoesnotexist.com
(A Style-Based Generator Architecture for Generative Adversarial Networks. Karras et al. 2018)

13. So what do they actually do?

18. Image credit Animation Mentor

20. In the end… It’s all numbers
… in particular, M-dimensional vectors in 𝒳 ⊂ ℝ 𝑀
.

21. Data is far from random

22. Do we need M pixels to represent a face?
M=1000000 pixels!

23. Data is not really M-dimensional
It rather lays on a
lower dimensional
manifold!

24. Manifold?
𝑧2
𝑧1
𝑥 = (𝑥1, 𝑥2, 𝑥3)𝑧 = (𝑧1, 𝑧2)

25. Latent dimensions of data
Images credit Ward A.D. et al 2007
*Spoiler: Generative models learn both: the intrinsic geometry and the probability distribution!

26. Auto-Encoding Variational Bayes. Kingma et al. 2013
𝑧2
𝑧1 𝑧1
𝑧2

27. A walk through the latent space
A Style-Based Generator Architecture for Generative Adversarial Networks. Karras et al. 2018 (NVIDIA)

28. And how do they work?

29. Random variables and generative modelling
For us, each datapoint 𝑥𝑖 is just a realization of an underlying random variable.
𝐱 ∼ 𝑝(𝑥)
● Unsupervised learning is the field which attempts to infer properties of x from samples.
● Generative modelling is a subset of unsupervised learning which attempts to approximate
x as some parametrized combination of “simple” random variables which you can sample.
𝐱 ≈ 𝑓𝜃 𝐳 𝟏, 𝐳 𝟐, … , 𝐳 𝐊 ≜ 𝐱

30. Example: Gaussian Mixtures
Here every 𝐳𝑖 is normal (gaussian) 𝒩 𝜇𝑖, Σ𝑖 and the combination 𝑓𝜃(⋅) is a mixture.

31. Latent variable models
𝐳 ∼ 𝑝(𝑧)
𝐱
Transformation
Prior distribution

32. Architectures: neural networks
𝑓𝜃
𝜃 weights

33. Architectures: neural networks
Neural networks can approximate any function 𝑓(𝑥) to any precision! *
𝑥 𝑓𝜃(𝑥)
𝜃
*G. Cybenko 1989, K. Hornik 1991, Z. Lu et al 2017, B. Hanin 2017
Image credit Sydney Firmin

34. But to find the right 𝜃 (training)?
You optimize some loss (error) function!
min
𝜃
𝐿 𝜃; data

35. E.g. Classifier
𝑥
𝑓𝜃 𝑥
𝐿 𝜃; 𝑥, 𝑦 = −𝑦 𝑐𝑎𝑡 log 𝑓𝜃 𝑥 − 𝑦 𝑑𝑜𝑔 log 1 − 𝑓𝜃 𝑥
𝜃
𝑦 𝑐𝑎𝑡 ∈ {0,1}
𝑦 𝑑𝑜𝑔 ∈ {0,1}
Cross-entropy
𝑝 𝑐𝑎𝑡
I want it to be…

36. E.g. Classifier
𝑥
𝑓𝜃 𝑥
𝐿 𝜃; 𝑥, 𝑦 = −𝑦 𝑐𝑎𝑡 log 𝑓𝜃 𝑥 − 𝑦 𝑑𝑜𝑔 log 1 − 𝑓𝜃 𝑥
𝜃
𝑦 𝑐𝑎𝑡 ∈ {0,1}
𝑦 𝑑𝑜𝑔 ∈ {0,1}
Cross-entropy
𝑝 𝑐𝑎𝑡
I want it to be…

37. But to find the right 𝜃 (training)?
You optimize some loss (error) function!
𝜃𝑡+1 = 𝜃𝑡 − 𝛼∇ 𝜃 𝐿min
𝜃
𝐿 𝜃; data
Stochastic Gradient Descent

38. But to find the right 𝜃 (training)?
You optimize some loss (error) function!
𝜃𝑡+1 = 𝜃𝑡 − 𝛼∇ 𝜃 𝐿min
𝜃
𝐿 𝜃; data
Stochastic Gradient Descent

39. But to find the right 𝜃 (training)?
You optimize some loss (error) function!
𝜃𝑡+1 = 𝜃𝑡 − 𝛼∇ 𝜃 𝐿min
𝜃
𝐿 𝜃; data
Easy to gradient descent any
function with current frameworks!! 
Stochastic Gradient Descent

40. Latent variable models
𝐳 ∼ 𝑝 𝑧
𝐱
Prior distribution
Transformation

41. Deep latent variable models
𝐳 ∼ 𝑝 𝑧
𝐱 = 𝐺 𝜃(𝐳)
𝐺 𝜃(⋅)
Prior distribution

42. Training
We want to approximate 𝐱 as 𝐱 = 𝐺 𝜃(𝐳). How do we find the optimal θ?
Maximize the likelihood of the data!
max
𝜃
log ℒ(𝜃|𝑥 𝑡𝑟𝑎𝑖𝑛) = max
𝜃
𝑖=1
𝑁
log 𝑝 𝜃(𝑥𝑖)
max
𝜃
ℒ(𝜃|𝑥 𝑡𝑟𝑎𝑖𝑛) = max
𝜃
𝑖=1
𝑁
𝑝 𝜃(𝑥𝑖)
But… we need 𝑝 𝜃 𝑥 explicitly!
Probability that the model would generate 𝑥𝑖
Image credit: Colin Raffel

43. What about deep latent variable models?
𝑝(𝑧) 𝑝 𝜃(𝑥|𝑧)
𝐺 𝜃(⋅)𝐳 𝐱
𝑝 𝜃 𝑥 = 𝑝 𝜃 𝑥 𝑧 𝑝 𝑧 𝑑𝑧
What’s the total probability
of generating 𝑥?
𝑝 𝜃(𝑥)??

44. Different models – different methods
1. We have 𝑝 𝜃( 𝑥) explicitly: maximize the likelihood.
2. 𝑝 𝜃 𝑥 is intractable: we can approximate it instead
● Markov Chain Monte Carlo (MCMC) methods
● Variational methods (e.g. Variational Autoencoders)
3. We don’t need 𝑝 𝜃 𝑥 ; it’s implicit.
● Adversarial methods (e.g. GANs)

45. Goodfellow. 2016

46. Variational autoencoder (VAE)
𝑝(𝑧) 𝑝 𝜃(𝑥|𝑧)
𝐺 𝜃(⋅)𝐳 𝐱
𝑝 𝜃 𝑥 = 𝑝 𝜃 𝑥 𝑧 𝑝 𝑧 𝑑𝑧

47. Variational autoencoder (VAE)
𝑝(𝑧) 𝑝 𝜃(𝑥|𝑧)
𝐺 𝜃(⋅)𝐳 𝐱
𝑞 𝜙(𝑧|𝑥)
𝐸 𝜙(⋅)𝐱

48. Variational autoencoder (VAE)
𝐺 𝜃(⋅)𝐳 𝐱𝐱 𝐸 𝜙(⋅)
𝑞 𝜙(𝑧|𝑥)
𝑝(𝑧) 𝑝 𝜃(𝑥|𝑧)
log 𝑝 𝜃 𝑥 ≥ 𝔼 𝑧∼𝑞 𝜙(𝑧|𝑥) log 𝑝 𝜃 𝑥 𝑧 − KL 𝑞 𝜙 𝑧 𝑥 ∥ 𝑝 𝑧
Encoder (inference) Decoder (generation)
Maximize this
instead!

49. True likelihood
Lower bound
Iterations

50. Pros:
● Efficient inference for free!
o Great tool for modelling the hidden structure of data.
● Stable to train.
● Good theoretical ground.
Cons:
● Not very good samples.
Variational autoencoders

51. Generative adversarial networks (GANs)
𝑝(𝑧)
𝐺 𝜃(⋅)𝐳 𝐱
Why not just sample a bunch of
data and see if they look real?
𝐱
Sample
𝑥1, 𝑥2, … , 𝑥 𝐵
𝑥1, 𝑥2, … , 𝑥 𝐵
Objective:
Looks similar!

52. But… how do we measure similarity between groups of samples?

53. How to measure similarity of samples
One solution: train a classifier 𝐷 𝜙(𝑥) to discriminate!
● If the classifier can not tell if a sample is real or fake, both distributions are close.
● Trained with the standard cross-entropy loss:
max
𝜙
𝐿 𝑑(𝜙) = max
𝜙
𝔼 𝑥 𝑟∼𝑝 𝑟𝑒𝑎𝑙
log 𝐷 𝜙 𝑥 𝑟 + 𝔼 𝑥 𝑓∼𝑝 𝑓𝑎𝑘𝑒
log 1 − 𝐷 𝜙 𝑥𝑓
It can be shown that the optimal classifier performance 𝐿 𝑑(𝜙∗
) is related to the closeness
between both distributions (JS divergence).

54. The GAN game
We want to minimize “closeness” between the generated and real samples, as measured by
the discriminator loss:
min
𝜃
"closeness"
= min
𝜃
max
𝜙
𝔼 𝑥 𝑟∼𝑝 𝑟𝑒𝑎𝑙
log 𝐷 𝜙 𝑥 𝑟 + 𝔼 𝑥 𝑓∼𝑝 𝑓𝑎𝑘𝑒
log 1 − 𝐷 𝜙 𝑥𝑓
It’s formally a two-player minimax game!!

55. Generative adversarial networks
𝑝(𝑧)
𝐺 𝜃(⋅)𝐳 𝐱
Why not just sample a bunch of
data and see if they look real?
𝐱
Sample
𝑥1, 𝑥2, … , 𝑥 𝐵
𝑥1, 𝑥2, … , 𝑥 𝐵
Objective:
Looks similar!
𝐷 𝜙(⋅)
Objective:
Fool 𝐷 𝜙!

57. GANs
● Pros:
 Awesome samples
● Cons:
 Unstable training
 No explicit probability density
 No direct inference
Large Scale GAN Training for High Fidelity Natural Image Synthesis. Brock et al. 2018

58. Bonus: autoregressive methods
𝑝 𝜃 𝑥 =
𝑡=1
𝑇
𝑝 𝜃 𝑥 𝑡 𝑥1, … , 𝑥 𝑡−1
Generate little by little!
Wavenet: A Generative Model for Raw Audio. Van den Oord et al. 2016.

59. OK!

60. OK! I can generate stuff.

61. OK! I can generate stuff.
But how do I influence what I generate?

62. OK! I can generate stuff.
How do I remix existing stuff??

63. Conditional generative models
What if I have information 𝐜 to condition the generation/inference, e.g., class
labels?
● Just introduce them in the networks!
𝐺 𝜃(⋅)𝐳 𝐱𝐱 𝐸 𝜙(⋅)
𝑞 𝜙(𝑧|𝑥)
𝑝(𝑧) 𝑝 𝜃(𝑥|𝑧)
Encoder
(inference)
Decoder
(generation)
Variational
Autoencoder

64. 𝑞 𝜙(𝑧|𝑥, 𝑐)
𝑝 𝜃(𝑥|𝑧, 𝑐)
Conditional
Variational
Autoencoder
Conditional generative models
What if I have information 𝐜 to condition the generation/inference, e.g., class
labels?
● Just introduce them in the networks!
𝐺 𝜃(⋅)𝐳 𝐱𝐱 𝐸 𝜙(⋅)
𝑝(𝑧)
Encoder
(inference)
Decoder
(generation)
𝐜 𝐜

65. Conditional generative models
What if I have information 𝐜 to condition the generation/inference, e.g., class
labels?
● Just introduce them in the networks!
GAN
𝑝(𝑧)
𝐺 𝜃(⋅)𝐳 𝐱
𝐱
Sample
𝑥1, 𝑥2, … , 𝑥 𝐵
𝑥1, 𝑥2, … , 𝑥 𝐵
𝐷 𝜙(⋅)

66. Conditional
GAN
Conditional generative models
What if I have information 𝐜 to condition the generation/inference, e.g., class
labels?
● Just introduce them in the networks!
𝑝(𝑧)
𝐺 𝜃(⋅)𝐳 𝐱
𝐱
Sample
𝑥1, 𝑥2, … , 𝑥 𝐵
𝑥1, 𝑥2, … , 𝑥 𝐵
𝐷 𝜙(⋅)
𝐜
𝐜

67. Conditional GMs are very important!
Pix2Pix: Image-to-Image Translation with Conditional Adversarial Networks. Isola et al.

68. Conditional GMs are very important!
Pose Guided Person Image Generation. Ma et al. 2017.

69. Some applications to game dev so far?

70. Generation of terrain
Interactive Example-Based Terrain Authoring with Conditional Generative Adversarial Networks. Guérin et al. 2017.

71. 3D Content Generation
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation.
Park et al. 2019.
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling.
Wu et al. 2016

72. Face generation
GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction. Gecer et al. 2019 (FaceSoft.io)

73. Procedural placement
Deep Convolutional Priors for Indoor Scene Synthesis. Wang et al. 2018.

74. Generation of behaviour policies
Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow. Peng et al. 2018.

75. Generation of behaviour policies
Imitation Learning with Concurrent Actions in 3D Games. Harmer et al. 2018 (SEED)

76. Learn and accelerate physics
Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flow. Wiewel et al. 2018

78. Thanks for inspiration and insightful
conversations 
Anastasia Opara
Camilo Gordillo
Colin Barré-Brisebois
Hector Anadón León
Henrik Johansson
Jack Harmer
Joakim Bergdahl
Johan Andersson
Kristoffer Sjöö
Ken Brown
Linus Gisslen
Martin Singh-Blom
Mattias Teye
Mark Cerny
Mónica Villanueva Aylagas
Magnus Nordin
Olivier Pomarez
Paul Greveson
Roy Harvey
Tomasz Stachowiak

79. S E E D / / S E A R C H F O R E X T R A O R D I N A R Y E X P E R I E N C E S D I V I S I O N
S T O C K H O L M – L O S A N G E L E S – M O N T R É A L – R E M O T E
S E E D . E A . C O M
W E ‘ R E H I R I N G !
Jorge del Val Santos
jdelvalsantos@ea.com