10. -
- :
- :
- :
- :
- : SGD + BackProp
…
…x1 x2 xd
✓(2)
✓(1)
x
y
y(n)
=
X
j
✓
(2)
j (
X
i
✓
(1)
ji x
(n)
i ) + ✏(n)
p(y(n)
| x(n)
, ✓) = (
X
i
✓
(n)
i x
(n)
i )
✓
D = {x(n)
, y(n)
}N
n=1 = (X, y)
16. -
- :
- :
- :
- :
- : SGD + BackProp
…
…x1 x2 xd
✓(2)
✓(1)
x
y
y(n)
=
X
j
✓
(2)
j (
X
i
✓
(1)
ji x
(n)
i ) + ✏(n)
p(y(n)
| x(n)
, ✓) = (
X
i
✓
(n)
i x
(n)
i )
✓
D = {x(n)
, y(n)
}N
n=1 = (X, y)
17. - data hypothesis( )
- :
-
-
P(H | D) =
P(H)P(D | H)
P
H P(H)P(D|H)
P(x) =
X
y
P(x, y)
P(x, y) = P(x)P(y | x)
18. - :
- :
-
-
- :
P(H | D) =
P(H)P(D | H)
P
H P(H)P(D|H)
likelihood priorposterior
P(✓ | D, m) =
P(D | ✓, m)P(✓ | m)
P(D | m)
m:
P(x | D, m) =
Z
P(x | ✓, D, m)P(✓ | D, m)d✓
P(m | D) =
P(D | m)P(m)
P(D)
evidence
19. -
- :
- :
- :
- :
…
…x1 x2 xd
✓(2)
✓(1)
x
y
✓
D = {x(n)
, y(n)
}N
n=1 = (X, y)
P(✓ | D, m) =
P(D | ✓, m)P(✓ | m)
P(D | m)
m:
P(x | D, m) =
Z
P(x | ✓, D, m)P(✓ | D, m)d✓
prior
20. - (Variational Bayes)
- (MCMC)
P(✓ | D, m) =
P(D | ✓, m)P(✓ | m)
P(D | m)
m:
Z
P(D | ✓, m)P(✓)d✓
24. - VI
-
- David MacKay “Lecture 14 of the Cambridge Course”
- PRML 10
http://www.inference.org.uk/itprnn_lectures/
25. Reference
- Zoubin Ghahramani “History of Bayesian neural
networks” NIPS 2016 Workshop Bayesian Deep
Learning
- Yarin Gal “Bayesian Deep Learning"O'Reilly
Artificial Intelligence in New York, 2017
48. Edward
- Edward = TensorFlow + +
- TensorFlow
-
- TF GPU, TPU, TensorBoard, Keras
-
- Box’s Loop
- Python
49.
50. Refrence
•D. Tran, A. Kucukelbir, A. Dieng, M. Rudolph, D. Liang, and
D.M. Blei. Edward: A library for probabilistic modeling,
inference, and criticism.(arXiv preprint arXiv:1610.09787)
•D. Tran, M.D. Hoffman, R.A. Saurous, E. Brevdo, K. Murphy,
and D.M. Blei. Deep probabilistic programming.(arXiv
preprint arXiv:1701.03757)
•Box, G. E. (1976). Science and statistics. (Journal of the
American Statistical Association, 71(356), 791–799.)
•D.M. Blei. Build, Compute, Critique, Repeat: Data Analysis
with Latent Variable Models. (Annual Review of Statistics
and Its Application Volume 1, 2014)
51.
52. Dropout
- Yarin Gal ”Uncertainty in Deep Learning”
- Dropout
- Dropout : conv
- LeNet with Dropout http://mlg.eng.cam.ac.uk/yarin/blog_2248.html