More Related Content
Similar to Prml sec3 (20)
More from Keisuke OTAKI (16)
Prml sec3
- 1. PRML
§3
2010 / 06 / 27
id: taki0313
2010 7 9
- 2. CONTENTS
● §3.1
● §3.2
● §3.3
● §3.4
● §3.5
● §3.6
2010 7 9
- 3. Introduction
● →
●
{xn} , {tn}
x t
…
t = y(x) p( t | x ) …
Et[t | x] @ §1.5.5
2010 7 9
- 4. §3.1
●
● Φj
w
Φj(x) = x^j
2010 7 9
- 6. §3.1.1
● +
●
● : y(x,w)
2010 7 9
- 14. §3.1.2
● N
● M=2, N=3
j
N Φ j
Φ(xn) n
2010 7 9
- 15. ●N>M M S
●t y(xn,w) ∈ S
=
3.2 …
2010 7 9
- 16. §3.1.3
●
● E
τ η
(3.12)ED …
LMS ( )
2010 7 9
- 18. ●
●
● q=1(lasso) λ →
2010 7 9
- 19. ● …
λ →
M →λ
…?
? §3.4
● 2
2010 7 9
- 20. §3.1.5
●
y :K
W : (M,K)
2010 7 9
- 21. ●
tk n tnk,N
2010 7 9
- 22. §3.2
● §1.5.5
p(t|x) …
…
●
h(x)
y ( !)
2010 7 9
- 23. ∞ …
h(x)
y(x,w) …?
=
w
D
2010 7 9
- 24. ●
D
2*(...)
( )^2
Bias:
Variance:
2010 7 9
- 25. ● …
= Bias^2 + Variance + Noise
Bias Variance
Bias - Variance
h(x) = sin(2πx) , N=25 , M=25
100
λ
2010 7 9
- 27. §3.3
●
●
w
…
2010 7 9
- 28. w
2
w ∝ ×
?→2
w mN
2010 7 9
- 31. ●1 x, 1 t, y(x,w) = w0 + x*w1
● f(x, a) = a0 + x*a1 (a0=-0.3, a1=0.5)
xn ∈ U(x| -1, 1) f(xn, a)
σ=0.2 tn
● : a0=-0.3 , a1=0.5
● α=2.0 β=(1 / 0.2)^2 = 25 ( )
● ∝ *
●w p(w|α)=N(w|0,1/αE)
2010 7 9
- 32. y(x,w)
1 ( )
2 ( )
w …
2010 7 9
- 33. q=2
q=2 …
w =
2010 7 9
- 34. §3.3.2
● t →
● ?2 k& &
1/β + w
N→∞ →0 (w )
2010 7 9
- 36. ● sin(2πx) , N = 1 , 2 , 4 , 25. ,
● ±
● 9
●w y(x,w)
2010 7 9
- 38. …
k(x,x’) :
x’ , x
k(x,x’)
x x’
x‘
2010 7 9
- 39. y(x) y(x’)
:
k(x,x’)
2010 7 9
- 40. §3.4
●
L {Mi}
D
P(Mi)
D
2010 7 9
- 41. p(Mi) :
p(D|Mi) :
?
p(D|Mi) / p(D|Mj) :
( )
2010 7 9
- 43. p(w) ~ 1 / Δwprior
: wMAP
Δwposterior
w
( )
2010 7 9
- 44. M : +
M
…
M1 M2 M3
2010 7 9
- 45. D
p(w) w
p(D|w)
M1 :
D
M3 :
D
p(D|Mi) D0
M2
M1 D0
M3 D0
2010 7 9
- 46. M
D
?
2 M1, M2. M1
D
2010 7 9
- 47. §3.5
●
α,β
…
w
2 …
2010 7 9
- 48. :w α β
p(t|w,β) : (3.8) , p(w|t,α,β) : (3.49),(3.53),(3.54)
p(α,β|t)
α β
α β w
2010 7 9
- 49. α,β
p(t|α,β)
α,β
EM
2010 7 9
- 50. §3.5.1
● p(t|α,β)
(3.11), (3.12), (3.52) …
2010 7 9
- 52. (3.27)
w
A ∇∇E(w) :
A
mN
2010 7 9
- 54. ● M
● α = 5 * 10^{-3}
● 3~8 3
● M=3
2010 7 9
- 55. §3.5.2
● p(t|α,β) α
●
→A α + λi
● ln |A| α
2010 7 9
- 56. α
2α &Σ M
α …
2010 7 9
- 57. α → β
γ, mN α
α →γ,mN →α →…
× β → λi
β
λi β
2010 7 9
- 58. §3.5.3
● α
●
→
● λi
2010 7 9
- 59. λ
γ γ
well-determined
λ1 < λ2
λ2 λ1
γ←
γ →
γ:
γ → → Wml
2010 7 9
- 60. β
1
μML
γ
γ → 1 / N-γ
2010 7 9
- 61. ● +9 M=10
●β 11.1 α
● γ 2αEw(mn)
●
● α
●
2010 7 9
- 62. ● γ
● 0≦α≦∞ → 0≦γ≦M
{wi}
α→γ
γ→
α
● M << N well-determined
●
●α β …
2010 7 9
- 63. §3.6
● →
●
●
●
1.
2. →M
●
1.
2.
2010 7 9
