This document discusses adaptive Markov chain Monte Carlo (MCMC) algorithms. It focuses on adapting the scale matrix of an adaptive random-walk Metropolis Hastings scheme. It notes some theoretical challenges with this approach and limitations when applied to high-dimensional problems. It also introduces Population Monte Carlo as an alternative adaptive importance sampling method that produces a sequence of improved importance functions to better approximate the target distribution.

1. On stability and convergence
of adaptive MC[MC]
Discussion by Christian P. Robert
Universit´ Paris-Dauphine, IuF, and CREST
e
http://xianblog.wordpress.com
Adap’skiii, Park City, Utah, Jan. 03, 2011
C.P. Robert On stability and convergence of adaptive MC[MC]

2. The adaptive algorithm
Focussing on the scale matrix of an adaptive random-walk
Metropolis Hastings scheme is theoretically fascinating, especially
when considering
removal of containment for LLN [intuitive but hard to prove]
potential link with renewal theory [in the fixed component
version]
no lower bound on Σm
verifiable assumptions
[Roberts & Rosenthal, 2007]
C.P. Robert On stability and convergence of adaptive MC[MC]

3. The adaptive algorithm (2)
Limited applicability of the adaptive scheme when using a
random-walk Metropolis Hastings scheme because of
strong tail [or support] conditions
more generaly, dependence on parameterisation
curse of dimensionality [ellip. symm. less & less appropriate]
unavailability of the performance target [e.g., acceptance of
0.234]
C.P. Robert On stability and convergence of adaptive MC[MC]

4. The adaptive algorithm (2)
Limited applicability of the adaptive scheme when using a
random-walk Metropolis Hastings scheme because of
strong tail [or support] conditions
more generaly, dependence on parameterisation
curse of dimensionality [ellip. symm. less & less appropriate]
unavailability of the performance target [e.g., acceptance of
0.234]
Congratulations and thanks for adding a software to the theoretical
development!
C.P. Robert On stability and convergence of adaptive MC[MC]

5. Initialising importance sampling
Alternative: Population Monte Carlo (PMC) offers a solution to the
difficulty of picking the importance function q through adaptivity:
Given a target π, PMC produces a sequence q t of importance
functions (t = 1, . . . , T ) aimed at improving the approximation of
π
[Capp´, Douc, Guillin, Marin & X., 2007]
e
C.P. Robert On stability and convergence of adaptive MC[MC]

6. Adaptive importance sampling
Use of mixture densities
D
q (x) = q(x; α , θ ) =
t t t
αd ϕ(x; θd )
t t
d=1
[West, 1993]
where
αt = (α1 , . . . , αD ) is a vector of adaptable weights for the D
t t
mixture components
θt = (θ1 , . . . , θD ) is a vector of parameters which specify the
t t
components
ϕ is a parameterised density (usually taken to be multivariate
Gaussian or Student-t)
C.P. Robert On stability and convergence of adaptive MC[MC]

7. PMC updates
Maximization of Lt (α, θ) leads to closed form solutions in
exponential families (and for the t distributions)
For instance for Np (µd , Σd ):
αd =
t+1
ρd (x; αt , µt , Σt )π(x)dx,
xρd (x; αt , µt , Σt )π(x)dx
µt+1 =
d ,
αd t+1
(x − µt+1 )(x − µt+1 )T ρd (x; αt , µt , Σt )π(x)dx
Σt+1 =
d
d d
.
αd t+1
C.P. Robert On stability and convergence of adaptive MC[MC]

8. Banana benchmark
Twisted Np (0, Σ) target with Σ = diag(σ1 , 1, . . . , 1), changing the
2
second co-ordinate x2 to x2 + b(x1 − σ1 )
2 2
20
10
0
−10
x2
−20
−30
−40
−40 −20 0 20 40
x1
p = 10, σ1 = 100, b = 0.03
2
[Haario et al. 1999]
C.P. Robert On stability and convergence of adaptive MC[MC]

9. Simulation
C.P. Robert On stability and convergence of adaptive MC[MC]

10. Comparison to MCMC
Adaptive MCMC: Proposal is a multivariate Gaussian with Σ
updated/based on previous values in the chain. Scale and update
times chosen for optimal results.
[Kilbinger, Wraith et al., 2010]
PMC MCMC
fa fa
fb fb
Evolution of π(fa ) (top panels) and π(fb ) (bottom panels) from 10k points to 100k points for both PMC (left
panels) and MCMC (right panels).
C.P. Robert On stability and convergence of adaptive MC[MC]