This document discusses trans-dimensional Bayesian inference problems and proposes a Markov chain Monte Carlo approach. It presents two examples: detection and estimation of sinusoids in noise, and detection and estimation of muons in the Auger project. Key aspects covered include variable-dimensional models, parameter estimation algorithms, and summarizing posterior distributions that address label switching issues.
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
1. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Signal decompositions using
trans-dimensional Bayesian methods
Alireza Roodaki
Ph.D. Thesis Defense
Department of Signal Processing and Electronic Systems
2012, May 14th
Advisors: Julien Bect and Gilles Fleury
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
2. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 1: Detection and estimation of muons in the
Auger project
Ultra high energy particles
coming from space
(E ∼ 1019 eV)
How and where?
What is their composition
(Proton, Iron)?
Figure: A conceptual shower
(http://auger.org).
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
3. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 1: Detection and estimation of muons in the
Auger project
Ultra high energy particles
coming from space
(E ∼ 1019 eV)
How and where?
What is their composition
(Proton, Iron)?
Figure: A conceptual shower
(http://auger.org).
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
4. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 1: Detection and estimation of muons in the
Auger project
Ultra high energy particles
coming from space
(E ∼ 1019 eV)
How and where?
What is their composition
(Proton, Iron)?
Figure: A conceptual shower
(http://auger.org).
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
5. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 1: Detection and estimation of muons in the
Auger project (Contd.)
muons are generated
when particles cross the
atmosphere
the number k of muons
and their arrival times t µ
are indicators of the origin
Figure: A conceptual shower and and composition of particle
detectors (water tanks)
(http://auger.org).
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 2/ 52
6. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 1: Detection and estimation of muons in the
Auger project (Contd.)
muons are generated
when particles cross the
atmosphere
the number k of muons
and their arrival times t µ
are indicators of the origin
Figure: A conceptual shower and and composition of particle
detectors (water tanks)
(http://auger.org).
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 2/ 52
7. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 1: Detection and estimation of muons in the
Auger project (Contd.)
muons are generated
when particles cross the
atmosphere
the number k of muons
and their arrival times t µ
are indicators of the origin
Figure: A conceptual shower and and composition of particle
detectors (water tanks)
(http://auger.org).
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 2/ 52
8. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 1: Detection and estimation of muons in the
Auger project (Contd.)
Figure: Water tank detector.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 3/ 52
9. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 1: Detection and estimation of muons in the
Auger project (Contd.)
#PE 20
10
0
1.5
intensity
1
0.5
0
100 200 300 400 500 600
t[ns]
Figure: Observed signal (n)
Prof. Balázs Kégl from LAL, University of Paris 11.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 4/ 52
10. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 1: Detection and estimation of muons in the
Auger project (Contd.)
#PE 20
10
0
1.5
intensity
1
0.5
0
100 200 300 400 500 600
t[ns]
Figure: Observed signal (n)
Prof. Balázs Kégl from LAL, University of Paris 11.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 4/ 52
11. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 2: Spectral analysis
10
0
−10 Applications
0 10 20 30 40 50 60
time RADAR / SONAR
150
100 Array signal processing
Power
50 Vibration analysis
0 ...
0 0.5 1 1.5 2 2.5 3
radial frequency
Figure: Observed signal (top) and its
periodogram (bottom).
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 5/ 52
12. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 2: Spectral analysis (Cont.)
Detection and estimation of sinusoids in white noise
model the observed signal y by sinusoidal components
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 6/ 52
13. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 2: Spectral analysis (Cont.)
Detection and estimation of sinusoids in white noise
model the observed signal y by sinusoidal components
observed signal
k
Mk : y [i] = aj cos[ωj i] + bj sin[ωj i] + n[i].
j=1
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 6/ 52
14. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 2: Spectral analysis (Cont.)
Detection and estimation of sinusoids in white noise
model the observed signal y by sinusoidal components
observed signal
k
Mk : y [i] = aj cos[ωj i] + bj sin[ωj i] + n[i].
j=1
Joint model selection and parameter estimation problem
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 6/ 52
15. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Trans-dimensional problems
Def. The problems in which the number of things that we
don′ t know is one of the things that we don′ t know [Green,
2003.]
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 7/ 52
16. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Trans-dimensional problems
Def. The problems in which the number of things that we
don′ t know is one of the things that we don′ t know [Green,
2003.]
space X = k ∈K {k } × Θk with points x = (k , θk )
« k ∈ K denotes number of components
« θk ∈ Θk is a vector of component-specific
parameters
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 7/ 52
17. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Trans-dimensional problems
Def. The problems in which the number of things that we
don′ t know is one of the things that we don′ t know [Green,
2003.]
space X = k ∈K {k } × Θk with points x = (k , θk )
« k ∈ K denotes number of components
« θk ∈ Θk is a vector of component-specific
parameters
Applications:
« Spectral Analysis (Array signal processing)
« (Gaussian) Mixture modeling & Clustering
« Object detection and recognition
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 7/ 52
18. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Bayesian inference
Likelihood
p(y | x) p(x)
p(x | y) =
X p(y | x ′ ) p(x ′ )dx ′
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
19. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Bayesian inference
Likelihood
p(y | x) p(x)
p(x | y) =
X p(y | x ′ ) p(x ′ )dx ′
Prior distribution
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
20. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Bayesian inference
Likelihood
p(y | x) p(x)
p(x | y) =
X p(y | x ′ ) p(x ′ )dx ′
Prior distribution
Posterior distribution
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
21. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Bayesian inference
Likelihood
p(y | x) p(x)
p(x | y) =
X p(y | x ′ ) p(x ′ )dx ′
Prior distribution
Posterior distribution
x = (k , θk )
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
22. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Bayesian inference
Likelihood
p(y | x) p(x)
p(x | y) =
X p(y | x ′ ) p(x ′ )dx ′
Prior distribution
Posterior distribution
x = (k , θk )
« both detection and estimation problems
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
23. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Bayesian inference
Likelihood
p(y | x) p(x)
p(x | y) =
X p(y | x ′ ) p(x ′ )dx ′
Prior distribution
Posterior distribution
x = (k , θk )
« both detection and estimation problems
high-dimensional / intractable integrals
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
24. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Markov Chain Monte Carlo (MCMC) methods
generate samples from the posterior distribution of interest
(target distribution), say, π.
construct a Markov chain (x (1) , . . . , x (M) ) that under some
conditions converges to π.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 9/ 52
25. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Markov Chain Monte Carlo (MCMC) methods
generate samples from the posterior distribution of interest
(target distribution), say, π.
construct a Markov chain (x (1) , . . . , x (M) ) that under some
conditions converges to π.
Famous algorithms:
« Metropolis-Hastings (MH) sampler [Metropolis, et al.
1953, Hastings, 1970.]
« Gibbs sampler [Geman and Geman, 1984.]
« RJ-MCMC sampler [Green, 1995.]
[Robert and Casella, 2004.]
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 9/ 52
26. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Markov Chain Monte Carlo (MCMC) methods
generate samples from the posterior distribution of interest
(target distribution), say, π.
construct a Markov chain (x (1) , . . . , x (M) ) that under some
conditions converges to π.
Famous algorithms:
« Metropolis-Hastings (MH) sampler [Metropolis, et al.
1953, Hastings, 1970.]
« Gibbs sampler [Geman and Geman, 1984.]
« RJ-MCMC sampler [Green, 1995.]
[Robert and Casella, 2004.]
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 9/ 52
27. Relabeling and summarizing posterior distributions
Proposed approach
Results
Conclusion
Example 2: spectral analysis (Cont.)
RJ-MCMC sampler ⇒ variable dimensional samples
0.8
0.6
ωk
0.4
4
3
k
2
160 170 180 190 200
Iteration number
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 10/ 52
29. Outline
1 Relabeling and summarizing posterior distributions
Label-switching issue
Variable-dimensional summarization
2 Proposed approach
An original variable-dimensional parametric model
Estimating the model parameters (SEM-type algorithms)
Robustifying strategies
30. Outline
1 Relabeling and summarizing posterior distributions
Label-switching issue
Variable-dimensional summarization
2 Proposed approach
An original variable-dimensional parametric model
Estimating the model parameters (SEM-type algorithms)
Robustifying strategies
3 Results
Detection and estimation of sinusoids in white noise
Detection and estimation of muons in the Auger project
31. Outline
1 Relabeling and summarizing posterior distributions
Label-switching issue
Variable-dimensional summarization
2 Proposed approach
An original variable-dimensional parametric model
Estimating the model parameters (SEM-type algorithms)
Robustifying strategies
3 Results
Detection and estimation of sinusoids in white noise
Detection and estimation of muons in the Auger project
4 Conclusion
32. Outline
1 Relabeling and summarizing posterior distributions
Label-switching issue
Variable-dimensional summarization
2 Proposed approach
An original variable-dimensional parametric model
Estimating the model parameters (SEM-type algorithms)
Robustifying strategies
3 Results
Detection and estimation of sinusoids in white noise
Detection and estimation of muons in the Auger project
4 Conclusion
33. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
summarizing posterior distributions
Posterior = all the information
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
34. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
summarizing posterior distributions
Posterior = all the information
« It is a complex mathematical object (not easy to
manipulate)
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
35. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
summarizing posterior distributions
Posterior = all the information
« It is a complex mathematical object (not easy to
manipulate)
(RJ)-MCMC sampler
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
36. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
summarizing posterior distributions
Posterior = all the information
« It is a complex mathematical object (not easy to
manipulate)
(RJ)-MCMC sampler
« What to do with the generated samples?
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
37. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
summarizing posterior distributions
Posterior = all the information
« It is a complex mathematical object (not easy to
manipulate)
(RJ)-MCMC sampler
« What to do with the generated samples?
Summarization
human readable summaries
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
38. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
summarizing posterior distributions
Posterior = all the information
« It is a complex mathematical object (not easy to
manipulate)
(RJ)-MCMC sampler
« What to do with the generated samples?
Summarization
human readable summaries
interpretable figures (e.g., histograms)
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
39. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
summarizing posterior distributions
Posterior = all the information
« It is a complex mathematical object (not easy to
manipulate)
(RJ)-MCMC sampler
« What to do with the generated samples?
Summarization
human readable summaries
interpretable figures (e.g., histograms)
statistical measures (e.g., mean and variance)
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
40. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Fixed-dimensional problems
uni-modal uni-variate case
0.6
p 0.4
0.2
0
0 2 4
Samples
report location (mean and median) and dispersion
(variance and confidence intervals) parameters
µ σ2
2.0 0.25
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 13/ 52
41. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Label-switching issue
Additive mixture: lack of identifiability
the likelihood is invariant under relabeling of components
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
42. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Label-switching issue
Additive mixture: lack of identifiability
the likelihood is invariant under relabeling of components
the posterior distribution is invariant under permutation of
components
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
43. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Label-switching issue
Additive mixture: lack of identifiability
the likelihood is invariant under relabeling of components
the posterior distribution is invariant under permutation of
components
Marginal posteriors of
Comp. #1 10 component-specific
0 parameters are identical!
Comp. #2 10
0
Comp. #3 10
0
0.5 0.75 1
ω
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
44. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Label-switching issue
Additive mixture: lack of identifiability
the likelihood is invariant under relabeling of components
the posterior distribution is invariant under permutation of
components
Marginal posteriors of
Comp. #1 10 component-specific
0 parameters are identical!
Comp. #2 10
How to summarize the
0 posterior information?
Comp. #3 10
0
0.5 0.75 1
ω
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
45. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
strategies to deal with label-switching
imposing artificial “identifiability constraints”
Exp: sorting the components [Richardson and Green, 1997.]
Comp. #1
10
0
Comp. #2 10
0
Comp. #3 10
0
0.5 0.75 1
Figure: components are sorted based on ω.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 15/ 52
46. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
strategies to deal with label-switching
imposing artificial “identifiability constraints”
Exp: sorting the components [Richardson and Green, 1997.]
Comp. #1
10
0
Comp. #2 10
0
Comp. #3 10
0
0.5 0.75 1
Figure: components are sorted based on ω.
relabeling algorithms [Celeux, et al. 1998, Stephens, 2000, Jasra,
et al, 2005, Sperrin et al, 2010, Yao, 2011.].
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 15/ 52
47. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Example 2: spectral analysis (Cont.)
Variable-dimensional posterior distribution
4
3
k
2
0 0.3 0.6 0.5 0.75 1
p(k |y) ω
Figure: Posteriors of k and sorted radial frequencies, ωk , given k.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 16/ 52
48. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Classical Bayesian approaches
Bayesian Model Selection (BMS)
One model is selected (estimated) by looking at the MAP,
ˆ
i.e. k = argmax p(k |y).
Component-specific parameters are summarized given
ˆ
k = k.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 17/ 52
49. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Bayesian Model Selection (BMS)
3
4
3
k
2
0 0.3 0.6 0.5 0.75 1
p(k |y) ω
Figure: The model with k = 2 is selected.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 18/ 52
50. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Classical Bayesian approaches
Bayesian Model Selection (BMS)
One model is selected (estimated) by looking at the MAP,
ˆ
i.e. k = argmax p(k |y).
Component-specific parameters are summarized given
ˆ
k = k.
Bayesian Model Averaging (BMA)
Use the information from all possible models:
p(∆|y) = k p(∆|k , y)p(k |y)
However, ∆ cannot be ωk as its size changes from model
to model.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 19/ 52
51. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Bayesian Model Averaging (BMA)
Binned data representation (∆ = N(Bj )):
kmax
E(N(Bj ) | y) = E(N(Bj ) | k , y) · p(k | y)
k =1
1
expected nbr comp
where .75
j = 1, . . . , Nbin .5
and E(N(Bj )) is the
.25
expected number of
components in bin 0
0 0.5 1 1.5 2 2.5 3
Bj . ω
Figure: Expected number of components
using BMA.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 20/ 52
52. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Are BMS and BMA approaches satisfactory?
Bayesian Model Selection (BMS)
« selects a model ⇒ component-specific parameters
« losing information from the discarded models
« ignoring the uncertainties about the presence of
components.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 21/ 52
53. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
Are BMS and BMA approaches satisfactory? (Cont.)
Bayesian Model Averaging (BMA)
« appropriate for signal reconstruction and prediction
« does not provide information about component-specific
parameters
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 22/ 52
54. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
A novel approach is needed!
A novel approach is needed!
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 23/ 52
55. Relabeling and summarizing posterior distributions
Proposed approach Label-switching issue
Results Variable-dimensional summarization
Conclusion
A novel approach is needed!
A novel approach is needed!
Properties of an “ideal” approach
information from all (plausible) models
« interpretable summaries for component-specific
parameters
uncertainties about the presence of components
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 23/ 52
56. Outline
1 Relabeling and summarizing posterior distributions
Label-switching issue
Variable-dimensional summarization
2 Proposed approach
An original variable-dimensional parametric model
Estimating the model parameters (SEM-type algorithms)
Robustifying strategies
3 Results
Detection and estimation of sinusoids in white noise
Detection and estimation of muons in the Auger project
4 Conclusion
57. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Big picture: relabeling and summarizing posterior
distributions
True posterior f = p(· | y) Approximate posterior qη
2 2
1 1
0 0
0 1 2 3 0 1 2 3
Parametric family {qη , η ∈ N}
Measure of “distance”
[Stephens, 2000.]
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 24/ 52
58. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Big picture: relabeling and summarizing posterior
distributions
True posterior f = p(· | y) Samples Approximate posterior qη
2 2 2
1 1 1
0 0 0
0 1 2 3 0 1 2 3 0 1 2 3
Parametric family {qη , η ∈ N}
Measure of “distance”
Samples
[Stephens, 2000.]
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 24/ 52
59. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Fixed-dimensional problems
uni-modal uni-variate case
0.6
p 0.4
0.2
0
0 2 4
Samples
report location (mean and median) and dispersion
(variance and confidence intervals) parameters
µ σ2
2.0 0.25
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 25/ 52
60. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
variable-dimensional approximate posterior
An original (variable-dimensional) parametric model qη
Four main requirements:
1 Must be defined on the same space X = k ∈K {k } × Θk
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
61. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
variable-dimensional approximate posterior
An original (variable-dimensional) parametric model qη
Four main requirements:
1 Must be defined on the same space X = k ∈K {k } × Θk
2 Must be permutation invariance
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
62. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
variable-dimensional approximate posterior
An original (variable-dimensional) parametric model qη
Four main requirements:
1 Must be defined on the same space X = k ∈K {k } × Θk
2 Must be permutation invariance
3 Must be “simple” (small number of parameters)
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
63. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
variable-dimensional approximate posterior
An original (variable-dimensional) parametric model qη
Four main requirements:
1 Must be defined on the same space X = k ∈K {k } × Θk
2 Must be permutation invariance
3 Must be “simple” (small number of parameters)
4 Must be able to capture the main features of the posterior
distributions typically met in practice.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
64. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
···
1 2 L
x = (k , θk ) ∈ X = k {k } × Θk
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
65. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
···
1 2 L
u1 u2 uL
ul ∈ Θ
x = (k , θk ) ∈ X = k {k } × Θk
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
66. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
···
1 2 L
ξ1 u1 ξ2 u2 ξL uL
ul ∈ Θ
ξl ∈ {0, 1}
x = (k , θk ) ∈ X = k {k } × Θk
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
67. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
···
1 2 L
ξ1 u1 ξ2 u2 ξL uL
ul ∈ Θ
ξl ∈ {0, 1}
{u l | ξl = 1}
x = (k , θk ) ∈ X = k {k } × Θk
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
68. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
···
1 2 L
ξ1 u1 ξ2 u2 ξL uL
ul ∈ Θ
ξl ∈ {0, 1}
{u l | ξl = 1}
random arrangement
x = (k , θk ) ∈ X = k {k } × Θk
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
69. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
π1 π2 πL
···
1 2 L
ξ1 u1 ξ2 u2 ξL uL
ul ∈ Θ
ξl ∈ {0, 1}
{u l | ξl = 1}
ξl ∼ B(πl )
random arrangement
x = (k , θk ) ∈ X = k {k } × Θk
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
70. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
π1 π2 πL
µ1 Σ1 µ2 Σ2 ··· µL ΣL
1 2 L
ξ1 u1 ξ2 u2 ξL uL
ul ∈ Θ
ξl ∈ {0, 1}
{u l | ξl = 1}
ξl ∼ B(πl )
u l ∼ N (µl , Σl )
random arrangement
x = (k , θk ) ∈ X = k {k } × Θk
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
71. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
π1 π2 πL
µ1 Σ1 µ2 Σ2 ··· µL ΣL
1 2 L
ξ1 u1 ξ2 u2 ξL uL
ul ∈ Θ
ξl ∈ {0, 1}
{u l | ξl = 1}
ξl ∼ B(πl )
u l ∼ N (µl , Σl )
random arrangement
ηl = {πl , µl , Σl }
x = (k , θk ) ∈ X = k {k } × Θk
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
72. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
1 for l = 1, . . . , L
generate binary number ξl ∼ B (πl ) end
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
73. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
1 for l = 1, . . . , L
generate binary number ξl ∼ B (πl ) end
L
2 set k = l=1 ξl ;
3 for each l such that ξl = 1
generate random sample u l ∼ N (µl , Σl ) end
4 Random arrangement ⇒ θk
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
74. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
1 for l = 1, . . . , L
generate binary number ξl ∼ B (πl ) end
L
2 set k = l=1 ξl ;
3 for each l such that ξl = 1
generate random sample u l ∼ N (µl , Σl ) end
4 Random arrangement ⇒ θk
Example:
18
L = 3
12
π = (0.4, 0.9, 0.7)
µ = (0.2, 0.5, 1)
s2 = (0.05, 0.02, 0.1) 6
0.4 0.8 1.2
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
75. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
1 for l = 1, . . . , L
generate binary number ξl ∼ B (πl ) end
L
2 set k = l=1 ξl ;
3 for each l such that ξl = 1
generate random sample u l ∼ N (µl , Σl ) end
4 Random arrangement ⇒ θk
Example:
L = 3 18
π = (0.4, 0.9, 0.7)
µ = (0.2, 0.5, 1) 12
s2 = (0.05, 0.02, 0.1)
6
ξ = (0, 1, 1) ⇒ k = 2 &
θk = (0.52, 1.05)
0.4 0.8 1.2
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
76. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
1 for l = 1, . . . , L
generate binary number ξl ∼ B (πl ) end
L
2 set k = l=1 ξl ;
3 for each l such that ξl = 1
generate random sample u l ∼ N (µl , Σl ) end
4 Random arrangement ⇒ θk
Example:
L = 3 18
π = (0.4, 0.9, 0.7)
µ = (0.2, 0.5, 1) 12
s2 = (0.05, 0.02, 0.1)
6
ξ = (0, 1, 0) ⇒ k = 1 &
θk = 0.49
0.4 0.8 1.2
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
77. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
1 for l = 1, . . . , L
generate binary number ξl ∼ B (πl ) end
L
2 set k = l=1 ξl ;
3 for each l such that ξl = 1
generate random sample u l ∼ N (µl , Σl ) end
4 Random arrangement ⇒ θk
Example:
L = 3 18
π = (0.4, 0.9, 0.7)
µ = (0.2, 0.5, 1) 12
s2 = (0.05, 0.02, 0.1)
6
ξ = (1, 0, 1) ⇒ k = 2 &
θk = (0.27, 1.03)
0.4 0.8 1.2
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
78. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
1 for l = 1, . . . , L
generate binary number ξl ∼ B (πl ) end
L
2 set k = l=1 ξl ;
3 for each l such that ξl = 1
generate random sample u l ∼ N (µl , Σl ) end
4 Random arrangement ⇒ θk
Example:
L = 3 18
π = (0.4, 0.9, 0.7)
µ = (0.2, 0.5, 1) 12
s2 = (0.05, 0.02, 0.1)
6
ξ = (0, 0, 1) ⇒ k = 1 &
θk = 1.05
0.4 0.8 1.2
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
79. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
1 for l = 1, . . . , L
generate binary number ξl ∼ B (πl ) end
L
2 set k = l=1 ξl ;
3 for each l such that ξl = 1
generate random sample u l ∼ N (µl , Σl ) end
4 Random arrangement ⇒ θk
Example:
L = 3 18
π = (0.4, 0.9, 0.7)
µ = (0.2, 0.5, 1) 12
s2 = (0.05, 0.02, 0.1)
6
ξ = (1, 1, 1) ⇒ k = 3 &
θk = (0.27, 0.53, 1.15)
0.4 0.8 1.2
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
80. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
A generative model point of view
1 for l = 1, . . . , L
generate binary number ξl ∼ B (πl ) end
L
2 set k = l=1 ξl ;
3 for each l such that ξl = 1
generate random sample u l ∼ N (µl , Σl ) end
4 Random arrangement ⇒ θk
Example:
L = 3 18
π = (0.4, 0.9, 0.7)
µ = (0.2, 0.5, 1) 12
s2 = (0.05, 0.02, 0.1)
6
ξ = (0, 0, 0) ⇒ k = 0 &
θk = ()
0.4 0.8 1.2
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
81. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Big picture: relabeling and summarizing posterior
distributions
True posterior f = p(· | y) Samples Approximate posterior qη
2 2 2
1 1 1
0 0 0
0 1 2 3 0 1 2 3 0 1 2 3
Parametric family {qη , η ∈ N}
Measure of “distance”
Samples
[Stephens, 2000.]
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 29/ 52
82. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
fitting the parametric model qη to the posterior f
minimizing the Kullback-Leibler divergence
f (x)
J (η) DKL (f (x) qη (x)) = f (x) log dx
qη (x)
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 30/ 52
83. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
fitting the parametric model qη to the posterior f
minimizing the Kullback-Leibler divergence
f (x)
J (η) DKL (f (x) qη (x)) = f (x) log dx
qη (x)
A key point: samples x (i) , i = 1, 2, · · · , M, are generated
from f , so
M
1
J (η) ≃ − log qη (x (i) ) + Const.
M
i=1
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 30/ 52
84. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
fitting the parametric model qη to the posterior f
minimizing the Kullback-Leibler divergence
f (x)
J (η) DKL (f (x) qη (x)) = f (x) log dx
qη (x)
A key point: samples x (i) , i = 1, 2, · · · , M, are generated
from f , so
M
1
J (η) ≃ − log qη (x (i) ) + Const.
M
i=1
M
Objective: η = argmaxη
ˆ i=1 log qη (x (i) ) .
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 30/ 52
85. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Expectation Maximization (EM)
latent (hidden) variable:
« Binary indicator vector ξ
« Random permutation
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
86. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Expectation Maximization (EM)
latent (hidden) variable:
« Binary indicator vector ξ
« Random permutation
« Define z = (z1 , . . . , zk ) as an allocation vector for x
« zj = l ⇒ x j comes from component l
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
87. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Expectation Maximization (EM)
latent (hidden) variable:
« Binary indicator vector ξ
« Random permutation
« Define z = (z1 , . . . , zk ) as an allocation vector for x
« zj = l ⇒ x j comes from component l
idea: use EM to maximize the likelihood.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
88. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Expectation Maximization (EM)
latent (hidden) variable:
« Binary indicator vector ξ
« Random permutation
« Define z = (z1 , . . . , zk ) as an allocation vector for x
« zj = l ⇒ x j comes from component l
idea: use EM to maximize the likelihood.
The E-step is computationally expensive!
For example, assuming L = 15 and k (i) = 10, then, it
contains 1.1 × 1010 terms.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
89. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Stochastic EM (SEM)
at iteration (r + 1)
Stochastic (S)-step: for i = 1, . . . , M
generate z(i) from p( · | x (i) , η (r ) ) end
ˆ
M
M-step: η (r +1) = argmaxη
ˆ i=1 log(p(x (i) , z(i) | η))
[Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt,
1993.]
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
90. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Stochastic EM (SEM)
at iteration (r + 1)
Stochastic (S)-step: for i = 1, . . . , M
generate z(i) from p( · | x (i) , η (r ) ) end
ˆ
M
M-step: η (r +1) = argmaxη
ˆ i=1 log(p(x (i) , z(i) | η))
[Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt,
1993.]
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
91. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Stochastic EM (SEM)
at iteration (r + 1)
Stochastic (S)-step: for i = 1, . . . , M
generate z(i) from p( · | x (i) , η (r ) ) end
ˆ
M
M-step: η (r +1) = argmaxη
ˆ i=1 log(p(x (i) , z(i) | η))
[Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt,
1993.]
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
92. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Stochastic EM (SEM)
at iteration (r + 1)
Stochastic (S)-step: for i = 1, . . . , M
generate z(i) from p( · | x (i) , η (r ) ) end
ˆ
M
M-step: η (r +1) = argmaxη
ˆ i=1 log(p(x (i) , z(i) | η))
[Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt,
1993.]
S-step
To draw z(i) ∼ p( · | x (i) , η (r ) ) we developed an I-MH sampler.
ˆ
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
93. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Example 2: spectral analysis (Cont.)
Variable-dimensional posterior distribution
4
3
k
2
0 0.3 0.6 0.5 0.75 1
p(k |y) ω
Figure: Posteriors of k and sorted radial frequencies, ωk , given k.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 33/ 52
94. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Robustifying solutions
Add a Poisson point process component
To capture the “outliers”
λ is the mean parameter
points are uniformly distributed on Θ
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
95. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Robustifying solutions
Add a Poisson point process component
To capture the “outliers”
λ is the mean parameter
points are uniformly distributed on Θ
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
96. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Robustifying solutions
Add a Poisson point process component
To capture the “outliers”
λ is the mean parameter
points are uniformly distributed on Θ
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
97. Relabeling and summarizing posterior distributions
An original variable-dimensional parametric model
Proposed approach
Estimating the model parameters (SEM-type algorithms)
Results
Robustifying strategies
Conclusion
Robustifying solutions
Add a Poisson point process component
To capture the “outliers”
λ is the mean parameter
points are uniformly distributed on Θ
Other possibilities
Robust estimates in the M-step.
« Median instead of mean
« interquartile range instead of variance
Using another divergence measure.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
98. Outline
1 Relabeling and summarizing posterior distributions
Label-switching issue
Variable-dimensional summarization
2 Proposed approach
An original variable-dimensional parametric model
Estimating the model parameters (SEM-type algorithms)
Robustifying strategies
3 Results
Detection and estimation of sinusoids in white noise
Detection and estimation of muons in the Auger project
4 Conclusion
99. Relabeling and summarizing posterior distributions
Proposed approach Detection and estimation of sinusoids in white noise
Results Detection and estimation of muons in the Auger project
Conclusion
Example 2: Spectral analysis
10
0
−10
0 10 20 30 40 50 60
time
150
100
Power
50
0
0 0.5 1 1.5 2 2.5 3
radial frequency
Figure: Observed signal (top) and its periodogram (bottom).
k
Mk : y [i] = aj cos[ωj i] + bj sin[ωj i] + n[i].
j=1
[Andrieu and Doucet, 1999.]
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 35/ 52
100. Relabeling and summarizing posterior distributions
Proposed approach Detection and estimation of sinusoids in white noise
Results Detection and estimation of muons in the Auger project
Conclusion
Example 2: spectral analysis (Cont.)
Variable-dimensional posterior distribution
4
3
k
2
0 0.3 0.6 0.5 0.75 1
p(k |y) ω
Figure: Posteriors of k and sorted radial frequencies, ωk , given k.
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 36/ 52
101. Relabeling and summarizing posterior distributions
Proposed approach Detection and estimation of sinusoids in white noise
Results Detection and estimation of muons in the Auger project
Conclusion
S-step: randomized allocation procedure
0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72
z1 z2 z1 z2 z3 z4 z1 z2 z3
M-step
norm. density
1
0.5
0
0.5 0.75 1
ω ˆ
λ = 0.1 Iter = 0
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
102. Relabeling and summarizing posterior distributions
Proposed approach Detection and estimation of sinusoids in white noise
Results Detection and estimation of muons in the Auger project
Conclusion
S-step: randomized allocation procedure
0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72
3 1 4 2 3 1 1 4 2
M-step
norm. density
1
0.5
0
0.5 0.75 1
ω ˆ
λ = 0.1 Iter = 0
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
103. Relabeling and summarizing posterior distributions
Proposed approach Detection and estimation of sinusoids in white noise
Results Detection and estimation of muons in the Auger project
Conclusion
S-step: randomized allocation procedure
0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72
3 1 4 2 3 1 1 4 2
M-step
norm. density
1
0.5
0
0.5 0.75 1
ω ˆ
λ = 0.167 Iter = 1
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
104. Relabeling and summarizing posterior distributions
Proposed approach Detection and estimation of sinusoids in white noise
Results Detection and estimation of muons in the Auger project
Conclusion
S-step: randomized allocation procedure
0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72
3 1 4 2 3 1 1 3 2
M-step
norm. density
1
0.5
0
0.5 0.75 1
ω ˆ
λ = 0.167 Iter = 1
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
105. Relabeling and summarizing posterior distributions
Proposed approach Detection and estimation of sinusoids in white noise
Results Detection and estimation of muons in the Auger project
Conclusion
S-step: randomized allocation procedure
0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72
3 1 4 2 3 1 1 3 2
M-step
norm. density
1
0.5
0
0.5 0.75 1
ω ˆ
λ = 0.226 Iter = 2
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
106. Relabeling and summarizing posterior distributions
Proposed approach Detection and estimation of sinusoids in white noise
Results Detection and estimation of muons in the Auger project
Conclusion
S-step: randomized allocation procedure
0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72
3 1 2 4 3 1 1 2 3
M-step
norm. density
1
0.5
0
0.5 0.75 1
ω ˆ
λ = 0.226 Iter = 3
Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52