SlideShare a Scribd company logo
1 of 149
Download to read offline
Relabeling and summarizing posterior distributions
                            Proposed approach
                                          Results
                                      Conclusion




             Signal decompositions using
         trans-dimensional Bayesian methods

                                       Alireza Roodaki

                                       Ph.D. Thesis Defense
               Department of Signal Processing and Electronic Systems




                                          2012, May 14th


                  Advisors: Julien Bect and Gilles Fleury

                     Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   1/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 1: Detection and estimation of muons in the
Auger project



                                                                 Ultra high energy particles
                                                                 coming from space
                                                                 (E ∼ 1019 eV)
                                                                 How and where?
                                                                 What is their composition
                                                                 (Proton, Iron)?
Figure: A conceptual shower
(http://auger.org).


                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   1/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 1: Detection and estimation of muons in the
Auger project



                                                                 Ultra high energy particles
                                                                 coming from space
                                                                 (E ∼ 1019 eV)
                                                                 How and where?
                                                                 What is their composition
                                                                 (Proton, Iron)?
Figure: A conceptual shower
(http://auger.org).


                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   1/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 1: Detection and estimation of muons in the
Auger project



                                                                 Ultra high energy particles
                                                                 coming from space
                                                                 (E ∼ 1019 eV)
                                                                 How and where?
                                                                 What is their composition
                                                                 (Proton, Iron)?
Figure: A conceptual shower
(http://auger.org).


                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   1/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 1: Detection and estimation of muons in the
Auger project (Contd.)



                                                                 muons are generated
                                                                 when particles cross the
                                                                 atmosphere
                                                                 the number k of muons
                                                                 and their arrival times t µ
                                                                 are indicators of the origin
Figure: A conceptual shower and                                  and composition of particle
detectors (water tanks)
(http://auger.org).


                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   2/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 1: Detection and estimation of muons in the
Auger project (Contd.)



                                                                 muons are generated
                                                                 when particles cross the
                                                                 atmosphere
                                                                 the number k of muons
                                                                 and their arrival times t µ
                                                                 are indicators of the origin
Figure: A conceptual shower and                                  and composition of particle
detectors (water tanks)
(http://auger.org).


                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   2/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 1: Detection and estimation of muons in the
Auger project (Contd.)



                                                                 muons are generated
                                                                 when particles cross the
                                                                 atmosphere
                                                                 the number k of muons
                                                                 and their arrival times t µ
                                                                 are indicators of the origin
Figure: A conceptual shower and                                  and composition of particle
detectors (water tanks)
(http://auger.org).


                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   2/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 1: Detection and estimation of muons in the
Auger project (Contd.)




                                  Figure: Water tank detector.


                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   3/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 1: Detection and estimation of muons in the
Auger project (Contd.)

                        #PE         20

                                    10

                                     0

                                    1.5
                        intensity




                                     1
                                    0.5
                                     0
                                     100      200    300           400     500        600
                                                           t[ns]

                                           Figure: Observed signal (n)

  Prof. Balázs Kégl from LAL, University of Paris 11.
                             Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .   4/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 1: Detection and estimation of muons in the
Auger project (Contd.)

                        #PE         20

                                    10

                                     0

                                    1.5
                        intensity




                                     1
                                    0.5
                                     0
                                     100      200    300           400     500        600
                                                           t[ns]

                                           Figure: Observed signal (n)

  Prof. Balázs Kégl from LAL, University of Paris 11.
                             Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .   4/ 52
Relabeling and summarizing posterior distributions
                                         Proposed approach
                                                       Results
                                                   Conclusion


Example 2: Spectral analysis


        10
         0
 −10                                                                 Applications
    0             10       20      30      40      50       60
                                   time                                       RADAR / SONAR
 150
 100                                                                          Array signal processing
Power




        50                                                                    Vibration analysis
        0                                                                     ...
         0        0.5       1     1.5       2      2.5       3
                            radial frequency

Figure: Observed signal (top) and its
periodogram (bottom).


                                  Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   5/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 2: Spectral analysis (Cont.)


  Detection and estimation of sinusoids in white noise
      model the observed signal y by sinusoidal components




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   6/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 2: Spectral analysis (Cont.)


  Detection and estimation of sinusoids in white noise
      model the observed signal y by sinusoidal components
        observed signal
                                            k
                  Mk : y [i] =                   aj cos[ωj i] + bj sin[ωj i] + n[i].
                                          j=1




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   6/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 2: Spectral analysis (Cont.)


  Detection and estimation of sinusoids in white noise
      model the observed signal y by sinusoidal components
        observed signal
                                            k
                  Mk : y [i] =                   aj cos[ωj i] + bj sin[ωj i] + n[i].
                                          j=1

        Joint model selection and parameter estimation problem




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   6/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Trans-dimensional problems

       Def. The problems in which the number of things that we
       don′ t know is one of the things that we don′ t know [Green,
       2003.]




                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   7/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Trans-dimensional problems

       Def. The problems in which the number of things that we
       don′ t know is one of the things that we don′ t know [Green,
       2003.]
       space X = k ∈K {k } × Θk with points x = (k , θk )
          « k ∈ K denotes number of components

              « θk ∈ Θk is a vector of component-specific
                parameters




                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   7/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Trans-dimensional problems

       Def. The problems in which the number of things that we
       don′ t know is one of the things that we don′ t know [Green,
       2003.]
       space X = k ∈K {k } × Θk with points x = (k , θk )
          « k ∈ K denotes number of components

              « θk ∈ Θk is a vector of component-specific
                parameters
       Applications:
          « Spectral Analysis (Array signal processing)

              « (Gaussian) Mixture modeling & Clustering

              « Object detection and recognition
                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   7/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Bayesian inference
       Likelihood

                                                        p(y | x) p(x)
                         p(x | y) =
                                                    X p(y   | x ′ ) p(x ′ )dx ′




                        Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .   8/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Bayesian inference
       Likelihood

                                                        p(y | x) p(x)
                         p(x | y) =
                                                    X p(y   | x ′ ) p(x ′ )dx ′

       Prior distribution




                        Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .   8/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Bayesian inference
       Likelihood

                                                        p(y | x) p(x)
                         p(x | y) =
                                                    X p(y   | x ′ ) p(x ′ )dx ′

       Prior distribution
       Posterior distribution




                        Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .   8/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Bayesian inference
       Likelihood

                                                        p(y | x) p(x)
                         p(x | y) =
                                                    X p(y   | x ′ ) p(x ′ )dx ′

       Prior distribution
       Posterior distribution

       x = (k , θk )




                        Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .   8/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Bayesian inference
       Likelihood

                                                        p(y | x) p(x)
                         p(x | y) =
                                                    X p(y   | x ′ ) p(x ′ )dx ′

       Prior distribution
       Posterior distribution

       x = (k , θk )
          « both detection and estimation problems




                        Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .   8/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Bayesian inference
       Likelihood

                                                        p(y | x) p(x)
                         p(x | y) =
                                                    X p(y   | x ′ ) p(x ′ )dx ′

       Prior distribution
       Posterior distribution

       x = (k , θk )
          « both detection and estimation problems

       high-dimensional / intractable integrals


                        Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .   8/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Markov Chain Monte Carlo (MCMC) methods

       generate samples from the posterior distribution of interest
       (target distribution), say, π.
       construct a Markov chain (x (1) , . . . , x (M) ) that under some
       conditions converges to π.




                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   9/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Markov Chain Monte Carlo (MCMC) methods

       generate samples from the posterior distribution of interest
       (target distribution), say, π.
       construct a Markov chain (x (1) , . . . , x (M) ) that under some
       conditions converges to π.
       Famous algorithms:
          « Metropolis-Hastings (MH) sampler [Metropolis, et al.
                    1953, Hastings, 1970.]

              « Gibbs sampler [Geman and Geman, 1984.]

              « RJ-MCMC sampler [Green, 1995.]
       [Robert and Casella, 2004.]


                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   9/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach
                                             Results
                                         Conclusion


Markov Chain Monte Carlo (MCMC) methods

       generate samples from the posterior distribution of interest
       (target distribution), say, π.
       construct a Markov chain (x (1) , . . . , x (M) ) that under some
       conditions converges to π.
       Famous algorithms:
          « Metropolis-Hastings (MH) sampler [Metropolis, et al.
                    1953, Hastings, 1970.]

              « Gibbs sampler [Geman and Geman, 1984.]

              « RJ-MCMC sampler [Green, 1995.]
       [Robert and Casella, 2004.]


                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   9/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach
                                              Results
                                          Conclusion


Example 2: spectral analysis (Cont.)

          RJ-MCMC sampler ⇒ variable dimensional samples
          0.8

          0.6
     ωk




          0.4



            4
            3
    k




            2

             160                    170                  180                  190                    200
                                                  Iteration number


                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   10/ 52
Outline

  1   Relabeling and summarizing posterior distributions
        Label-switching issue
        Variable-dimensional summarization
Outline

  1   Relabeling and summarizing posterior distributions
        Label-switching issue
        Variable-dimensional summarization

  2   Proposed approach
        An original variable-dimensional parametric model
        Estimating the model parameters (SEM-type algorithms)
        Robustifying strategies
Outline

  1   Relabeling and summarizing posterior distributions
        Label-switching issue
        Variable-dimensional summarization

  2   Proposed approach
        An original variable-dimensional parametric model
        Estimating the model parameters (SEM-type algorithms)
        Robustifying strategies

  3   Results
        Detection and estimation of sinusoids in white noise
        Detection and estimation of muons in the Auger project
Outline

  1   Relabeling and summarizing posterior distributions
        Label-switching issue
        Variable-dimensional summarization

  2   Proposed approach
        An original variable-dimensional parametric model
        Estimating the model parameters (SEM-type algorithms)
        Robustifying strategies

  3   Results
        Detection and estimation of sinusoids in white noise
        Detection and estimation of muons in the Auger project

  4   Conclusion
Outline

  1   Relabeling and summarizing posterior distributions
        Label-switching issue
        Variable-dimensional summarization

  2   Proposed approach
        An original variable-dimensional parametric model
        Estimating the model parameters (SEM-type algorithms)
        Robustifying strategies

  3   Results
        Detection and estimation of sinusoids in white noise
        Detection and estimation of muons in the Auger project

  4   Conclusion
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


summarizing posterior distributions

        Posterior = all the information




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   12/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


summarizing posterior distributions

        Posterior = all the information
            « It is a complex mathematical object (not easy to
               manipulate)




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   12/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


summarizing posterior distributions

        Posterior = all the information
            « It is a complex mathematical object (not easy to
               manipulate)

        (RJ)-MCMC sampler




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   12/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


summarizing posterior distributions

        Posterior = all the information
            « It is a complex mathematical object (not easy to
               manipulate)

        (RJ)-MCMC sampler
            « What to do with the generated samples?




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   12/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


summarizing posterior distributions

        Posterior = all the information
            « It is a complex mathematical object (not easy to
               manipulate)

        (RJ)-MCMC sampler
            « What to do with the generated samples?

  Summarization
     human readable summaries




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   12/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


summarizing posterior distributions

        Posterior = all the information
            « It is a complex mathematical object (not easy to
               manipulate)

        (RJ)-MCMC sampler
            « What to do with the generated samples?

  Summarization
     human readable summaries
        interpretable figures (e.g., histograms)



                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   12/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


summarizing posterior distributions

        Posterior = all the information
            « It is a complex mathematical object (not easy to
               manipulate)

        (RJ)-MCMC sampler
            « What to do with the generated samples?

  Summarization
     human readable summaries
        interpretable figures (e.g., histograms)
        statistical measures (e.g., mean and variance)

                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   12/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach            Label-switching issue
                                              Results        Variable-dimensional summarization
                                          Conclusion


Fixed-dimensional problems

  uni-modal uni-variate case

                                       0.6


                                   p   0.4


                                       0.2


                                        0
                                               0            2           4
                                                         Samples


        report location (mean and median) and dispersion
        (variance and confidence intervals) parameters
                                 µ    σ2
                                2.0 0.25


                         Alireza Roodaki (SUPELEC)           Signal decompositions using trans-dimensional . . .   13/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


Label-switching issue

  Additive mixture: lack of identifiability
        the likelihood is invariant under relabeling of components




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   14/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


Label-switching issue

  Additive mixture: lack of identifiability
        the likelihood is invariant under relabeling of components
        the posterior distribution is invariant under permutation of
        components




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   14/ 52
Relabeling and summarizing posterior distributions
                                 Proposed approach        Label-switching issue
                                               Results    Variable-dimensional summarization
                                           Conclusion


Label-switching issue

  Additive mixture: lack of identifiability
         the likelihood is invariant under relabeling of components
         the posterior distribution is invariant under permutation of
         components

                                                                Marginal posteriors of
Comp. #1 10                                                     component-specific
          0                                                     parameters are identical!
Comp. #2 10
          0
Comp. #3 10
           0
            0.5                      0.75                   1
                                      ω
                          Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   14/ 52
Relabeling and summarizing posterior distributions
                                 Proposed approach        Label-switching issue
                                               Results    Variable-dimensional summarization
                                           Conclusion


Label-switching issue

  Additive mixture: lack of identifiability
         the likelihood is invariant under relabeling of components
         the posterior distribution is invariant under permutation of
         components

                                                                Marginal posteriors of
Comp. #1 10                                                     component-specific
          0                                                     parameters are identical!
Comp. #2 10
                                                                How to summarize the
          0                                                     posterior information?
Comp. #3 10
           0
            0.5                      0.75                   1
                                      ω
                          Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   14/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


strategies to deal with label-switching

        imposing artificial “identifiability constraints”
        Exp: sorting the components [Richardson and Green, 1997.]

                   Comp. #1
                                 10
                                  0
                   Comp. #2 10
                             0
                   Comp. #3 10
                                 0
                                  0.5                    0.75                       1

                     Figure: components are sorted based on ω.




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   15/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


strategies to deal with label-switching

        imposing artificial “identifiability constraints”
        Exp: sorting the components [Richardson and Green, 1997.]

                   Comp. #1
                                 10
                                  0
                   Comp. #2 10
                             0
                   Comp. #3 10
                                 0
                                  0.5                    0.75                       1

                     Figure: components are sorted based on ω.

        relabeling algorithms [Celeux, et al. 1998, Stephens, 2000, Jasra,
        et al, 2005, Sperrin et al, 2010, Yao, 2011.].

                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   15/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


Example 2: spectral analysis (Cont.)
  Variable-dimensional posterior distribution


                        4




                        3
                   k




                        2


                         0 0.3 0.6 0.5                           0.75                         1
                             p(k |y)                               ω

   Figure: Posteriors of k and sorted radial frequencies, ωk , given k.
                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   16/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach        Label-switching issue
                                             Results    Variable-dimensional summarization
                                         Conclusion


Classical Bayesian approaches

  Bayesian Model Selection (BMS)
       One model is selected (estimated) by looking at the MAP,
            ˆ
       i.e. k = argmax p(k |y).
       Component-specific parameters are summarized given
           ˆ
       k = k.




                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   17/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach        Label-switching issue
                                             Results    Variable-dimensional summarization
                                         Conclusion


Bayesian Model Selection (BMS)
            3

                      4



                      3
                k




                      2


                          0 0.3 0.6 0.5                       0.75                         1
                            p(k |y)                              ω

                     Figure: The model with k = 2 is selected.

                          Alireza Roodaki (SUPELEC)     Signal decompositions using trans-dimensional . . .   18/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach        Label-switching issue
                                             Results    Variable-dimensional summarization
                                         Conclusion


Classical Bayesian approaches

  Bayesian Model Selection (BMS)
       One model is selected (estimated) by looking at the MAP,
            ˆ
       i.e. k = argmax p(k |y).
       Component-specific parameters are summarized given
           ˆ
       k = k.

  Bayesian Model Averaging (BMA)
       Use the information from all possible models:
       p(∆|y) = k p(∆|k , y)p(k |y)
       However, ∆ cannot be ωk as its size changes from model
       to model.


                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   19/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach         Label-switching issue
                                              Results     Variable-dimensional summarization
                                          Conclusion


Bayesian Model Averaging (BMA)
  Binned data representation (∆ = N(Bj )):
                                               kmax
                  E(N(Bj ) | y) =                     E(N(Bj ) | k , y) · p(k | y)
                                               k =1

                                                                              1




                                                          expected nbr comp
     where                                                                .75
     j = 1, . . . , Nbin                                                      .5
     and E(N(Bj )) is the
                                                                          .25
     expected number of
     components in bin                                                        0
                                                                               0 0.5 1 1.5 2 2.5 3
     Bj .                                                                               ω

                                               Figure: Expected number of components
                                               using BMA.

                         Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .   20/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach        Label-switching issue
                                             Results    Variable-dimensional summarization
                                         Conclusion


Are BMS and BMA approaches satisfactory?



  Bayesian Model Selection (BMS)
      « selects a model ⇒ component-specific parameters

      « losing information from the discarded models

      « ignoring the uncertainties about the presence of
        components.




                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   21/ 52
Relabeling and summarizing posterior distributions
                               Proposed approach        Label-switching issue
                                             Results    Variable-dimensional summarization
                                         Conclusion


Are BMS and BMA approaches satisfactory? (Cont.)




  Bayesian Model Averaging (BMA)
      « appropriate for signal reconstruction and prediction

      « does not provide information about component-specific
        parameters




                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   22/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


A novel approach is needed!



  A novel approach is needed!




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   23/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Label-switching issue
                                              Results    Variable-dimensional summarization
                                          Conclusion


A novel approach is needed!



  A novel approach is needed!

  Properties of an “ideal” approach
        information from all (plausible) models
             « interpretable summaries for component-specific
               parameters
        uncertainties about the presence of components




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   23/ 52
Outline

  1   Relabeling and summarizing posterior distributions
        Label-switching issue
        Variable-dimensional summarization

  2   Proposed approach
        An original variable-dimensional parametric model
        Estimating the model parameters (SEM-type algorithms)
        Robustifying strategies

  3   Results
        Detection and estimation of sinusoids in white noise
        Detection and estimation of muons in the Auger project

  4   Conclusion
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


Big picture: relabeling and summarizing posterior
distributions

            True posterior f = p(· | y)                                       Approximate posterior qη


        2                                                                 2

        1                                                                 1

        0                                                                 0
             0        1       2           3                                    0       1      2          3


        Parametric family {qη , η ∈ N}
        Measure of “distance”


  [Stephens, 2000.]

                            Alireza Roodaki (SUPELEC)    Signal decompositions using trans-dimensional . . .   24/ 52
Relabeling and summarizing posterior distributions
                                                             An original variable-dimensional parametric model
                                Proposed approach
                                                             Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                             Robustifying strategies
                                          Conclusion


Big picture: relabeling and summarizing posterior
distributions

            True posterior f = p(· | y)               Samples                     Approximate posterior qη


        2                                     2                               2

        1                                     1                               1

        0                                     0                               0
             0        1       2           3       0      1      2    3             0       1      2          3


        Parametric family {qη , η ∈ N}
        Measure of “distance”
        Samples
  [Stephens, 2000.]

                            Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .   24/ 52
Relabeling and summarizing posterior distributions
                                                             An original variable-dimensional parametric model
                                Proposed approach
                                                             Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                             Robustifying strategies
                                          Conclusion


Fixed-dimensional problems

  uni-modal uni-variate case

                                       0.6


                                   p   0.4


                                       0.2


                                        0
                                               0            2           4
                                                         Samples


        report location (mean and median) and dispersion
        (variance and confidence intervals) parameters
                                 µ    σ2
                                2.0 0.25


                         Alireza Roodaki (SUPELEC)           Signal decompositions using trans-dimensional . . .   25/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


variable-dimensional approximate posterior


  An original (variable-dimensional) parametric model qη
  Four main requirements:
   1    Must be defined on the same space X =                                      k ∈K {k }      × Θk




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   26/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


variable-dimensional approximate posterior


  An original (variable-dimensional) parametric model qη
  Four main requirements:
   1    Must be defined on the same space X =                                      k ∈K {k }      × Θk
   2    Must be permutation invariance




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   26/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


variable-dimensional approximate posterior


  An original (variable-dimensional) parametric model qη
  Four main requirements:
   1    Must be defined on the same space X =                                      k ∈K {k }      × Θk
   2    Must be permutation invariance
   3    Must be “simple” (small number of parameters)




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   26/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


variable-dimensional approximate posterior


  An original (variable-dimensional) parametric model qη
  Four main requirements:
   1    Must be defined on the same space X =                                      k ∈K {k }      × Θk
   2    Must be permutation invariance
   3    Must be “simple” (small number of parameters)
   4    Must be able to capture the main features of the posterior
        distributions typically met in practice.




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   26/ 52
Relabeling and summarizing posterior distributions
                                                           An original variable-dimensional parametric model
                                Proposed approach
                                                           Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                           Robustifying strategies
                                          Conclusion


A generative model point of view


                                               ···
             1                        2                                  L




            x = (k , θk ) ∈ X =            k {k }   × Θk

                         Alireza Roodaki (SUPELEC)         Signal decompositions using trans-dimensional . . .   27/ 52
Relabeling and summarizing posterior distributions
                                                           An original variable-dimensional parametric model
                                Proposed approach
                                                           Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                           Robustifying strategies
                                          Conclusion


A generative model point of view


                                               ···
             1                        2                                  L
        u1                       u2                                 uL
                                                                                       ul ∈ Θ




             x = (k , θk ) ∈ X =           k {k }   × Θk

                         Alireza Roodaki (SUPELEC)         Signal decompositions using trans-dimensional . . .   27/ 52
Relabeling and summarizing posterior distributions
                                                             An original variable-dimensional parametric model
                                  Proposed approach
                                                             Estimating the model parameters (SEM-type algorithms)
                                                Results
                                                             Robustifying strategies
                                            Conclusion


A generative model point of view


                                                 ···
               1                        2                                  L
 ξ1       u1             ξ2        u2                        ξL       uL
                                                                                         ul ∈ Θ
                                                                                         ξl ∈ {0, 1}




               x = (k , θk ) ∈ X =           k {k }   × Θk

                           Alireza Roodaki (SUPELEC)         Signal decompositions using trans-dimensional . . .   27/ 52
Relabeling and summarizing posterior distributions
                                                             An original variable-dimensional parametric model
                                  Proposed approach
                                                             Estimating the model parameters (SEM-type algorithms)
                                                Results
                                                             Robustifying strategies
                                            Conclusion


A generative model point of view


                                                 ···
               1                        2                                  L
 ξ1       u1             ξ2        u2                        ξL       uL
                                                                                         ul ∈ Θ
                                                                                         ξl ∈ {0, 1}

                              {u l | ξl = 1}




               x = (k , θk ) ∈ X =           k {k }   × Θk

                           Alireza Roodaki (SUPELEC)         Signal decompositions using trans-dimensional . . .   27/ 52
Relabeling and summarizing posterior distributions
                                                             An original variable-dimensional parametric model
                                  Proposed approach
                                                             Estimating the model parameters (SEM-type algorithms)
                                                Results
                                                             Robustifying strategies
                                            Conclusion


A generative model point of view


                                                 ···
               1                        2                                  L
 ξ1       u1             ξ2        u2                        ξL       uL
                                                                                         ul ∈ Θ
                                                                                         ξl ∈ {0, 1}

                              {u l | ξl = 1}



                      random arrangement



               x = (k , θk ) ∈ X =           k {k }   × Θk

                           Alireza Roodaki (SUPELEC)         Signal decompositions using trans-dimensional . . .   27/ 52
Relabeling and summarizing posterior distributions
                                                             An original variable-dimensional parametric model
                                  Proposed approach
                                                             Estimating the model parameters (SEM-type algorithms)
                                                Results
                                                             Robustifying strategies
                                            Conclusion


A generative model point of view


 π1                      π2                                  πL
                                                 ···
               1                        2                                  L
 ξ1       u1             ξ2        u2                        ξL       uL
                                                                                         ul ∈ Θ
                                                                                         ξl ∈ {0, 1}

                              {u l | ξl = 1}
                                                                                         ξl ∼ B(πl )

                      random arrangement



               x = (k , θk ) ∈ X =           k {k }   × Θk

                           Alireza Roodaki (SUPELEC)         Signal decompositions using trans-dimensional . . .   27/ 52
Relabeling and summarizing posterior distributions
                                                              An original variable-dimensional parametric model
                                  Proposed approach
                                                              Estimating the model parameters (SEM-type algorithms)
                                                Results
                                                              Robustifying strategies
                                            Conclusion


A generative model point of view


 π1                      π2                                   πL
      µ1      Σ1                µ2     Σ2        ···               µL      ΣL
                1                         2                                  L
 ξ1        u1            ξ2          u2                       ξL        uL
                                                                                          ul ∈ Θ
                                                                                          ξl ∈ {0, 1}

                              {u l | ξl = 1}
                                                                                          ξl ∼ B(πl )
                                                                                          u l ∼ N (µl , Σl )
                      random arrangement



                x = (k , θk ) ∈ X =           k {k }   × Θk

                           Alireza Roodaki (SUPELEC)          Signal decompositions using trans-dimensional . . .   27/ 52
Relabeling and summarizing posterior distributions
                                                              An original variable-dimensional parametric model
                                  Proposed approach
                                                              Estimating the model parameters (SEM-type algorithms)
                                                Results
                                                              Robustifying strategies
                                            Conclusion


A generative model point of view


 π1                      π2                                   πL
      µ1      Σ1                µ2     Σ2        ···               µL      ΣL
                1                         2                                  L
 ξ1        u1            ξ2          u2                       ξL        uL
                                                                                          ul ∈ Θ
                                                                                          ξl ∈ {0, 1}

                              {u l | ξl = 1}
                                                                                          ξl ∼ B(πl )
                                                                                          u l ∼ N (µl , Σl )
                      random arrangement
                                                                                          ηl = {πl , µl , Σl }


                x = (k , θk ) ∈ X =           k {k }   × Θk

                           Alireza Roodaki (SUPELEC)          Signal decompositions using trans-dimensional . . .   27/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


A generative model point of view


   1    for l = 1, . . . , L
                generate binary number ξl ∼ B (πl ) end




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   28/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


A generative model point of view


   1    for l = 1, . . . , L
                generate binary number ξl ∼ B (πl ) end
                          L
   2    set k =           l=1 ξl ;
   3    for each l such that ξl = 1
                generate random sample u l ∼ N (µl , Σl ) end
   4    Random arrangement ⇒ θk




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   28/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


A generative model point of view
   1    for l = 1, . . . , L
                generate binary number ξl ∼ B (πl ) end
                          L
   2    set k =           l=1 ξl ;
   3    for each l such that ξl = 1
                generate random sample u l ∼ N (µl , Σl ) end
   4 Random arrangement ⇒ θk
  Example:
                                                             18


        L = 3
                                                             12
       π = (0.4, 0.9, 0.7)
       µ = (0.2, 0.5, 1)
       s2 = (0.05, 0.02, 0.1)                                 6



                                                                         0.4         0.8        1.2

                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   28/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


A generative model point of view
   1    for l = 1, . . . , L
                generate binary number ξl ∼ B (πl ) end
                          L
   2    set k =           l=1 ξl ;
   3    for each l such that ξl = 1
                generate random sample u l ∼ N (µl , Σl ) end
   4 Random arrangement ⇒ θk
  Example:
        L = 3                                                18
       π = (0.4, 0.9, 0.7)
       µ = (0.2, 0.5, 1)                                     12
       s2 = (0.05, 0.02, 0.1)

                                                              6
       ξ = (0, 1, 1) ⇒ k = 2 &
       θk = (0.52, 1.05)
                                                                         0.4         0.8        1.2

                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   28/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


A generative model point of view
   1    for l = 1, . . . , L
                generate binary number ξl ∼ B (πl ) end
                          L
   2    set k =           l=1 ξl ;
   3    for each l such that ξl = 1
                generate random sample u l ∼ N (µl , Σl ) end
   4 Random arrangement ⇒ θk
  Example:
        L = 3                                                18
       π = (0.4, 0.9, 0.7)
       µ = (0.2, 0.5, 1)                                     12
       s2 = (0.05, 0.02, 0.1)

                                                              6
       ξ = (0, 1, 0) ⇒ k = 1 &
       θk = 0.49
                                                                         0.4         0.8        1.2

                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   28/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


A generative model point of view
   1    for l = 1, . . . , L
                generate binary number ξl ∼ B (πl ) end
                          L
   2    set k =           l=1 ξl ;
   3    for each l such that ξl = 1
                generate random sample u l ∼ N (µl , Σl ) end
   4 Random arrangement ⇒ θk
  Example:
        L = 3                                                18
       π = (0.4, 0.9, 0.7)
       µ = (0.2, 0.5, 1)                                     12
       s2 = (0.05, 0.02, 0.1)

                                                              6
       ξ = (1, 0, 1) ⇒ k = 2 &
       θk = (0.27, 1.03)
                                                                         0.4         0.8        1.2

                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   28/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


A generative model point of view
   1    for l = 1, . . . , L
                generate binary number ξl ∼ B (πl ) end
                          L
   2    set k =           l=1 ξl ;
   3    for each l such that ξl = 1
                generate random sample u l ∼ N (µl , Σl ) end
   4 Random arrangement ⇒ θk
  Example:
        L = 3                                                18
       π = (0.4, 0.9, 0.7)
       µ = (0.2, 0.5, 1)                                     12
       s2 = (0.05, 0.02, 0.1)

                                                              6
       ξ = (0, 0, 1) ⇒ k = 1 &
       θk = 1.05
                                                                         0.4         0.8        1.2

                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   28/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


A generative model point of view
   1    for l = 1, . . . , L
                generate binary number ξl ∼ B (πl ) end
                          L
   2    set k =           l=1 ξl ;
   3    for each l such that ξl = 1
                generate random sample u l ∼ N (µl , Σl ) end
   4 Random arrangement ⇒ θk
  Example:
        L = 3                                                18
       π = (0.4, 0.9, 0.7)
       µ = (0.2, 0.5, 1)                                     12
       s2 = (0.05, 0.02, 0.1)

                                                              6
       ξ = (1, 1, 1) ⇒ k = 3 &
       θk = (0.27, 0.53, 1.15)
                                                                         0.4         0.8        1.2

                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   28/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


A generative model point of view
   1    for l = 1, . . . , L
                generate binary number ξl ∼ B (πl ) end
                          L
   2    set k =           l=1 ξl ;
   3    for each l such that ξl = 1
                generate random sample u l ∼ N (µl , Σl ) end
   4 Random arrangement ⇒ θk
  Example:
        L = 3                                                18
       π = (0.4, 0.9, 0.7)
       µ = (0.2, 0.5, 1)                                     12
       s2 = (0.05, 0.02, 0.1)

                                                              6
       ξ = (0, 0, 0) ⇒ k = 0 &
       θk = ()
                                                                         0.4         0.8        1.2

                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   28/ 52
Relabeling and summarizing posterior distributions
                                                             An original variable-dimensional parametric model
                                Proposed approach
                                                             Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                             Robustifying strategies
                                          Conclusion


Big picture: relabeling and summarizing posterior
distributions

            True posterior f = p(· | y)               Samples                     Approximate posterior qη


        2                                     2                               2

        1                                     1                               1

        0                                     0                               0
             0        1       2           3       0      1      2    3             0       1      2          3


        Parametric family {qη , η ∈ N}
        Measure of “distance”
        Samples
  [Stephens, 2000.]

                            Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .   29/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


fitting the parametric model qη to the posterior f

        minimizing the Kullback-Leibler divergence
                                                                                       f (x)
                 J (η)         DKL (f (x) qη (x)) =                    f (x) log             dx
                                                                                      qη (x)




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   30/ 52
Relabeling and summarizing posterior distributions
                                                           An original variable-dimensional parametric model
                                Proposed approach
                                                           Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                           Robustifying strategies
                                          Conclusion


fitting the parametric model qη to the posterior f

        minimizing the Kullback-Leibler divergence
                                                                                         f (x)
                 J (η)         DKL (f (x) qη (x)) =                      f (x) log             dx
                                                                                        qη (x)

        A key point: samples x (i) , i = 1, 2, · · · , M, are generated
        from f , so
                                                  M
                                           1
                         J (η) ≃ −                       log qη (x (i) ) + Const.
                                           M
                                                 i=1




                         Alireza Roodaki (SUPELEC)         Signal decompositions using trans-dimensional . . .   30/ 52
Relabeling and summarizing posterior distributions
                                                           An original variable-dimensional parametric model
                                Proposed approach
                                                           Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                           Robustifying strategies
                                          Conclusion


fitting the parametric model qη to the posterior f

        minimizing the Kullback-Leibler divergence
                                                                                         f (x)
                 J (η)         DKL (f (x) qη (x)) =                      f (x) log             dx
                                                                                        qη (x)

        A key point: samples x (i) , i = 1, 2, · · · , M, are generated
        from f , so
                                                  M
                                           1
                         J (η) ≃ −                       log qη (x (i) ) + Const.
                                           M
                                                 i=1

                                                         M
        Objective: η = argmaxη
                   ˆ                                     i=1 log     qη (x (i) ) .



                         Alireza Roodaki (SUPELEC)         Signal decompositions using trans-dimensional . . .   30/ 52
Relabeling and summarizing posterior distributions
                                                        An original variable-dimensional parametric model
                               Proposed approach
                                                        Estimating the model parameters (SEM-type algorithms)
                                             Results
                                                        Robustifying strategies
                                         Conclusion


Expectation Maximization (EM)

       latent (hidden) variable:
           « Binary indicator vector ξ

              « Random permutation




                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   31/ 52
Relabeling and summarizing posterior distributions
                                                        An original variable-dimensional parametric model
                               Proposed approach
                                                        Estimating the model parameters (SEM-type algorithms)
                                             Results
                                                        Robustifying strategies
                                         Conclusion


Expectation Maximization (EM)

       latent (hidden) variable:
           « Binary indicator vector ξ

              « Random permutation

              « Define z = (z1 , . . . , zk ) as an allocation vector for x

              « zj = l ⇒ x j comes from component l




                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   31/ 52
Relabeling and summarizing posterior distributions
                                                        An original variable-dimensional parametric model
                               Proposed approach
                                                        Estimating the model parameters (SEM-type algorithms)
                                             Results
                                                        Robustifying strategies
                                         Conclusion


Expectation Maximization (EM)

       latent (hidden) variable:
           « Binary indicator vector ξ

              « Random permutation

              « Define z = (z1 , . . . , zk ) as an allocation vector for x

              « zj = l ⇒ x j comes from component l
       idea: use EM to maximize the likelihood.




                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   31/ 52
Relabeling and summarizing posterior distributions
                                                        An original variable-dimensional parametric model
                               Proposed approach
                                                        Estimating the model parameters (SEM-type algorithms)
                                             Results
                                                        Robustifying strategies
                                         Conclusion


Expectation Maximization (EM)

       latent (hidden) variable:
           « Binary indicator vector ξ

              « Random permutation

              « Define z = (z1 , . . . , zk ) as an allocation vector for x

              « zj = l ⇒ x j comes from component l
       idea: use EM to maximize the likelihood.

       The E-step is computationally expensive!
       For example, assuming L = 15 and k (i) = 10, then, it
       contains 1.1 × 1010 terms.

                        Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   31/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


Stochastic EM (SEM)


  at iteration (r + 1)
        Stochastic (S)-step: for i = 1, . . . , M
        generate z(i) from p( · | x (i) , η (r ) ) end
                                          ˆ
                                                         M
        M-step: η (r +1) = argmaxη
                ˆ                                        i=1    log(p(x (i) , z(i) | η))
  [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt,
  1993.]




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   32/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


Stochastic EM (SEM)


  at iteration (r + 1)
        Stochastic (S)-step: for i = 1, . . . , M
        generate z(i) from p( · | x (i) , η (r ) ) end
                                          ˆ
                                                         M
        M-step: η (r +1) = argmaxη
                ˆ                                        i=1    log(p(x (i) , z(i) | η))
  [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt,
  1993.]




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   32/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


Stochastic EM (SEM)


  at iteration (r + 1)
        Stochastic (S)-step: for i = 1, . . . , M
        generate z(i) from p( · | x (i) , η (r ) ) end
                                          ˆ
                                                         M
        M-step: η (r +1) = argmaxη
                ˆ                                        i=1    log(p(x (i) , z(i) | η))
  [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt,
  1993.]




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   32/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


Stochastic EM (SEM)


  at iteration (r + 1)
        Stochastic (S)-step: for i = 1, . . . , M
        generate z(i) from p( · | x (i) , η (r ) ) end
                                          ˆ
                                                         M
        M-step: η (r +1) = argmaxη
                ˆ                                        i=1    log(p(x (i) , z(i) | η))
  [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt,
  1993.]

  S-step
  To draw z(i) ∼ p( · | x (i) , η (r ) ) we developed an I-MH sampler.
                                ˆ




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   32/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


Example 2: spectral analysis (Cont.)
  Variable-dimensional posterior distribution


                        4




                        3
                   k




                        2


                         0 0.3 0.6 0.5                           0.75                         1
                             p(k |y)                               ω

   Figure: Posteriors of k and sorted radial frequencies, ωk , given k.
                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   33/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


Robustifying solutions

  Add a Poisson point process component
        To capture the “outliers”
        λ is the mean parameter
        points are uniformly distributed on Θ




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   34/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


Robustifying solutions

  Add a Poisson point process component
        To capture the “outliers”
        λ is the mean parameter
        points are uniformly distributed on Θ




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   34/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


Robustifying solutions

  Add a Poisson point process component
        To capture the “outliers”
        λ is the mean parameter
        points are uniformly distributed on Θ




                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   34/ 52
Relabeling and summarizing posterior distributions
                                                         An original variable-dimensional parametric model
                                Proposed approach
                                                         Estimating the model parameters (SEM-type algorithms)
                                              Results
                                                         Robustifying strategies
                                          Conclusion


Robustifying solutions

  Add a Poisson point process component
        To capture the “outliers”
        λ is the mean parameter
        points are uniformly distributed on Θ

  Other possibilities
        Robust estimates in the M-step.
           « Median instead of mean

               « interquartile range instead of variance
        Using another divergence measure.


                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .   34/ 52
Outline

  1   Relabeling and summarizing posterior distributions
        Label-switching issue
        Variable-dimensional summarization

  2   Proposed approach
        An original variable-dimensional parametric model
        Estimating the model parameters (SEM-type algorithms)
        Robustifying strategies

  3   Results
        Detection and estimation of sinusoids in white noise
        Detection and estimation of muons in the Auger project

  4   Conclusion
Relabeling and summarizing posterior distributions
                                Proposed approach             Detection and estimation of sinusoids in white noise
                                              Results         Detection and estimation of muons in the Auger project
                                          Conclusion


Example 2: Spectral analysis
                                  10
                                   0
                           −10
                              0         10       20      30       40       50      60
                                                         time
                           150
                           100
                          Power




                                  50
                                  0
                                   0   0.5       1       1.5       2      2.5       3
                                                 radial frequency

                 Figure: Observed signal (top) and its periodogram (bottom).


                                             k
                        Mk : y [i] =             aj cos[ωj i] + bj sin[ωj i] + n[i].
                                          j=1

  [Andrieu and Doucet, 1999.]
                         Alireza Roodaki (SUPELEC)            Signal decompositions using trans-dimensional . . .      35/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach        Detection and estimation of sinusoids in white noise
                                              Results    Detection and estimation of muons in the Auger project
                                          Conclusion


Example 2: spectral analysis (Cont.)
  Variable-dimensional posterior distribution


                        4




                        3
                   k




                        2


                         0 0.3 0.6 0.5                           0.75                         1
                             p(k |y)                               ω

   Figure: Posteriors of k and sorted radial frequencies, ωk , given k.
                         Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .      36/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach             Detection and estimation of sinusoids in white noise
                                              Results         Detection and estimation of muons in the Auger project
                                          Conclusion


S-step: randomized allocation procedure

        0.72 0.62                0.73 0.83 0.67 0.64                            0.64 0.74 0.72


                 z1     z2       z1         z2           z3          z4         z1          z2          z3


M-step
 norm. density




                  1

                 0.5

                  0
                  0.5                 0.75                      1
                                       ω                                       ˆ
                                                                               λ = 0.1            Iter = 0

                             Alireza Roodaki (SUPELEC)        Signal decompositions using trans-dimensional . . .      37/ 52
Relabeling and summarizing posterior distributions
                                Proposed approach            Detection and estimation of sinusoids in white noise
                                              Results        Detection and estimation of muons in the Auger project
                                          Conclusion


S-step: randomized allocation procedure

           0.72 0.62              0.73 0.83 0.67 0.64                          0.64 0.74 0.72


                 3       1       4          2            3           1        1            4            2


M-step
 norm. density




                     1

                 0.5

                  0
                  0.5                0.75                      1
                                      ω                                       ˆ
                                                                              λ = 0.1            Iter = 0

                             Alireza Roodaki (SUPELEC)       Signal decompositions using trans-dimensional . . .      37/ 52
Relabeling and summarizing posterior distributions
                                       Proposed approach            Detection and estimation of sinusoids in white noise
                                                     Results        Detection and estimation of muons in the Auger project
                                                 Conclusion


S-step: randomized allocation procedure

                0.72 0.62             0.73 0.83 0.67 0.64                             0.64 0.74 0.72


                 3          1        4            2             3           1        1            4            2


M-step
norm. density




                 1

                0.5

                 0
                 0.5                     0.75                       1
                                          ω                                        ˆ
                                                                                   λ = 0.167              Iter = 1

                                Alireza Roodaki (SUPELEC)           Signal decompositions using trans-dimensional . . .      37/ 52
Relabeling and summarizing posterior distributions
                                       Proposed approach            Detection and estimation of sinusoids in white noise
                                                     Results        Detection and estimation of muons in the Auger project
                                                 Conclusion


S-step: randomized allocation procedure

                0.72 0.62             0.73 0.83 0.67 0.64                             0.64 0.74 0.72


                 3          1        4            2             3           1        1            3            2


M-step
norm. density




                 1

                0.5

                 0
                 0.5                     0.75                       1
                                          ω                                        ˆ
                                                                                   λ = 0.167              Iter = 1

                                Alireza Roodaki (SUPELEC)           Signal decompositions using trans-dimensional . . .      37/ 52
Relabeling and summarizing posterior distributions
                                       Proposed approach            Detection and estimation of sinusoids in white noise
                                                     Results        Detection and estimation of muons in the Auger project
                                                 Conclusion


S-step: randomized allocation procedure

                0.72 0.62             0.73 0.83 0.67 0.64                             0.64 0.74 0.72


                 3          1        4            2             3           1        1            3            2


M-step
norm. density




                 1

                0.5

                 0
                 0.5                     0.75                       1
                                          ω                                        ˆ
                                                                                   λ = 0.226              Iter = 2

                                Alireza Roodaki (SUPELEC)           Signal decompositions using trans-dimensional . . .      37/ 52
Relabeling and summarizing posterior distributions
                                       Proposed approach            Detection and estimation of sinusoids in white noise
                                                     Results        Detection and estimation of muons in the Auger project
                                                 Conclusion


S-step: randomized allocation procedure

                0.72 0.62             0.73 0.83 0.67 0.64                             0.64 0.74 0.72


                 3          1        2            4             3           1        1            2            3


M-step
norm. density




                 1

                0.5

                 0
                 0.5                     0.75                       1
                                          ω                                        ˆ
                                                                                   λ = 0.226              Iter = 3

                                Alireza Roodaki (SUPELEC)           Signal decompositions using trans-dimensional . . .      37/ 52
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense

More Related Content

Viewers also liked

Top Cross Section Measurement
Top Cross Section MeasurementTop Cross Section Measurement
Top Cross Section MeasurementAkira Shibata
 
Results from WZ, Higgs and Top for the Summer Conferences
Results from WZ, Higgs and Top for the Summer ConferencesResults from WZ, Higgs and Top for the Summer Conferences
Results from WZ, Higgs and Top for the Summer ConferencesMedicineAndHealthNeurolog
 
Presentation on the inclusive analysis
Presentation on the inclusive analysisPresentation on the inclusive analysis
Presentation on the inclusive analysisRuturaj Apte
 
Searches with LHC
Searches with LHCSearches with LHC
Searches with LHCkneemo
 
Bimolecular chemical reaction in two-dimensional Navier-Stokes flow
Bimolecular chemical reaction in two-dimensional Navier-Stokes flowBimolecular chemical reaction in two-dimensional Navier-Stokes flow
Bimolecular chemical reaction in two-dimensional Navier-Stokes flowFarid Ait-Chaalal, PhD
 
Search for Excited Randall-Sundrum Gravitons from Warped Extra Dimensions wit...
Search for Excited Randall-Sundrum Gravitons from Warped Extra Dimensions wit...Search for Excited Randall-Sundrum Gravitons from Warped Extra Dimensions wit...
Search for Excited Randall-Sundrum Gravitons from Warped Extra Dimensions wit...Eric Williams
 
The search for new resonances using the ATLAS detector at the LHC: A statisti...
The search for new resonances using the ATLAS detector at the LHC: A statisti...The search for new resonances using the ATLAS detector at the LHC: A statisti...
The search for new resonances using the ATLAS detector at the LHC: A statisti...Eric Williams
 
Dissertation Defense Presentation
Dissertation Defense PresentationDissertation Defense Presentation
Dissertation Defense PresentationBrian Dorney
 

Viewers also liked (9)

Notre Dame Afb Seminar 2009
Notre Dame Afb Seminar 2009Notre Dame Afb Seminar 2009
Notre Dame Afb Seminar 2009
 
Top Cross Section Measurement
Top Cross Section MeasurementTop Cross Section Measurement
Top Cross Section Measurement
 
Results from WZ, Higgs and Top for the Summer Conferences
Results from WZ, Higgs and Top for the Summer ConferencesResults from WZ, Higgs and Top for the Summer Conferences
Results from WZ, Higgs and Top for the Summer Conferences
 
Presentation on the inclusive analysis
Presentation on the inclusive analysisPresentation on the inclusive analysis
Presentation on the inclusive analysis
 
Searches with LHC
Searches with LHCSearches with LHC
Searches with LHC
 
Bimolecular chemical reaction in two-dimensional Navier-Stokes flow
Bimolecular chemical reaction in two-dimensional Navier-Stokes flowBimolecular chemical reaction in two-dimensional Navier-Stokes flow
Bimolecular chemical reaction in two-dimensional Navier-Stokes flow
 
Search for Excited Randall-Sundrum Gravitons from Warped Extra Dimensions wit...
Search for Excited Randall-Sundrum Gravitons from Warped Extra Dimensions wit...Search for Excited Randall-Sundrum Gravitons from Warped Extra Dimensions wit...
Search for Excited Randall-Sundrum Gravitons from Warped Extra Dimensions wit...
 
The search for new resonances using the ATLAS detector at the LHC: A statisti...
The search for new resonances using the ATLAS detector at the LHC: A statisti...The search for new resonances using the ATLAS detector at the LHC: A statisti...
The search for new resonances using the ATLAS detector at the LHC: A statisti...
 
Dissertation Defense Presentation
Dissertation Defense PresentationDissertation Defense Presentation
Dissertation Defense Presentation
 

Similar to Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense

Deep Impact Poster Revised
Deep Impact Poster RevisedDeep Impact Poster Revised
Deep Impact Poster RevisedAlejandro Cota
 
APPLICATION OF TAGUCHI METHOD FOR PARAMETRIC STUDIES OF A FUNNEL SHAPED STRUC...
APPLICATION OF TAGUCHI METHOD FOR PARAMETRIC STUDIES OF A FUNNEL SHAPED STRUC...APPLICATION OF TAGUCHI METHOD FOR PARAMETRIC STUDIES OF A FUNNEL SHAPED STRUC...
APPLICATION OF TAGUCHI METHOD FOR PARAMETRIC STUDIES OF A FUNNEL SHAPED STRUC...ijmech
 
Ultrasonic wave generation by time gated microwaves
Ultrasonic wave generation by time gated microwavesUltrasonic wave generation by time gated microwaves
Ultrasonic wave generation by time gated microwavesMidoOoz
 
DEVELOPMENT OF OPTICAL PARAMETER CALCULATIONS OF THE PROBES IN WATER
DEVELOPMENT OF OPTICAL PARAMETER CALCULATIONS OF THE PROBES IN WATERDEVELOPMENT OF OPTICAL PARAMETER CALCULATIONS OF THE PROBES IN WATER
DEVELOPMENT OF OPTICAL PARAMETER CALCULATIONS OF THE PROBES IN WATERDr. Ved Nath Jha
 
Scanning Electron Microscopes (SEM) & Transmission Electron Microscopes (TEM)
Scanning Electron Microscopes (SEM) & Transmission Electron Microscopes (TEM)Scanning Electron Microscopes (SEM) & Transmission Electron Microscopes (TEM)
Scanning Electron Microscopes (SEM) & Transmission Electron Microscopes (TEM)Shahbaz844
 
Microwave emission and scattering of foam
Microwave emission and scattering of foamMicrowave emission and scattering of foam
Microwave emission and scattering of foamRoopam Poddar
 
Ultraviolet visible (uv vis) spectroscopy Likhith K
Ultraviolet visible (uv vis) spectroscopy Likhith KUltraviolet visible (uv vis) spectroscopy Likhith K
Ultraviolet visible (uv vis) spectroscopy Likhith KLIKHITHK1
 
Wave_discrimination_at_C-band_frequencies_in_micro.pdf
Wave_discrimination_at_C-band_frequencies_in_micro.pdfWave_discrimination_at_C-band_frequencies_in_micro.pdf
Wave_discrimination_at_C-band_frequencies_in_micro.pdfDeepakSinghNagarkoti
 
Behavior of Ultrasound Energy in the Presence of Obstacle
Behavior of Ultrasound Energy in the Presence of ObstacleBehavior of Ultrasound Energy in the Presence of Obstacle
Behavior of Ultrasound Energy in the Presence of ObstacleIRJET Journal
 
A weak lensing_mass_reconstruction_of _the_large_scale_filament_massive_galax...
A weak lensing_mass_reconstruction_of _the_large_scale_filament_massive_galax...A weak lensing_mass_reconstruction_of _the_large_scale_filament_massive_galax...
A weak lensing_mass_reconstruction_of _the_large_scale_filament_massive_galax...Sérgio Sacani
 

Similar to Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense (20)

Deep Impact Poster Revised
Deep Impact Poster RevisedDeep Impact Poster Revised
Deep Impact Poster Revised
 
APPLICATION OF TAGUCHI METHOD FOR PARAMETRIC STUDIES OF A FUNNEL SHAPED STRUC...
APPLICATION OF TAGUCHI METHOD FOR PARAMETRIC STUDIES OF A FUNNEL SHAPED STRUC...APPLICATION OF TAGUCHI METHOD FOR PARAMETRIC STUDIES OF A FUNNEL SHAPED STRUC...
APPLICATION OF TAGUCHI METHOD FOR PARAMETRIC STUDIES OF A FUNNEL SHAPED STRUC...
 
Ultrasonic wave generation by time gated microwaves
Ultrasonic wave generation by time gated microwavesUltrasonic wave generation by time gated microwaves
Ultrasonic wave generation by time gated microwaves
 
DEVELOPMENT OF OPTICAL PARAMETER CALCULATIONS OF THE PROBES IN WATER
DEVELOPMENT OF OPTICAL PARAMETER CALCULATIONS OF THE PROBES IN WATERDEVELOPMENT OF OPTICAL PARAMETER CALCULATIONS OF THE PROBES IN WATER
DEVELOPMENT OF OPTICAL PARAMETER CALCULATIONS OF THE PROBES IN WATER
 
Scanning Electron Microscopes (SEM) & Transmission Electron Microscopes (TEM)
Scanning Electron Microscopes (SEM) & Transmission Electron Microscopes (TEM)Scanning Electron Microscopes (SEM) & Transmission Electron Microscopes (TEM)
Scanning Electron Microscopes (SEM) & Transmission Electron Microscopes (TEM)
 
Microwave emission and scattering of foam
Microwave emission and scattering of foamMicrowave emission and scattering of foam
Microwave emission and scattering of foam
 
Ultraviolet visible (uv vis) spectroscopy Likhith K
Ultraviolet visible (uv vis) spectroscopy Likhith KUltraviolet visible (uv vis) spectroscopy Likhith K
Ultraviolet visible (uv vis) spectroscopy Likhith K
 
Wave_discrimination_at_C-band_frequencies_in_micro.pdf
Wave_discrimination_at_C-band_frequencies_in_micro.pdfWave_discrimination_at_C-band_frequencies_in_micro.pdf
Wave_discrimination_at_C-band_frequencies_in_micro.pdf
 
report
reportreport
report
 
Behavior of Ultrasound Energy in the Presence of Obstacle
Behavior of Ultrasound Energy in the Presence of ObstacleBehavior of Ultrasound Energy in the Presence of Obstacle
Behavior of Ultrasound Energy in the Presence of Obstacle
 
263 4.pdf
263 4.pdf263 4.pdf
263 4.pdf
 
263 4.pdf
263 4.pdf263 4.pdf
263 4.pdf
 
A weak lensing_mass_reconstruction_of _the_large_scale_filament_massive_galax...
A weak lensing_mass_reconstruction_of _the_large_scale_filament_massive_galax...A weak lensing_mass_reconstruction_of _the_large_scale_filament_massive_galax...
A weak lensing_mass_reconstruction_of _the_large_scale_filament_massive_galax...
 
Compton effect
Compton effectCompton effect
Compton effect
 
Quantum dOTS updated
Quantum dOTS  updated Quantum dOTS  updated
Quantum dOTS updated
 
LEUKOS_International_Journal
LEUKOS_International_JournalLEUKOS_International_Journal
LEUKOS_International_Journal
 
12098
1209812098
12098
 
Bocchino ea08a
Bocchino ea08aBocchino ea08a
Bocchino ea08a
 
Newly born pulsars
Newly born pulsarsNewly born pulsars
Newly born pulsars
 
spectrophotometer 2.pdf
spectrophotometer 2.pdfspectrophotometer 2.pdf
spectrophotometer 2.pdf
 

Recently uploaded

Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphNetziValdelomar1
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17Celine George
 
Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxraviapr7
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...Nguyen Thanh Tu Collection
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptxraviapr7
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxAditiChauhan701637
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17Celine George
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxraviapr7
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationMJDuyan
 
Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxKatherine Villaluna
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.raviapr7
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxDr. Santhosh Kumar. N
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17Celine George
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17Celine George
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfYu Kanazawa / Osaka University
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfMohonDas
 

Recently uploaded (20)

Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a Paragraph
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17
 
Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptx
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptx
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptx
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive Education
 
Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptx
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.
 
Finals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quizFinals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quiz
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptx
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17
 
Prelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quizPrelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quiz
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdf
 

Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense

  • 1. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Signal decompositions using trans-dimensional Bayesian methods Alireza Roodaki Ph.D. Thesis Defense Department of Signal Processing and Electronic Systems 2012, May 14th Advisors: Julien Bect and Gilles Fleury Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
  • 2. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 1: Detection and estimation of muons in the Auger project Ultra high energy particles coming from space (E ∼ 1019 eV) How and where? What is their composition (Proton, Iron)? Figure: A conceptual shower (http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
  • 3. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 1: Detection and estimation of muons in the Auger project Ultra high energy particles coming from space (E ∼ 1019 eV) How and where? What is their composition (Proton, Iron)? Figure: A conceptual shower (http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
  • 4. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 1: Detection and estimation of muons in the Auger project Ultra high energy particles coming from space (E ∼ 1019 eV) How and where? What is their composition (Proton, Iron)? Figure: A conceptual shower (http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
  • 5. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 1: Detection and estimation of muons in the Auger project (Contd.) muons are generated when particles cross the atmosphere the number k of muons and their arrival times t µ are indicators of the origin Figure: A conceptual shower and and composition of particle detectors (water tanks) (http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 2/ 52
  • 6. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 1: Detection and estimation of muons in the Auger project (Contd.) muons are generated when particles cross the atmosphere the number k of muons and their arrival times t µ are indicators of the origin Figure: A conceptual shower and and composition of particle detectors (water tanks) (http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 2/ 52
  • 7. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 1: Detection and estimation of muons in the Auger project (Contd.) muons are generated when particles cross the atmosphere the number k of muons and their arrival times t µ are indicators of the origin Figure: A conceptual shower and and composition of particle detectors (water tanks) (http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 2/ 52
  • 8. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 1: Detection and estimation of muons in the Auger project (Contd.) Figure: Water tank detector. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 3/ 52
  • 9. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 1: Detection and estimation of muons in the Auger project (Contd.) #PE 20 10 0 1.5 intensity 1 0.5 0 100 200 300 400 500 600 t[ns] Figure: Observed signal (n) Prof. Balázs Kégl from LAL, University of Paris 11. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 4/ 52
  • 10. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 1: Detection and estimation of muons in the Auger project (Contd.) #PE 20 10 0 1.5 intensity 1 0.5 0 100 200 300 400 500 600 t[ns] Figure: Observed signal (n) Prof. Balázs Kégl from LAL, University of Paris 11. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 4/ 52
  • 11. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 2: Spectral analysis 10 0 −10 Applications 0 10 20 30 40 50 60 time RADAR / SONAR 150 100 Array signal processing Power 50 Vibration analysis 0 ... 0 0.5 1 1.5 2 2.5 3 radial frequency Figure: Observed signal (top) and its periodogram (bottom). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 5/ 52
  • 12. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 2: Spectral analysis (Cont.) Detection and estimation of sinusoids in white noise model the observed signal y by sinusoidal components Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 6/ 52
  • 13. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 2: Spectral analysis (Cont.) Detection and estimation of sinusoids in white noise model the observed signal y by sinusoidal components observed signal k Mk : y [i] = aj cos[ωj i] + bj sin[ωj i] + n[i]. j=1 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 6/ 52
  • 14. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 2: Spectral analysis (Cont.) Detection and estimation of sinusoids in white noise model the observed signal y by sinusoidal components observed signal k Mk : y [i] = aj cos[ωj i] + bj sin[ωj i] + n[i]. j=1 Joint model selection and parameter estimation problem Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 6/ 52
  • 15. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Trans-dimensional problems Def. The problems in which the number of things that we don′ t know is one of the things that we don′ t know [Green, 2003.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 7/ 52
  • 16. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Trans-dimensional problems Def. The problems in which the number of things that we don′ t know is one of the things that we don′ t know [Green, 2003.] space X = k ∈K {k } × Θk with points x = (k , θk ) « k ∈ K denotes number of components « θk ∈ Θk is a vector of component-specific parameters Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 7/ 52
  • 17. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Trans-dimensional problems Def. The problems in which the number of things that we don′ t know is one of the things that we don′ t know [Green, 2003.] space X = k ∈K {k } × Θk with points x = (k , θk ) « k ∈ K denotes number of components « θk ∈ Θk is a vector of component-specific parameters Applications: « Spectral Analysis (Array signal processing) « (Gaussian) Mixture modeling & Clustering « Object detection and recognition Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 7/ 52
  • 18. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Bayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  • 19. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Bayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Prior distribution Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  • 20. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Bayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Prior distribution Posterior distribution Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  • 21. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Bayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Prior distribution Posterior distribution x = (k , θk ) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  • 22. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Bayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Prior distribution Posterior distribution x = (k , θk ) « both detection and estimation problems Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  • 23. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Bayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Prior distribution Posterior distribution x = (k , θk ) « both detection and estimation problems high-dimensional / intractable integrals Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  • 24. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Markov Chain Monte Carlo (MCMC) methods generate samples from the posterior distribution of interest (target distribution), say, π. construct a Markov chain (x (1) , . . . , x (M) ) that under some conditions converges to π. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 9/ 52
  • 25. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Markov Chain Monte Carlo (MCMC) methods generate samples from the posterior distribution of interest (target distribution), say, π. construct a Markov chain (x (1) , . . . , x (M) ) that under some conditions converges to π. Famous algorithms: « Metropolis-Hastings (MH) sampler [Metropolis, et al. 1953, Hastings, 1970.] « Gibbs sampler [Geman and Geman, 1984.] « RJ-MCMC sampler [Green, 1995.] [Robert and Casella, 2004.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 9/ 52
  • 26. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Markov Chain Monte Carlo (MCMC) methods generate samples from the posterior distribution of interest (target distribution), say, π. construct a Markov chain (x (1) , . . . , x (M) ) that under some conditions converges to π. Famous algorithms: « Metropolis-Hastings (MH) sampler [Metropolis, et al. 1953, Hastings, 1970.] « Gibbs sampler [Geman and Geman, 1984.] « RJ-MCMC sampler [Green, 1995.] [Robert and Casella, 2004.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 9/ 52
  • 27. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Example 2: spectral analysis (Cont.) RJ-MCMC sampler ⇒ variable dimensional samples 0.8 0.6 ωk 0.4 4 3 k 2 160 170 180 190 200 Iteration number Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 10/ 52
  • 28. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization
  • 29. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies
  • 30. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies 3 Results Detection and estimation of sinusoids in white noise Detection and estimation of muons in the Auger project
  • 31. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies 3 Results Detection and estimation of sinusoids in white noise Detection and estimation of muons in the Auger project 4 Conclusion
  • 32. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies 3 Results Detection and estimation of sinusoids in white noise Detection and estimation of muons in the Auger project 4 Conclusion
  • 33. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion summarizing posterior distributions Posterior = all the information Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  • 34. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion summarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  • 35. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion summarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) (RJ)-MCMC sampler Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  • 36. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion summarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) (RJ)-MCMC sampler « What to do with the generated samples? Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  • 37. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion summarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) (RJ)-MCMC sampler « What to do with the generated samples? Summarization human readable summaries Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  • 38. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion summarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) (RJ)-MCMC sampler « What to do with the generated samples? Summarization human readable summaries interpretable figures (e.g., histograms) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  • 39. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion summarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) (RJ)-MCMC sampler « What to do with the generated samples? Summarization human readable summaries interpretable figures (e.g., histograms) statistical measures (e.g., mean and variance) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  • 40. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Fixed-dimensional problems uni-modal uni-variate case 0.6 p 0.4 0.2 0 0 2 4 Samples report location (mean and median) and dispersion (variance and confidence intervals) parameters µ σ2 2.0 0.25 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 13/ 52
  • 41. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Label-switching issue Additive mixture: lack of identifiability the likelihood is invariant under relabeling of components Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
  • 42. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Label-switching issue Additive mixture: lack of identifiability the likelihood is invariant under relabeling of components the posterior distribution is invariant under permutation of components Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
  • 43. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Label-switching issue Additive mixture: lack of identifiability the likelihood is invariant under relabeling of components the posterior distribution is invariant under permutation of components Marginal posteriors of Comp. #1 10 component-specific 0 parameters are identical! Comp. #2 10 0 Comp. #3 10 0 0.5 0.75 1 ω Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
  • 44. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Label-switching issue Additive mixture: lack of identifiability the likelihood is invariant under relabeling of components the posterior distribution is invariant under permutation of components Marginal posteriors of Comp. #1 10 component-specific 0 parameters are identical! Comp. #2 10 How to summarize the 0 posterior information? Comp. #3 10 0 0.5 0.75 1 ω Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
  • 45. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion strategies to deal with label-switching imposing artificial “identifiability constraints” Exp: sorting the components [Richardson and Green, 1997.] Comp. #1 10 0 Comp. #2 10 0 Comp. #3 10 0 0.5 0.75 1 Figure: components are sorted based on ω. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 15/ 52
  • 46. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion strategies to deal with label-switching imposing artificial “identifiability constraints” Exp: sorting the components [Richardson and Green, 1997.] Comp. #1 10 0 Comp. #2 10 0 Comp. #3 10 0 0.5 0.75 1 Figure: components are sorted based on ω. relabeling algorithms [Celeux, et al. 1998, Stephens, 2000, Jasra, et al, 2005, Sperrin et al, 2010, Yao, 2011.]. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 15/ 52
  • 47. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Example 2: spectral analysis (Cont.) Variable-dimensional posterior distribution 4 3 k 2 0 0.3 0.6 0.5 0.75 1 p(k |y) ω Figure: Posteriors of k and sorted radial frequencies, ωk , given k. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 16/ 52
  • 48. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Classical Bayesian approaches Bayesian Model Selection (BMS) One model is selected (estimated) by looking at the MAP, ˆ i.e. k = argmax p(k |y). Component-specific parameters are summarized given ˆ k = k. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 17/ 52
  • 49. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Bayesian Model Selection (BMS) 3 4 3 k 2 0 0.3 0.6 0.5 0.75 1 p(k |y) ω Figure: The model with k = 2 is selected. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 18/ 52
  • 50. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Classical Bayesian approaches Bayesian Model Selection (BMS) One model is selected (estimated) by looking at the MAP, ˆ i.e. k = argmax p(k |y). Component-specific parameters are summarized given ˆ k = k. Bayesian Model Averaging (BMA) Use the information from all possible models: p(∆|y) = k p(∆|k , y)p(k |y) However, ∆ cannot be ωk as its size changes from model to model. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 19/ 52
  • 51. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Bayesian Model Averaging (BMA) Binned data representation (∆ = N(Bj )): kmax E(N(Bj ) | y) = E(N(Bj ) | k , y) · p(k | y) k =1 1 expected nbr comp where .75 j = 1, . . . , Nbin .5 and E(N(Bj )) is the .25 expected number of components in bin 0 0 0.5 1 1.5 2 2.5 3 Bj . ω Figure: Expected number of components using BMA. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 20/ 52
  • 52. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Are BMS and BMA approaches satisfactory? Bayesian Model Selection (BMS) « selects a model ⇒ component-specific parameters « losing information from the discarded models « ignoring the uncertainties about the presence of components. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 21/ 52
  • 53. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion Are BMS and BMA approaches satisfactory? (Cont.) Bayesian Model Averaging (BMA) « appropriate for signal reconstruction and prediction « does not provide information about component-specific parameters Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 22/ 52
  • 54. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion A novel approach is needed! A novel approach is needed! Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 23/ 52
  • 55. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusion A novel approach is needed! A novel approach is needed! Properties of an “ideal” approach information from all (plausible) models « interpretable summaries for component-specific parameters uncertainties about the presence of components Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 23/ 52
  • 56. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies 3 Results Detection and estimation of sinusoids in white noise Detection and estimation of muons in the Auger project 4 Conclusion
  • 57. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Big picture: relabeling and summarizing posterior distributions True posterior f = p(· | y) Approximate posterior qη 2 2 1 1 0 0 0 1 2 3 0 1 2 3 Parametric family {qη , η ∈ N} Measure of “distance” [Stephens, 2000.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 24/ 52
  • 58. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Big picture: relabeling and summarizing posterior distributions True posterior f = p(· | y) Samples Approximate posterior qη 2 2 2 1 1 1 0 0 0 0 1 2 3 0 1 2 3 0 1 2 3 Parametric family {qη , η ∈ N} Measure of “distance” Samples [Stephens, 2000.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 24/ 52
  • 59. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Fixed-dimensional problems uni-modal uni-variate case 0.6 p 0.4 0.2 0 0 2 4 Samples report location (mean and median) and dispersion (variance and confidence intervals) parameters µ σ2 2.0 0.25 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 25/ 52
  • 60. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion variable-dimensional approximate posterior An original (variable-dimensional) parametric model qη Four main requirements: 1 Must be defined on the same space X = k ∈K {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
  • 61. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion variable-dimensional approximate posterior An original (variable-dimensional) parametric model qη Four main requirements: 1 Must be defined on the same space X = k ∈K {k } × Θk 2 Must be permutation invariance Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
  • 62. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion variable-dimensional approximate posterior An original (variable-dimensional) parametric model qη Four main requirements: 1 Must be defined on the same space X = k ∈K {k } × Θk 2 Must be permutation invariance 3 Must be “simple” (small number of parameters) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
  • 63. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion variable-dimensional approximate posterior An original (variable-dimensional) parametric model qη Four main requirements: 1 Must be defined on the same space X = k ∈K {k } × Θk 2 Must be permutation invariance 3 Must be “simple” (small number of parameters) 4 Must be able to capture the main features of the posterior distributions typically met in practice. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
  • 64. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view ··· 1 2 L x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  • 65. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view ··· 1 2 L u1 u2 uL ul ∈ Θ x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  • 66. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view ··· 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  • 67. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view ··· 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} {u l | ξl = 1} x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  • 68. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view ··· 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} {u l | ξl = 1} random arrangement x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  • 69. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view π1 π2 πL ··· 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} {u l | ξl = 1} ξl ∼ B(πl ) random arrangement x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  • 70. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view π1 π2 πL µ1 Σ1 µ2 Σ2 ··· µL ΣL 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} {u l | ξl = 1} ξl ∼ B(πl ) u l ∼ N (µl , Σl ) random arrangement x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  • 71. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view π1 π2 πL µ1 Σ1 µ2 Σ2 ··· µL ΣL 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} {u l | ξl = 1} ξl ∼ B(πl ) u l ∼ N (µl , Σl ) random arrangement ηl = {πl , µl , Σl } x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  • 72. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  • 73. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  • 74. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: 18 L = 3 12 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) s2 = (0.05, 0.02, 0.1) 6 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  • 75. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (0, 1, 1) ⇒ k = 2 & θk = (0.52, 1.05) 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  • 76. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (0, 1, 0) ⇒ k = 1 & θk = 0.49 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  • 77. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (1, 0, 1) ⇒ k = 2 & θk = (0.27, 1.03) 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  • 78. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (0, 0, 1) ⇒ k = 1 & θk = 1.05 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  • 79. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (1, 1, 1) ⇒ k = 3 & θk = (0.27, 0.53, 1.15) 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  • 80. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion A generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (0, 0, 0) ⇒ k = 0 & θk = () 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  • 81. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Big picture: relabeling and summarizing posterior distributions True posterior f = p(· | y) Samples Approximate posterior qη 2 2 2 1 1 1 0 0 0 0 1 2 3 0 1 2 3 0 1 2 3 Parametric family {qη , η ∈ N} Measure of “distance” Samples [Stephens, 2000.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 29/ 52
  • 82. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion fitting the parametric model qη to the posterior f minimizing the Kullback-Leibler divergence f (x) J (η) DKL (f (x) qη (x)) = f (x) log dx qη (x) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 30/ 52
  • 83. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion fitting the parametric model qη to the posterior f minimizing the Kullback-Leibler divergence f (x) J (η) DKL (f (x) qη (x)) = f (x) log dx qη (x) A key point: samples x (i) , i = 1, 2, · · · , M, are generated from f , so M 1 J (η) ≃ − log qη (x (i) ) + Const. M i=1 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 30/ 52
  • 84. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion fitting the parametric model qη to the posterior f minimizing the Kullback-Leibler divergence f (x) J (η) DKL (f (x) qη (x)) = f (x) log dx qη (x) A key point: samples x (i) , i = 1, 2, · · · , M, are generated from f , so M 1 J (η) ≃ − log qη (x (i) ) + Const. M i=1 M Objective: η = argmaxη ˆ i=1 log qη (x (i) ) . Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 30/ 52
  • 85. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Expectation Maximization (EM) latent (hidden) variable: « Binary indicator vector ξ « Random permutation Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
  • 86. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Expectation Maximization (EM) latent (hidden) variable: « Binary indicator vector ξ « Random permutation « Define z = (z1 , . . . , zk ) as an allocation vector for x « zj = l ⇒ x j comes from component l Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
  • 87. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Expectation Maximization (EM) latent (hidden) variable: « Binary indicator vector ξ « Random permutation « Define z = (z1 , . . . , zk ) as an allocation vector for x « zj = l ⇒ x j comes from component l idea: use EM to maximize the likelihood. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
  • 88. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Expectation Maximization (EM) latent (hidden) variable: « Binary indicator vector ξ « Random permutation « Define z = (z1 , . . . , zk ) as an allocation vector for x « zj = l ⇒ x j comes from component l idea: use EM to maximize the likelihood. The E-step is computationally expensive! For example, assuming L = 15 and k (i) = 10, then, it contains 1.1 × 1010 terms. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
  • 89. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Stochastic EM (SEM) at iteration (r + 1) Stochastic (S)-step: for i = 1, . . . , M generate z(i) from p( · | x (i) , η (r ) ) end ˆ M M-step: η (r +1) = argmaxη ˆ i=1 log(p(x (i) , z(i) | η)) [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt, 1993.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
  • 90. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Stochastic EM (SEM) at iteration (r + 1) Stochastic (S)-step: for i = 1, . . . , M generate z(i) from p( · | x (i) , η (r ) ) end ˆ M M-step: η (r +1) = argmaxη ˆ i=1 log(p(x (i) , z(i) | η)) [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt, 1993.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
  • 91. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Stochastic EM (SEM) at iteration (r + 1) Stochastic (S)-step: for i = 1, . . . , M generate z(i) from p( · | x (i) , η (r ) ) end ˆ M M-step: η (r +1) = argmaxη ˆ i=1 log(p(x (i) , z(i) | η)) [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt, 1993.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
  • 92. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Stochastic EM (SEM) at iteration (r + 1) Stochastic (S)-step: for i = 1, . . . , M generate z(i) from p( · | x (i) , η (r ) ) end ˆ M M-step: η (r +1) = argmaxη ˆ i=1 log(p(x (i) , z(i) | η)) [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt, 1993.] S-step To draw z(i) ∼ p( · | x (i) , η (r ) ) we developed an I-MH sampler. ˆ Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
  • 93. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Example 2: spectral analysis (Cont.) Variable-dimensional posterior distribution 4 3 k 2 0 0.3 0.6 0.5 0.75 1 p(k |y) ω Figure: Posteriors of k and sorted radial frequencies, ωk , given k. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 33/ 52
  • 94. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Robustifying solutions Add a Poisson point process component To capture the “outliers” λ is the mean parameter points are uniformly distributed on Θ Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
  • 95. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Robustifying solutions Add a Poisson point process component To capture the “outliers” λ is the mean parameter points are uniformly distributed on Θ Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
  • 96. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Robustifying solutions Add a Poisson point process component To capture the “outliers” λ is the mean parameter points are uniformly distributed on Θ Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
  • 97. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusion Robustifying solutions Add a Poisson point process component To capture the “outliers” λ is the mean parameter points are uniformly distributed on Θ Other possibilities Robust estimates in the M-step. « Median instead of mean « interquartile range instead of variance Using another divergence measure. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
  • 98. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies 3 Results Detection and estimation of sinusoids in white noise Detection and estimation of muons in the Auger project 4 Conclusion
  • 99. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project Conclusion Example 2: Spectral analysis 10 0 −10 0 10 20 30 40 50 60 time 150 100 Power 50 0 0 0.5 1 1.5 2 2.5 3 radial frequency Figure: Observed signal (top) and its periodogram (bottom). k Mk : y [i] = aj cos[ωj i] + bj sin[ωj i] + n[i]. j=1 [Andrieu and Doucet, 1999.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 35/ 52
  • 100. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project Conclusion Example 2: spectral analysis (Cont.) Variable-dimensional posterior distribution 4 3 k 2 0 0.3 0.6 0.5 0.75 1 p(k |y) ω Figure: Posteriors of k and sorted radial frequencies, ωk , given k. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 36/ 52
  • 101. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project Conclusion S-step: randomized allocation procedure 0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72 z1 z2 z1 z2 z3 z4 z1 z2 z3 M-step norm. density 1 0.5 0 0.5 0.75 1 ω ˆ λ = 0.1 Iter = 0 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
  • 102. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project Conclusion S-step: randomized allocation procedure 0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72 3 1 4 2 3 1 1 4 2 M-step norm. density 1 0.5 0 0.5 0.75 1 ω ˆ λ = 0.1 Iter = 0 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
  • 103. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project Conclusion S-step: randomized allocation procedure 0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72 3 1 4 2 3 1 1 4 2 M-step norm. density 1 0.5 0 0.5 0.75 1 ω ˆ λ = 0.167 Iter = 1 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
  • 104. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project Conclusion S-step: randomized allocation procedure 0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72 3 1 4 2 3 1 1 3 2 M-step norm. density 1 0.5 0 0.5 0.75 1 ω ˆ λ = 0.167 Iter = 1 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
  • 105. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project Conclusion S-step: randomized allocation procedure 0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72 3 1 4 2 3 1 1 3 2 M-step norm. density 1 0.5 0 0.5 0.75 1 ω ˆ λ = 0.226 Iter = 2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
  • 106. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project Conclusion S-step: randomized allocation procedure 0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72 3 1 2 4 3 1 1 2 3 M-step norm. density 1 0.5 0 0.5 0.75 1 ω ˆ λ = 0.226 Iter = 3 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52