SlideShare una empresa de Scribd logo
1 de 60
Descargar para leer sin conexión
Introduction
Sources of Variability and Threshold Derivation
  Information-Theoretic Analysis of IF Neuron
                                     Conclusion




Capacity Analysis of Neurons with Descending
         Action Potential Thresholds

                                Prapun Suksompong

                          Electrical and Computer Engineering
                          Cornell University, Ithaca, NY 14853
                                     ps92@cornell.edu


    Final Examination for the Doctoral Degree (“B” Exam)
                        July 24, 2008



                          Prapun Suksompong       Capacity Analysis of Neurons
Introduction
      Sources of Variability and Threshold Derivation
        Information-Theoretic Analysis of IF Neuron
                                           Conclusion


Outline



   Introduction

   Sources of Variability for the ISIs and Derivation of the Threshold

   Information-Theoretic Analysis of IF Neuron

   Conclusion




                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                     Neuron Morphology
   Sources of Variability and Threshold Derivation
                                                     Integrate-and-Fire Neurons
     Information-Theoretic Analysis of IF Neuron
                                                     Goal
                                        Conclusion




Introduction
    Neuron Morphology
    Integrate-and-Fire Neurons
    Goal

Sources of Variability for the ISIs and Derivation of the Threshold

Information-Theoretic Analysis of IF Neuron

Conclusion




                             Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                        Neuron Morphology
      Sources of Variability and Threshold Derivation
                                                        Integrate-and-Fire Neurons
        Information-Theoretic Analysis of IF Neuron
                                                        Goal
                                           Conclusion


Neuron Morphology

   A neuron is the basic working unit of the nervous system.
                                                                     Dendrite
 A typical neuron has three                                                                 Nucleus
                                                                                                    Axon Hillock

 functionally distinct parts, called                                                                Axon



      dendrites,                                                                                                                  Axon
                                                           Axon from                                                            Terminals
      soma, and                                          another neuron                 Cell Body
                                                                                        (Soma)
                                                              Synapse
      axon.                                                                                           Myelin
                                                                                                      Sheath
                                                                                                                   Node of
                                                                                                                   Ranvier

 The junction between two                                                                                      Synaptic Vesicle
                                                                 Presynaptic
 neurons is called a synapse.                                      Axom
                                                                  Terminal
                                                                                                               Synaptic Cleft
                                                                                                               Postsynaptic
                                                                                                                Dendrite
                                                                          Ion Channel




                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                          Neuron Morphology
      Sources of Variability and Threshold Derivation
                                                          Integrate-and-Fire Neurons
        Information-Theoretic Analysis of IF Neuron
                                                          Goal
                                           Conclusion


Action Potentials (Spikes)

   Looking at a synapse, we refer to the sending neuron as the
   presynaptic neuron and to the receiving neuron as the
   postsynaptic neuron.


                              presynaptic               postsynaptic


                                                        synapse
                                              axon




   The neuronal signals consist of short electrical pulses called action
   potentials (APs) or spikes. A chain of APs emitted by a single
   neuron is called a spike train.

                                Prapun Suksompong         Capacity Analysis of Neurons
Introduction
                                                         Neuron Morphology
     Sources of Variability and Threshold Derivation
                                                         Integrate-and-Fire Neurons
       Information-Theoretic Analysis of IF Neuron
                                                         Goal
                                          Conclusion


Quantal Synaptic Failure (QSF)


                             presynaptic               postsynaptic


                                                       synapse
                                             axon




       Synaptic failure: It is possible that an AP fails to get
       “across” the synapse.
               We may model a synapse as a Z -channel.
       Spikes which successfully cross the synapse then propagate
       down to soma.

                               Prapun Suksompong         Capacity Analysis of Neurons
Introduction
                                                       Neuron Morphology
     Sources of Variability and Threshold Derivation
                                                       Integrate-and-Fire Neurons
       Information-Theoretic Analysis of IF Neuron
                                                       Goal
                                          Conclusion


Integrate-and-Fire Neurons




                                                              Assumption: ∼ 104
                                                              pre-synaptic neurons.
                                                              True in cortex (higher brain
                                                              functions).




                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction   which are drawn here are in fact decreasing. We will return to t
                                                       Neuron Morphology
     Sources of Variability and Threshold Derivation
                                                       Integrate-and-Fire Neurons
       Information-Theoretic Analysis of IF Neuron
                                                       Goal
                                          Conclusion


Integrate-and-Fire Neurons



                                                                                 Descending Threshold




                                                                          Ascending
                                                                          Membrane
                                                                          Potential

                                                                                                        time
                                                                        Spike Train
     Spikes generated when the
     membrane potentials hit the
                                                                                                         time
     thresholds.
                                                       •   First jitter: Spike generation
     Descending thresholds.                                • Poisson approximation

                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                        Neuron Morphology
      Sources of Variability and Threshold Derivation
                                                        Integrate-and-Fire Neurons
        Information-Theoretic Analysis of IF Neuron
                                                        Goal
                                           Conclusion


(Leaky) Integrate-and-Fire Model: LIF or IF

   Let τ1 , τ2 , τ3 , . . . be the sequence of time that the spikes arrive at
   the spike generating region. The membrane potential at time t is
   then
                 X (t) =          h (t, τm , Ym ) =  Ym h(t − τm ).
                                 m                            m

   This is the “integrate” part of the integrate-and-fire neuron.

                                                                                  X (t )
      Ym is the weight for the mth
      spike due to propagation
      loss, synaptic strength,
      synaptic failure, etc.                                                                            Yi + 2 h ( t − τ i + 2 )
                                                                                  Yi h ( t − τ i )            Yi +1h ( t − τ i +1 )
      h is the shape function.
                                                                 τi      τ i +1                 τ i+2                      time


                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                           Neuron Morphology
     Sources of Variability and Threshold Derivation
                                                           Integrate-and-Fire Neurons
       Information-Theoretic Analysis of IF Neuron
                                                           Goal
                                          Conclusion


Integrate-and-Fire Neurons (Con’t)

     Constant bombardment of spikes
     leads to increase in membrane
     potential.
     As soon as the membrane potential
     reaches a critical value or
     threshold, the neuron “fires” an                                                             time (t)
     action potential. Then, everything
     resets.
     Refractory period: The time after
     a AP is produced, during which it
     is impossible to generate another                                              
                                                                                          Threshold
     AP.                                                                                       time (t)
           Set T (t) to be ∞ during this
           period.
                               Prapun Suksompong           Capacity Analysis of Neurons
Introduction
                                                       Neuron Morphology
     Sources of Variability and Threshold Derivation
                                                       Integrate-and-Fire Neurons
       Information-Theoretic Analysis of IF Neuron
                                                       Goal
                                          Conclusion


Integrate-and-Fire Neurons (Summary)


     ∼ 104 pre-synaptic neurons.                                                 Descending Threshold




                                                                          Ascending
                                                                          Membrane
                                                                          Potential

                                                                                                        time
                                                                        Spike Train


                                                                                                        time

                                                       •   First jitter: Spike generation
       Descending thresholds.
                                                           • Poisson approximation


                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                       Neuron Morphology
     Sources of Variability and Threshold Derivation
                                                       Integrate-and-Fire Neurons
       Information-Theoretic Analysis of IF Neuron
                                                       Goal
                                          Conclusion


Theoretical Approaches to Neuroscience


       We use IF model, but more biologically-realistic models exist
       (e.g. Hodgkin and Huxley [’52] model).
          1. Too many parameters.
                       Physical measurements “fundamentally disturb cell properties”
          2. Provide less insight.
       Biological structures have evolved via natural selection to
       operate optimally.
               See the book Optima for Animals by R. McNeill Alexander.
                       What is the best strength for a bone?
                       At what speed should humans change from walking to
                       running?




                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                        Neuron Morphology
      Sources of Variability and Threshold Derivation
                                                        Integrate-and-Fire Neurons
        Information-Theoretic Analysis of IF Neuron
                                                        Goal
                                           Conclusion


Information-Theoretic Optimization


   Application of information theory has already found success in
   many areas of neuroscience.
       Barlow’s “economy of impulses”[’59, ’69]
                Minimize redundancy.
        Linsker’s InfoMax principle [’88, ’89]
                Maximize the mutual information.
        Levy and Baxter’s energy-efficient coding [’96, ’02]
                Maximize mutual information per unit energy expended.




                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                        Neuron Morphology
      Sources of Variability and Threshold Derivation
                                                        Integrate-and-Fire Neurons
        Information-Theoretic Analysis of IF Neuron
                                                        Goal
                                           Conclusion


Motivation and Goal


        Integrate-and-Fire (IF) model is very popular.
        The threshold function is a crucial element of the IF model.
        Little amount of work exists on deriving the form of the
        threshold curve.
        In fact, using constant thresholding is also popular.
                This leads to large jitter in the spike timing and hence
                discourages the use of time coding.
   Goal: Find (1) an expression for threshold curve under biologically
   realistic constraints and (2) the optimal operating point of neuron
   under such threshold.


                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                     First Jitter: Spike generation
   Sources of Variability and Threshold Derivation
                                                     Second and Third Jitters
     Information-Theoretic Analysis of IF Neuron
                                                     The Threshold Curve
                                        Conclusion




Introduction

Sources of Variability for the ISIs and Derivation of the Threshold
   First Jitter: Spike generation
   Second and Third Jitters
   The Threshold Curve

Information-Theoretic Analysis of IF Neuron

Conclusion




                             Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                        First Jitter: Spike generation
      Sources of Variability and Threshold Derivation
                                                        Second and Third Jitters
        Information-Theoretic Analysis of IF Neuron
                                                        The Threshold Curve
                                           Conclusion


Three sources of variability for inter-spike intervals

   1. Spike generation
   2. Spike propagation
   3. Time-of-arrival estimation




                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                                 First Jitter: Spike generation
          Sources of Variability and Threshold Derivation
                                                                 Second and Third Jitters
            Information-Theoretic Analysis of IF Neuron
                                                                 The Threshold Curve
                                               Conclusion


First jitter: Spike generation
 formula that governs how the membrane potential of this middle neuron rises given the
 combined incoming rate λ .

                      λ1



                       λ2

                                                    ⊕
                       λ3



                    λ = λ1 + λ2 + λ3 +

                                              Now, of course, there is some jitter in the timing of the
                                              paper today.
             Large number of presynaptic neurons source of jitter comes from the fact that the in
                                              The first allows Poisson
             approximation for the superposed in fact some jitter in large. That assumption allows
                                              has process.
                                              presynaptic neurons are
                                                                          them. Now, you may recall th

             The membrane potential is governed by a filtered Poisson to a Poisson p
                                              .. the superposed spike trains … is close
             process.                         spike train coming out of a single neuron is not a Poiss
                                              a tractable formula that governs how the membrane po
                           Prapun Suksompong  given the combined incoming rate λ .
                                             Capacity Analysis of Neurons
Introduction
                                                           First Jitter: Spike generation
      Sources of Variability and Threshold Derivation
                                                           Second and Third Jitters
        Information-Theoretic Analysis of IF Neuron
                                                           The Threshold Curve
                                           Conclusion


Approximation for the first timing jitter

          For fixed λ, different realizations of the membrane potential
          correspond to different spiking times.


                           Membrane potential X ( t )                          Filtered Poisson
     σX                                                                        approximation for
                                                                               amount of variation
                                                                               in vertical direction
                                                                               [Parzen’62].
                                                                               Linear approximation
                            Threshold T ( t )                                  for amount of
               σ time                                                          variation in horizontal
                                                        Time
                                                                               direction.

                                Prapun Suksompong          Capacity Analysis of Neurons
Introduction
                                                          First Jitter: Spike generation
        Sources of Variability and Threshold Derivation
                                                          Second and Third Jitters
          Information-Theoretic Analysis of IF Neuron
                                                          The Threshold Curve
                                             Conclusion
                                                                                                     Membrane potential X ( t )
                                                                               σX

Approximation for the first timing jitter (con’t)
                                                                                                         Threshold T ( t )
                                                                                        σ time
                                                                                                                                  Time

                                            H (τ )                            T (τ ) c2 H2 (τ )
            σtime (τ ) ≈                                                 The figure here shows different realizations of the membrane potentials for a fixed
                                                                         combined incoming rate λ . Here, we see that the randomness from the Poisson arrivals
                                T (τ ) h (τ ) − T (τ ) H (τ )                    c1 H (τ )
                                                                         causes fluctuation in the time that the membrane potentials hit the threshold. Under som
                                                                         linear approximation, we can relate the jitter in the vertical direction to the one in
                                                                         horizontal direction. This then gives us the formula for the magnitude of the timing jitte
                                                                         as a function of the spike time τ .


                                                                      h : shape function.
                         Membrane potential X ( t )                      Here, h is the shape function which describes how the membrane potential changes in
                                                                                 e.g. exponential
                                                                         response to a single input spike. For the usual leaky integrate-and-fire model, this h star
                                                                         with some amplitude and then decay exponentially.

   σX
                                                                                                                             h (t )



                                                                         For conciseness, we define these two integrations which get used in the formula here.
                                                                                                 t
                                                                      H (t) =                        h (µ)dµ.
                                                                                             0
                          Threshold T ( t )                                                          t
             σ time                                                   H2 (t) =                            h2 (µ)dµ.
                                                   Time                                          0


                                  Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                          First Jitter: Spike generation
        Sources of Variability and Threshold Derivation
                                                          Second and Third Jitters
          Information-Theoretic Analysis of IF Neuron
                                                          The Threshold Curve
                                             Conclusion


Approximation for the first timing jitter (con’t)

                                            H (τ )                           T (τ ) c2 H2 (τ )
            σtime (τ ) ≈
                                T (τ ) h (τ ) − T (τ ) H (τ )                   c1 H (τ )

                                                                      c1 and c2 are constants
                         Membrane potential X ( t )                   which depend on the
   σX                                                                 distribution of the weight
                                                                      (Ym ) for each spike.
                                                              Recall:
                                                              X (t) =            m   Ym h(t − τm ).

                          Threshold T ( t )
             σ time
                                                   Time


                                  Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                             First Jitter: Spike generation
           Sources of Variability and Threshold Derivation
                                                             Second and Third Jitters
             Information-Theoretic Analysis of IF Neuron
                                                             The Threshold Curve
                                                Conclusion


Jitter in rate estimation

             The only information contained in a Poisson process is its rate
             λ.
             Different λ’s ⇒ different spiking times τ ’s.


          Descending Threshold
                                                             Spike times vary inversely with λ.

                       PSP for Large λ
                                                                                               T (τ )
                                                                              λ(τ ) ≈
                                                                                              c1 H (τ )

                            PSP for Small λ                  Error in rate estimation:

                                                                                  1             T (τ ) c2 H2 (τ )
                                                              σλ (τ ) ≈                                           .
                                           Time                               c1 H (τ )            c1 H (τ )
However, this spike time has some jitter, so the $lambda$ estimation also have some
error. We then go on and approximate this error:
                                  Prapun Suksompong         Capacity Analysis of Neurons
The second jitter is the randomness in the length of time a spike takes to propa
                                            Introduction
                                                                First Jitter: Spike generation
         Sources of Variability and Threshold Derivationanother neuron. It is on the order of 10 microseconds. The third jitt
                                            synapse on
                                                                Second and Third Jitters
                                            time-of-arrival estimation error; that is, if this neuron tries to measure the inter
           Information-Theoretic Analysis of IF Neuron
                                                                The Threshold Curve
                                              Conclusion
                                            interval, it needs to find out what time a spike arrives. We borrow some formu
                                            Radar guys shown here because this is exactly the problem that they call the ra
Second and Third Jitters over N0 here. Iterror depends on the signal-to-noise the shape ofis show
                     ranging problem. The
                     this Es              also depends on the bandwidth for
                                                                              ratio which
                                                                                           the acti
                                            potential. The amount of error here is about 10 microseconds as well.


        Propagation Time.
        Time-of-Arrival Estimation
        Error (radar ranging
        problem):
                                        −1
                           2Es 2
                    4π 2      f              .
                           N0

                   f 2 : Gabor-bandwidth.
                ES : Signal energy.
                N0
                 2 : Spectral height of the
                Noise.
        Small: < 10µs.
                                    Prapun Suksompong          Capacity Analysis of Neurons
Introduction
                                                           First Jitter: Spike generation
      Sources of Variability and Threshold Derivation
                                                           Second and Third Jitters
        Information-Theoretic Analysis of IF Neuron
                                                           The Threshold Curve
                                           Conclusion


Deriving The Threshold Curves


   Recall: Deviation in time:

                                         H (τ )                              T (τ ) c2 H2 (τ )
         σtime (τ ) =                                                                          .
                             T (τ ) h (τ ) − T (τ ) H (τ )                      c1 H (τ )

   We consider the thresholds which
   1) preserve timing jitter σtime (τ ) ≡ σtime,0 , or
                                                        σtime (τ )
   2) preserve relative timing jitter                       τ        ≡ σ%time,0
   across spiking times (or spiking frequencies) of interest.



                                Prapun Suksompong          Capacity Analysis of Neurons
Introduction
                                                           First Jitter: Spike generation
      Sources of Variability and Threshold Derivation
                                                           Second and Third Jitters
        Information-Theoretic Analysis of IF Neuron
                                                           The Threshold Curve
                                           Conclusion


Deriving The Threshold Curves

   Recall:
                                                                     H(τ )                  T (τ )c2 H2 (τ )
   (a) Deviation in time: σtime (τ ) =                     T (τ )h(τ )−T (τ )H(τ )             c1 H(τ ) .
                                                                        1          T (τ )c2 H2 (τ )
   (b) Deviation in λ estimation: σλ (τ ) =                          c1 H(τ )         c1 H(τ ) .
   We consider the thresholds which
   1) preserve timing jitter σtime (τ ) ≡ σtime,0 , or
                                                        σtime (τ )
   2) preserve relative timing jitter                       τ        ≡ σ%time,0 , or
   3) preserve jitter in λ estimation σλ (τ ) ≡ σλ,0 , or
                                                                      σλ (τ )
   4) preserve relative jitter in λ estimation                          λ       ≡ σ%λ,0 , or
   5) preserve jitter in ln λ estimation
   across spiking times (or spiking frequencies) of interest.

                                Prapun Suksompong          Capacity Analysis of Neurons
Introduction
                                                       First Jitter: Spike generation
     Sources of Variability and Threshold Derivation
                                                       Second and Third Jitters
       Information-Theoretic Analysis of IF Neuron
                                                       The Threshold Curve
                                          Conclusion


Deriving The Threshold Curves (con’t)

       Constant timing jitter level:

                                       h (t)                              1             c2 H2 (t)
           T (t) = T (t)                           −    T (t)                                        .
                                       H (t)                          σtime,0           c1 H (t)

       Constant relative-timing-jitter level:

                                    h (t)                                 1              c2 H2 (t)
         T (t) = T (t)                           −     T (t)                                             .
                                    H (t)                          tσ%time,0             c1 H (t)

       Preserve relative error in λ estimation:
                                                           H2 (t)
                                            T (t) = c0            .
                                                           H (t)

                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                     First Jitter: Spike generation
   Sources of Variability and Threshold Derivation
                                                     Second and Third Jitters
     Information-Theoretic Analysis of IF Neuron
                                                     The Threshold Curve
                                        Conclusion


The differential equations are Bernoulli equations of the form

                      T (t) = T (t) P (t) −                     T (t)Q (t) .

They can be reduced to linear equation by introducing
v (t) = T (t) which gives

                                  1             1
                           v (t) = P (t) v (t) − Q (t) .
                                  2             2
Linear! The solution is
                                                          t
                                                              1
               v (t) = v (t0 ) φ (t, t0 ) −                     φ (t, τ ) Q (τ )dτ,
                                                              2
                                                     t0

                            t
                                1
                                2
                                  P(τ )dτ
where φ (t, s) = e s                        .
                                Prapun Suksompong    Capacity Analysis of Neurons
Introduction
                                                        First Jitter: Spike generation
     Sources of Variability and Threshold Derivation
                                                        Second and Third Jitters
       Information-Theoretic Analysis of IF Neuron
                                                        The Threshold Curve
                                          Conclusion


Comparison between derived thresholds




                                        Constant
                                        timing jitter


                                              Constant relative
                                              timing jitter

                                                              Exponential


                                                                       Heavy-tail
                    Linear

                                          6                    7                         8
                                                  Time [ms]




                               Prapun Suksompong        Capacity Analysis of Neurons
Introduction
                                                                 First Jitter: Spike generation
    Sources of Variability and Threshold Derivation
                                                                 Second and Third Jitters
      Information-Theoretic Analysis of IF Neuron
                                                                 The Threshold Curve
                                         Conclusion


Summary

      Analyze and quantify three sources of timing jitter




      Predict shape of threshold curves

                                                      Constant
                                                      timing jitter


                                                           Constant relative
                                                           timing jitter

                                                                           Exponential


                                                                                   Heavy-tail
                                            Linear

                                                       6                    7                   8
                                                               Time [ms]




                              Prapun Suksompong                  Capacity Analysis of Neurons
Introduction
                                                     OPT1: Maximization of Mutual Information
   Sources of Variability and Threshold Derivation
                                                     OPT2: Mutual Information per Unit Energy Cost
     Information-Theoretic Analysis of IF Neuron
                                                     Rate Matching
                                        Conclusion




Introduction

Sources of Variability for the ISIs and Derivation of the Threshold

Information-Theoretic Analysis of IF Neuron
    OPT1: Maximization of Mutual Information
    OPT2: Mutual Information per Unit Energy Cost
    Rate Matching

Conclusion




                             Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                          OPT1: Maximization of Mutual Information
      Sources of Variability and Threshold Derivation
                                                          OPT2: Mutual Information per Unit Energy Cost
        Information-Theoretic Analysis of IF Neuron
                                                          Rate Matching
                                           Conclusion


Conditional density Q(t|λ) = fτ |Λ (t|λ)
         

        We have formula(s) for the threshold curve T (t).
        Assumption: λ stays constant during each ISI.
        Given Poisson input intensity λ, can find the conditional
        density Q(t|λ) = fτ |Λ (t|λ).
        τ = g (Λ)+jitter.



                         Λ   λ                                                  τ         

                                                   τ |Λ   |λ


                                Prapun Suksompong         Capacity Analysis of Neurons
Introduction
                                                        OPT1: Maximization of Mutual Information
      Sources of Variability and Threshold Derivation
                                                        OPT2: Mutual Information per Unit Energy Cost
        Information-Theoretic Analysis of IF Neuron
                                                        Rate Matching
                                           Conclusion


Optimization 1: Mutual Information



   OPT1:
                                               sup I (Λ; τ )
   where
                                                         fΛ,τ (Λ, τ )
                                I (Λ; τ ) = E log
                                                        fΛ (Λ)fτ (τ )
   and the supremum is taken over all possible fΛ (λ).
        Blahut-Arimoto Algorithm (BAA)




                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                                                             OPT1: Maximization of Mutual Information
          Sources of Variability and Threshold Derivation
                                                                                             OPT2: Mutual Information per Unit Energy Cost
            Information-Theoretic Analysis of IF Neuron
                                                                                             Rate Matching
                                               Conclusion




                       Exponential                                          “Constant-Jitter”                         “Constant-Relative-Jitter”
                        Threshold                                              Threshold                                     Threshold
                  -3                                                         -3                                                 -3
              x 10                                                      x 10                                                x 10
          6                                                         6                                                   6


          4                                                         4                                                   4




                                                           f ()
 f ()




                                                                                                               f ()
          2                                                         2                                                   2


          0                                                         0                                                   0
              0        500        1000      1500   2000                 0         500   1000    1500   2000                 0        500        1000       1500   2000
                                                                                                                                               

        0.8                                                       0.8                                                 0.8

        0.6                                                       0.6                                                 0.6
                                                          f(t)
f(t)




                                                                                                              f(t)
        0.4                                                       0.4                                                 0.4

        0.2                                                       0.2                                                 0.2

          0                                                         0                                                   0
              4        6      8        10     12   14                   4         5      6       7      8                   4        5      6          7     8     9
                                   t                                                     t                                                       t

                           C = 5.457                                              C = 4.931                                              C = 5.109
                              (a)                                                    (b)                                                    (c)




                                                    Prapun Suksompong                        Capacity Analysis of Neurons
Introduction
                                                                  OPT1: Maximization of Mutual Information
   Sources of Variability and Threshold Derivation
                                                                  OPT2: Mutual Information per Unit Energy Cost
     Information-Theoretic Analysis of IF Neuron
                                                                  Rate Matching
                                        Conclusion




Capacity-achieving input densities look similar.
                                    −3
                                x 10
                            6
                                                                              exponential
                                                                              constant jitter
                            5                                                 constant relative jitter



                            4
                    fΛ(λ)




                            3



                            2



                            1



                            0
                                0      200   400   600   800   1000   1200   1400   1600    1800     2000
                                                                λ




                                       Prapun Suksompong          Capacity Analysis of Neurons
Introduction
                                                        OPT1: Maximization of Mutual Information
      Sources of Variability and Threshold Derivation
                                                        OPT2: Mutual Information per Unit Energy Cost
        Information-Theoretic Analysis of IF Neuron
                                                        Rate Matching
                                           Conclusion


Input-Intensity Density Approximation


      BAA does not provide any
      insight.
 Our simpler formula:

            σ0         (c1 H (t))3
 fΛ (λ) ≈
            d         T (t) c2 H2 (t)
                                              t=g (λ)

 where g (λ) = E [τ |λ].




                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                        OPT1: Maximization of Mutual Information
      Sources of Variability and Threshold Derivation
                                                        OPT2: Mutual Information per Unit Energy Cost
        Information-Theoretic Analysis of IF Neuron
                                                        Rate Matching
                                           Conclusion


Input-Intensity Density Approximation


                                                                        −3

      BAA does not provide any                                  6
                                                                    x 10

                                                                                                                  exponential

      insight.                                                  5
                                                                                                                  constant jitter
                                                                                                                  constant relative jitter
                                                                                                                  approximation

 Our simpler formula:                                           4




                                                        fΛ(λ)
                       (c1 H (t))3
                                                                3
          σ0
 fΛ (λ) ≈
          d           T (t) c2 H2 (t)                           2


                                              t=g (λ)           1



 where g (λ) = E [τ |λ].                                        0
                                                                    0      200   400   600   800   1000   1200   1400   1600    1800     2000
                                                                                                    λ




                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                        OPT1: Maximization of Mutual Information
      Sources of Variability and Threshold Derivation
                                                        OPT2: Mutual Information per Unit Energy Cost
        Information-Theoretic Analysis of IF Neuron
                                                        Rate Matching
                                           Conclusion


Input-Intensity Density Approximation


      BAA does not provide any                                                              exponential
                                                                                            constant jitter
      insight.                                                                              constant relative jitter
                                                                                            approximation

 Our simpler formula:




                                                         fΛ(λ)
                                                                  −3
                                                                 10

          σ0           (c1 H (t))3
 fΛ (λ) ≈
          d           T (t) c2 H2 (t)
                                              t=g (λ)

 where g (λ) = E [τ |λ].                                               1
                                                                      10         10
                                                                                   2

                                                                                       λ
                                                                                                         10
                                                                                                            3




                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                                                                                 OPT1: Maximization of Mutual Information
      Sources of Variability and Threshold Derivation
                                                                                                                 OPT2: Mutual Information per Unit Energy Cost
        Information-Theoretic Analysis of IF Neuron
                                                                                                                 Rate Matching
                                           Conclusion


Approximation Strategy

   Assume that g (Λ) is uniform and then find the corresponding fΛ .
                                         Exponential                                          “Constant-Jitter”                         “Constant-Relative-Jitter”
                                          Threshold                                              Threshold                                     Threshold
                                    -3                                                         -3                                                 -3
                                x 10                                                      x 10                                                x 10
                            6                                                         6                                                   6


                            4                                                         4                                                   4




                                                                             f ()
                   f ()




                                                                                                                                 f ()
                            2                                                         2                                                   2


                            0                                                         0                                                   0
                                0        500        1000      1500   2000                 0         500   1000    1500   2000                 0        500        1000       1500   2000
                                                                                                                                                                 

                          0.8                                                       0.8                                                 0.8

                          0.6                                                       0.6                                                 0.6
                                                                            f(t)
                  f(t)




                                                                                                                                f(t)
                          0.4                                                       0.4                                                 0.4

                          0.2                                                       0.2                                                 0.2

                            0                                                         0                                                   0
                                4        6      8        10     12   14                   4         5      6       7      8                   4        5      6          7     8     9
                                                     t                                                     t                                                       t

                                             C = 5.457                                              C = 4.931                                              C = 5.109
                                                (a)                                                    (b)                                                    (c)




                                         Exponential                                          “Constant-Jitter”                         “Constant-Relative-Jitter”
                                          Threshold                                              Threshold                                     Threshold
                                                           Prapun Suksompong                                     Capacity Analysis of Neurons
Introduction
                                                            OPT1: Maximization of Mutual Information
      Sources of Variability and Threshold Derivation
                                                            OPT2: Mutual Information per Unit Energy Cost
        Information-Theoretic Analysis of IF Neuron
                                                            Rate Matching
                                           Conclusion


Approximation Strategy
   Assume that g (Λ) is uniform and then find the corresponding fΛ .

                                                          ������������   
                            ������           ������


                                       ������������                                      ������

                                                            *
                                                        Convolution


   For invertible function g , the pdf of Z = g (Λ) is given by
                              d −1                      1
              fZ (z) =           g (z) fΛ g −1 (z) =         fΛ (λ) ,
                              dz                     |g (λ)|
   where z = g (λ).
                                  Prapun Suksompong         Capacity Analysis of Neurons
Introduction
                                                       OPT1: Maximization of Mutual Information
     Sources of Variability and Threshold Derivation
                                                       OPT2: Mutual Information per Unit Energy Cost
       Information-Theoretic Analysis of IF Neuron
                                                       Rate Matching
                                          Conclusion


OPT2: Energy-Efficient Neuron

       Brains consume 20% of energy consumption for adults and
       60% for infant [Laughlin and Sejnowski’03].
       Suppose neuron spends
               1 unit of energy per ms when it is idle, and
               e unit of energy per ms when AP is produced.
       e >> 1.
       If the time to the next spike is τ = t, the energy expended is

             bo (t) = 1 × (t − ∆) + e × ∆ = t + (e − 1)∆ = t + r .

       where ∆ is the time used to produce a spike.
               The value of r depends on the type of neurons under
               consideration.

                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                        OPT1: Maximization of Mutual Information
      Sources of Variability and Threshold Derivation
                                                        OPT2: Mutual Information per Unit Energy Cost
        Information-Theoretic Analysis of IF Neuron
                                                        Rate Matching
                                           Conclusion


Optimization 2: I/E

   OPT2:
                                                    I (Λ; τ )
                                             sup
                                                   E [bo (τ )]
   where the supremum is taken over all possible fΛ (λ).
                                                                                I (Λ;τ )
        Jimbo-Kunisawa algorithm (JKA) maximizes                                E[b(Λ)] .
                b is a function of input.
        Our bo is a function of output.
        We define b(λ) = E [bo (τ )|Λ = λ] and apply JKA.
                Because bo (τ ) = τ + r , we have

                                        b(λ) = E [τ |λ] + r = g (λ) + r .



                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                        OPT1: Maximization of Mutual Information
     Sources of Variability and Threshold Derivation
                                                        OPT2: Mutual Information per Unit Energy Cost
       Information-Theoretic Analysis of IF Neuron
                                                        Rate Matching
                                          Conclusion


Input-Intensity Density Approximation



       Can use the same technique as in OPT1 to do approximation
       of input-intensity density.
       In stead of uniform density, consider bounded exponential
       density of the form
                                                        γ
                       f (t; γ, α, β) =                         e −γt 1[α,β] (t) .
                                                e −γα   − e −γβ




                               Prapun Suksompong        Capacity Analysis of Neurons
Introduction
                                                       OPT1: Maximization of Mutual Information
     Sources of Variability and Threshold Derivation
                                                       OPT2: Mutual Information per Unit Energy Cost
       Information-Theoretic Analysis of IF Neuron
                                                       Rate Matching
                                          Conclusion


Input-Intensity Density Approximation


       Can use the same technique as in OPT1 to do approximation
       of input-intensity density.
       Result:

                                                                 (c1 H (t))3
              fΛ (λ) ≈ σ0 f (t; γ, g (b), g (a))                                                ,
                                                                T (t) c2 H2 (t)
                                                                                      t=g (λ)

       where f (t; γ, g (b), g (a)) is the bounded exponential pdf with
       support on the interval [g (b), g (a)] and parameter γ.



                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                                                                     OPT1: Maximization of Mutual Information
Sources of Variability and Threshold Derivation
                                                                                                     OPT2: Mutual Information per Unit Energy Cost
  Information-Theoretic Analysis of IF Neuron
                                                                                                     Rate Matching
                                     Conclusion


                                          −3
                                  x 10
                              6
                                                               exponential                                                                  exponential
                                                               constant jitter                                                              constant jitter
                              5                                constant relative jitter               0.5                                   constant relative jitter
                                                               approximation

                              4                                                                       0.4




                     fΛ(λ)




                                                                                              fτ(t)
                              3                                                                       0.3


                              2                                                                       0.2


                              1                                                                       0.1


                              0                                                                            0
                                  0             500    1000             1500          2000                     4       6            8            10       12           14
                                                        λ                                                                               t [ms]




                                                               exponential
                                                               constant jitter
                                                               constant relative jitter
                                                               approximation
                 fΛ(λ)




                             −3
                                                                                             fτ(t)


                         10



                                                                                                       −1
                                                                                                      10

                                                                                                                           exponential
                                                                                                                           constant jitter
                                                                                                                           constant relative jitter

                                      1                2                          3                            4   5            6          7          8     9          10
                              10                      10                       10
                                                           λ                                                                            t [ms]




                                               Prapun Suksompong                                     Capacity Analysis of Neurons
Introduction
                                                          OPT1: Maximization of Mutual Information
      Sources of Variability and Threshold Derivation
                                                          OPT2: Mutual Information per Unit Energy Cost
        Information-Theoretic Analysis of IF Neuron
                                                          Rate Matching
                                           Conclusion


Free Parameters - Revisited

   Recall, for example, the differential equation that define our
   “constant-relative-jitter” threshold:

                                 h (t)                                  1            c2 H2 (t)
      T (t) = T (t)                           −         T (t)                                      .
                                 H (t)                            tσ%time,0          c1 H (t)

   There are a couple of parameters which we want to revisit.
       The constants c1 and c2 .
                Embedded in them is the effect of QSF.
                           σtime,0
        σ%time,0 =           t0 .     What value should we set σtime,0 to be?
                > 10µs.



                                Prapun Suksompong         Capacity Analysis of Neurons
Introduction
                                                  OPT1: Maximization of Mutual Information
Sources of Variability and Threshold Derivation
                                                  OPT2: Mutual Information per Unit Energy Cost
  Information-Theoretic Analysis of IF Neuron
                                                  Rate Matching
                                     Conclusion




  By scaling the unit of the voltage, we can make c1 = 1.
  The scaling makes
                                            1           1
                               c2 ∝               =
                                         psuccess   1 − pfailure
  where pfailure is the QSF probability.
          pfailure depends on the type of neurons under consideration.
  Let σtime,0 = σ1 and play with it.




                          Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                        OPT1: Maximization of Mutual Information
      Sources of Variability and Threshold Derivation
                                                        OPT2: Mutual Information per Unit Energy Cost
        Information-Theoretic Analysis of IF Neuron
                                                        Rate Matching
                                           Conclusion


Average Rates



   From our optimization (either OPT1 or OPT2), we get the optimal
   input-intensity density fΛ (λ) and spiking-time density fτ (t) for each
   value of σ1 .
                                                    ¯
        Each value of σ1 gives average arrival rate λin and the average
                     ¯ out .
        spiking rate λ
        ¯
        λout = 1 .  E[τ ]
        ¯
        λin = E [Λ]?




                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                       OPT1: Maximization of Mutual Information
     Sources of Variability and Threshold Derivation
                                                       OPT2: Mutual Information per Unit Energy Cost
       Information-Theoretic Analysis of IF Neuron
                                                       Rate Matching
                                          Conclusion


Average Afferent Rate
       High value of Λ corresponds to low value of τ .
       Neuron experiences large Λ value for short amount of time.
       ¯
       λin should be lower than E [Λ].
       Let (Λi , τi ) be the pair of input-intensity and the length of the
       ISI associated with the ith spike.
       The number of input spikes during the ith ISI is a Poisson
       random variable Ni with mean Λi τi .
       The long-term average input rate is then
                                 k                1    k
                                 i=1 Ni           k    i=1 Ni         E [Λ1 τ1 ]
                                  k
                                             =    1     k
                                                                 →               .
                                  i=1 τi                i=1 τi
                                                                       E [τ1 ]
                                                  k

       ¯          E[Λτ ]
       λin =      E[τ ] .

                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                       OPT1: Maximization of Mutual Information
     Sources of Variability and Threshold Derivation
                                                       OPT2: Mutual Information per Unit Energy Cost
       Information-Theoretic Analysis of IF Neuron
                                                       Rate Matching
                                          Conclusion


Rate Matching


       On average, our neuron in consideration is bombarded with an
                               ¯
       average input-intensity λin .
       Suppose our neuron is receiving input from n other neurons.
       Then, on average, each of the sending neurons fire at a rate
          1¯
       of n λin .
       Our neuron should also generate spikes at this rate.
       Therefore, we must have
                                                 1¯      ¯
                                                   λin = λout .
                                                 n



                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                                                    OPT1: Maximization of Mutual Information
     Sources of Variability and Threshold Derivation
                                                                                    OPT2: Mutual Information per Unit Energy Cost
       Information-Theoretic Analysis of IF Neuron
                                                                                    Rate Matching
                                          Conclusion


Rate Matching (Con’t)

                 Average spiking rate [spikes/s]
                                                   200
                                                                                                             input
                                                   150                                                       output

                                                   100

                                                   50

                                                    0
                                                     10   20   30    40        50      60     70    80     90      100
                                                                              σ1 [microsec]



                                                   5.4
                        Capacity [bits]




                                                   5.2

                                                    5

                                                   4.8

                                                     10   20   30    40        50      60     70    80     90      100
                                                                              σ1 [microsec]



                                                          Prapun Suksompong         Capacity Analysis of Neurons
Introduction
                                                       OPT1: Maximization of Mutual Information
     Sources of Variability and Threshold Derivation
                                                       OPT2: Mutual Information per Unit Energy Cost
       Information-Theoretic Analysis of IF Neuron
                                                       Rate Matching
                                          Conclusion


Rate Matching (Con’t)


     Capacity value increases as
     we increase the noise level




                                                        Average spiking rate [spikes/s]
                                                                                          200

     σ1 .                                                                                 150
                                                                                                                                             input
                                                                                                                                             output

                                                                                          100
     Larger value of σ1 implies                                                           50

     threshold decays slower This                                                          0
                                                                                            10   15   20   25   30     35     40   45   50   55       60
                                                                                                                  σ1 [microsec]
     gives larger support for the
     spiking time.                                                                        5.8



                                                               Capacity [bits]
                                                                                          5.6
     Noise is small. Effect of                                                             5.4

     increasing the support of the                                                        5.2
                                                                                            10   15   20   25   30     35     40   45   50   55       60
     output is stronger than the                                                                                  σ1 [microsec]


     effect of increased noise.


                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                                                      OPT1: Maximization of Mutual Information
Sources of Variability and Threshold Derivation
                                                                                      OPT2: Mutual Information per Unit Energy Cost
  Information-Theoretic Analysis of IF Neuron
                                                                                      Rate Matching
                                     Conclusion




                                               60

                                               55
                                                                                            (c) max I/E, r = 100 
                                               50
            Optimal rate [spikes per second]




                                               45

                                               40
                                                                                            (b) max I/E, r = 400 
                                               35

                                               30

                                               25         exponential
                                                          constant relative jitter          (a) max I 
                                               20

                                               15
                                                    1   1.5       2        2.5        3     3.5      4       4.5     5
                                                                                     c2
                                                                                                                              




                                                        Prapun Suksompong             Capacity Analysis of Neurons
Introduction
                                                       OPT1: Maximization of Mutual Information
     Sources of Variability and Threshold Derivation
                                                       OPT2: Mutual Information per Unit Energy Cost
       Information-Theoretic Analysis of IF Neuron
                                                       Rate Matching
                                          Conclusion


Rate Matching (Con’t)


                                                                                           60

                                                                                           55
                                                                                                                                      (c) max I/E, r = 100 
     Firing rate decreases as the                                                          50




                                                        Optimal rate [spikes per second]
     failure probability increases.                                                        45


     Smaller value of r                                                                    40
                                                                                                                                      (b) max I/E, r = 400 
                                                                                           35
     corresponds to higher rate.
                                                                                           30

     As r → ∞, the rate                                                                    25         exponential
                                                                                                      constant relative jitter        (a) max I 
     converges to the max-I case.                                                          20

                                                                                           15
                                                                                                1   1.5       2        2.5        3   3.5    4     4.5    5
                                                                                                                                 c2
                                                                                                                                                               




                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
                                                          OPT1: Maximization of Mutual Information
     Sources of Variability and Threshold Derivation
                                                          OPT2: Mutual Information per Unit Energy Cost
       Information-Theoretic Analysis of IF Neuron
                                                          Rate Matching
                                          Conclusion


Optimal Threshold: c2 = 5 and r = 400
                800


                700                                         exponential
                                                            constant relative timing jitter

                600


                500


                400


                300


                200


                100


                  0
                      0   20     40     60     80      100    120   140      160     180      200
                                                    Time [ms]



                               Prapun Suksompong          Capacity Analysis of Neurons
Introduction
   Sources of Variability and Threshold Derivation
     Information-Theoretic Analysis of IF Neuron
                                        Conclusion




Introduction

Sources of Variability for the ISIs and Derivation of the Threshold

Information-Theoretic Analysis of IF Neuron

Conclusion




                             Prapun Suksompong       Capacity Analysis of Neurons
Introduction
     Sources of Variability and Threshold Derivation
       Information-Theoretic Analysis of IF Neuron
                                          Conclusion


Contributions




    1. Quantify the amount of timing jitter in neuron.
    2. Construct threshold functions.
    3. Provide optimal operating points for neurons which are close
       to experimentally observed values.
               Formulas to approximate the optimal input-intensity densities.




                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
     Sources of Variability and Threshold Derivation
       Information-Theoretic Analysis of IF Neuron
                                          Conclusion


References I


      P. Suksompong and T. Berger.
      Jitter Analysis of Timing Codes for Neurons with Descending
      Action Potential Thresholds.
      ISIT, 2006.
      P. Suksompong and T. Berger.
      Capacity Analysis of Neurons with Descending Action
      Potential Thresholds.
      In preparation for special issue of IEEE Tran. on Info. Theory,
      2009.



                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
      Sources of Variability and Threshold Derivation
        Information-Theoretic Analysis of IF Neuron
                                           Conclusion


References II

       P. Suksompong and T. Berger.
       Energy-Efficient Neurons with Descending Action Potential
       Thresholds.
       In preparation for Journal of Comp. Neuroscience, 2009.
       T. Berger and W.B. Levy.
       Encoding of excitation via dynamic thresholding.
       Society for Neuroscience, 2004.
       W.B. Levy and R.A. Baxter.
       Energy-Efficient Neuronal Computation via Quantal Synaptic
       Failures.
       Journal of Neuroscience, 2002.

                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
      Sources of Variability and Threshold Derivation
        Information-Theoretic Analysis of IF Neuron
                                           Conclusion


References III

       W.B. Levy and R.A. Baxter.
       Energy Efficient Neural Codes
       Neural Computation, 1996.
       Patrick Crotty and William Levy.
       Biophysical limits on axonal transmission rates in axons.
       CNS, 2005.
       E. Parzen.
       Stochastic Processes.
       Holden Day, 1962.
       H. Vincent Poor.
       An introduction to signal detection and estimation (2nd ed.).
       Springer-Verlag New York, Inc., New York, NY, USA, 1994.
                                Prapun Suksompong       Capacity Analysis of Neurons
Introduction
     Sources of Variability and Threshold Derivation
       Information-Theoretic Analysis of IF Neuron
                                          Conclusion


References IV




      Dale Purve et al.
      Neuroscience (3rd ed.).
      Sinauer Associates Inc., Sunderland, MA USA, 1997.




                               Prapun Suksompong       Capacity Analysis of Neurons
Introduction
Sources of Variability and Threshold Derivation
  Information-Theoretic Analysis of IF Neuron
                                     Conclusion




                                      THE
                                      END



                          Prapun Suksompong       Capacity Analysis of Neurons

Más contenido relacionado

Destacado

Total ITSM Presentation
Total ITSM PresentationTotal ITSM Presentation
Total ITSM PresentationRaj Shekhar
 
BIG DATA: Apache Hadoop
BIG DATA: Apache HadoopBIG DATA: Apache Hadoop
BIG DATA: Apache HadoopOleksiy Krotov
 
OFDMA System
OFDMA SystemOFDMA System
OFDMA Systemprapun
 
Neuroengineering Tutorial: Integrate and Fire neuron modeling
Neuroengineering Tutorial: Integrate and Fire neuron modelingNeuroengineering Tutorial: Integrate and Fire neuron modeling
Neuroengineering Tutorial: Integrate and Fire neuron modelingZubin Bhuyan
 
Big Data Business Wins: Real-time Inventory Tracking with Hadoop
Big Data Business Wins: Real-time Inventory Tracking with HadoopBig Data Business Wins: Real-time Inventory Tracking with Hadoop
Big Data Business Wins: Real-time Inventory Tracking with HadoopDataWorks Summit
 
Connected Retail Reference Architecture
Connected Retail Reference ArchitectureConnected Retail Reference Architecture
Connected Retail Reference ArchitectureWSO2
 
Retail Reference Architecture
Retail Reference ArchitectureRetail Reference Architecture
Retail Reference ArchitectureMongoDB
 
The Nervous System (Slide Show)
The Nervous System (Slide Show)The Nervous System (Slide Show)
The Nervous System (Slide Show)William Banaag
 
Retail Industry Enterprise Architecture Review
Retail Industry Enterprise Architecture ReviewRetail Industry Enterprise Architecture Review
Retail Industry Enterprise Architecture ReviewLakshmana Kattula
 

Destacado (11)

Total ITSM Presentation
Total ITSM PresentationTotal ITSM Presentation
Total ITSM Presentation
 
BIG DATA: Apache Hadoop
BIG DATA: Apache HadoopBIG DATA: Apache Hadoop
BIG DATA: Apache Hadoop
 
OFDMA System
OFDMA SystemOFDMA System
OFDMA System
 
NEUROINFORMATICS
NEUROINFORMATICSNEUROINFORMATICS
NEUROINFORMATICS
 
Neuroengineering Tutorial: Integrate and Fire neuron modeling
Neuroengineering Tutorial: Integrate and Fire neuron modelingNeuroengineering Tutorial: Integrate and Fire neuron modeling
Neuroengineering Tutorial: Integrate and Fire neuron modeling
 
Big Data Business Wins: Real-time Inventory Tracking with Hadoop
Big Data Business Wins: Real-time Inventory Tracking with HadoopBig Data Business Wins: Real-time Inventory Tracking with Hadoop
Big Data Business Wins: Real-time Inventory Tracking with Hadoop
 
Connected Retail Reference Architecture
Connected Retail Reference ArchitectureConnected Retail Reference Architecture
Connected Retail Reference Architecture
 
Retail Reference Architecture
Retail Reference ArchitectureRetail Reference Architecture
Retail Reference Architecture
 
The Nervous System (Slide Show)
The Nervous System (Slide Show)The Nervous System (Slide Show)
The Nervous System (Slide Show)
 
Oracle Retail
Oracle RetailOracle Retail
Oracle Retail
 
Retail Industry Enterprise Architecture Review
Retail Industry Enterprise Architecture ReviewRetail Industry Enterprise Architecture Review
Retail Industry Enterprise Architecture Review
 

Similar a Prapun B-Exam

Introduction to cns
Introduction to cnsIntroduction to cns
Introduction to cnsUrooj Arif
 
What makes you nervous
What makes you nervousWhat makes you nervous
What makes you nervouslr1083bnhs
 
What makes you nervous
What makes you nervousWhat makes you nervous
What makes you nervouspaulaballa
 
What makes you nervous
What makes you nervousWhat makes you nervous
What makes you nervousks0477bn
 
Ann by rutul mehta
Ann by rutul mehtaAnn by rutul mehta
Ann by rutul mehtaRutul Mehta
 
Motor synapse and mechanism of excitation
Motor synapse and mechanism of excitationMotor synapse and mechanism of excitation
Motor synapse and mechanism of excitationBharaniKrishna4
 
02 central-nervous-systemppt692 (4)
02 central-nervous-systemppt692 (4)02 central-nervous-systemppt692 (4)
02 central-nervous-systemppt692 (4)gloryprasanna
 
02 central-nervous-systemppt692
02 central-nervous-systemppt69202 central-nervous-systemppt692
02 central-nervous-systemppt692gloryprasanna
 
What makes me nervous artifact
What makes me nervous artifactWhat makes me nervous artifact
What makes me nervous artifactab1286bn
 
Nervous system and sense organs
Nervous system and sense organsNervous system and sense organs
Nervous system and sense organsDinDin Horneja
 
Local and Global Gating of Synaptic Plasticity
Local and Global Gating of Synaptic PlasticityLocal and Global Gating of Synaptic Plasticity
Local and Global Gating of Synaptic Plasticitycijat
 
Ns3 Review Of The Organization Of The Nervous System
Ns3 Review Of The Organization Of The Nervous SystemNs3 Review Of The Organization Of The Nervous System
Ns3 Review Of The Organization Of The Nervous Systemmedical
 
3. What is an ANN Describe various types of ANN. Which ANN do you p.pdf
3. What is an ANN Describe various types of ANN. Which ANN do you p.pdf3. What is an ANN Describe various types of ANN. Which ANN do you p.pdf
3. What is an ANN Describe various types of ANN. Which ANN do you p.pdfivylinvaydak64229
 
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdfCCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdfAsst.prof M.Gokilavani
 

Similar a Prapun B-Exam (20)

Cranial nerves
Cranial nervesCranial nerves
Cranial nerves
 
Introduction to cns
Introduction to cnsIntroduction to cns
Introduction to cns
 
What makes you nervous
What makes you nervousWhat makes you nervous
What makes you nervous
 
What makes you nervous
What makes you nervousWhat makes you nervous
What makes you nervous
 
What makes you nervous
What makes you nervousWhat makes you nervous
What makes you nervous
 
Ann by rutul mehta
Ann by rutul mehtaAnn by rutul mehta
Ann by rutul mehta
 
Motor synapse and mechanism of excitation
Motor synapse and mechanism of excitationMotor synapse and mechanism of excitation
Motor synapse and mechanism of excitation
 
02 central-nervous-systemppt692 (4)
02 central-nervous-systemppt692 (4)02 central-nervous-systemppt692 (4)
02 central-nervous-systemppt692 (4)
 
02 central-nervous-systemppt692
02 central-nervous-systemppt69202 central-nervous-systemppt692
02 central-nervous-systemppt692
 
What makes me nervous artifact
What makes me nervous artifactWhat makes me nervous artifact
What makes me nervous artifact
 
Synapse
SynapseSynapse
Synapse
 
Nervous System
Nervous SystemNervous System
Nervous System
 
Nervous system and sense organs
Nervous system and sense organsNervous system and sense organs
Nervous system and sense organs
 
Ann first lecture
Ann first lectureAnn first lecture
Ann first lecture
 
Local and Global Gating of Synaptic Plasticity
Local and Global Gating of Synaptic PlasticityLocal and Global Gating of Synaptic Plasticity
Local and Global Gating of Synaptic Plasticity
 
Image recognition
Image recognitionImage recognition
Image recognition
 
Ns3 Review Of The Organization Of The Nervous System
Ns3 Review Of The Organization Of The Nervous SystemNs3 Review Of The Organization Of The Nervous System
Ns3 Review Of The Organization Of The Nervous System
 
Neurons and Nervous Systems
Neurons and Nervous SystemsNeurons and Nervous Systems
Neurons and Nervous Systems
 
3. What is an ANN Describe various types of ANN. Which ANN do you p.pdf
3. What is an ANN Describe various types of ANN. Which ANN do you p.pdf3. What is an ANN Describe various types of ANN. Which ANN do you p.pdf
3. What is an ANN Describe various types of ANN. Which ANN do you p.pdf
 
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdfCCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
 

Último

Computed Fields and api Depends in the Odoo 17
Computed Fields and api Depends in the Odoo 17Computed Fields and api Depends in the Odoo 17
Computed Fields and api Depends in the Odoo 17Celine George
 
Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Celine George
 
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdf
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdfAMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdf
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdfphamnguyenenglishnb
 
ACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdfACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdfSpandanaRallapalli
 
Roles & Responsibilities in Pharmacovigilance
Roles & Responsibilities in PharmacovigilanceRoles & Responsibilities in Pharmacovigilance
Roles & Responsibilities in PharmacovigilanceSamikshaHamane
 
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTSGRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTSJoshuaGantuangco2
 
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...Nguyen Thanh Tu Collection
 
Barangay Council for the Protection of Children (BCPC) Orientation.pptx
Barangay Council for the Protection of Children (BCPC) Orientation.pptxBarangay Council for the Protection of Children (BCPC) Orientation.pptx
Barangay Council for the Protection of Children (BCPC) Orientation.pptxCarlos105
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxHumphrey A Beña
 
DATA STRUCTURE AND ALGORITHM for beginners
DATA STRUCTURE AND ALGORITHM for beginnersDATA STRUCTURE AND ALGORITHM for beginners
DATA STRUCTURE AND ALGORITHM for beginnersSabitha Banu
 
Judging the Relevance and worth of ideas part 2.pptx
Judging the Relevance  and worth of ideas part 2.pptxJudging the Relevance  and worth of ideas part 2.pptx
Judging the Relevance and worth of ideas part 2.pptxSherlyMaeNeri
 
Earth Day Presentation wow hello nice great
Earth Day Presentation wow hello nice greatEarth Day Presentation wow hello nice great
Earth Day Presentation wow hello nice greatYousafMalik24
 
Science 7 Quarter 4 Module 2: Natural Resources.pptx
Science 7 Quarter 4 Module 2: Natural Resources.pptxScience 7 Quarter 4 Module 2: Natural Resources.pptx
Science 7 Quarter 4 Module 2: Natural Resources.pptxMaryGraceBautista27
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)lakshayb543
 
What is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERPWhat is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERPCeline George
 
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdf
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdfInclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdf
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdfTechSoup
 
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITY
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITYISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITY
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITYKayeClaireEstoconing
 
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPTECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPTiammrhaywood
 

Último (20)

TataKelola dan KamSiber Kecerdasan Buatan v022.pdf
TataKelola dan KamSiber Kecerdasan Buatan v022.pdfTataKelola dan KamSiber Kecerdasan Buatan v022.pdf
TataKelola dan KamSiber Kecerdasan Buatan v022.pdf
 
Computed Fields and api Depends in the Odoo 17
Computed Fields and api Depends in the Odoo 17Computed Fields and api Depends in the Odoo 17
Computed Fields and api Depends in the Odoo 17
 
Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17
 
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdf
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdfAMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdf
AMERICAN LANGUAGE HUB_Level2_Student'sBook_Answerkey.pdf
 
ACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdfACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdf
 
Roles & Responsibilities in Pharmacovigilance
Roles & Responsibilities in PharmacovigilanceRoles & Responsibilities in Pharmacovigilance
Roles & Responsibilities in Pharmacovigilance
 
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTSGRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
 
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
 
Barangay Council for the Protection of Children (BCPC) Orientation.pptx
Barangay Council for the Protection of Children (BCPC) Orientation.pptxBarangay Council for the Protection of Children (BCPC) Orientation.pptx
Barangay Council for the Protection of Children (BCPC) Orientation.pptx
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
 
DATA STRUCTURE AND ALGORITHM for beginners
DATA STRUCTURE AND ALGORITHM for beginnersDATA STRUCTURE AND ALGORITHM for beginners
DATA STRUCTURE AND ALGORITHM for beginners
 
Judging the Relevance and worth of ideas part 2.pptx
Judging the Relevance  and worth of ideas part 2.pptxJudging the Relevance  and worth of ideas part 2.pptx
Judging the Relevance and worth of ideas part 2.pptx
 
Earth Day Presentation wow hello nice great
Earth Day Presentation wow hello nice greatEarth Day Presentation wow hello nice great
Earth Day Presentation wow hello nice great
 
Science 7 Quarter 4 Module 2: Natural Resources.pptx
Science 7 Quarter 4 Module 2: Natural Resources.pptxScience 7 Quarter 4 Module 2: Natural Resources.pptx
Science 7 Quarter 4 Module 2: Natural Resources.pptx
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
 
What is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERPWhat is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERP
 
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdf
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdfInclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdf
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdf
 
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITY
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITYISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITY
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITY
 
Raw materials used in Herbal Cosmetics.pptx
Raw materials used in Herbal Cosmetics.pptxRaw materials used in Herbal Cosmetics.pptx
Raw materials used in Herbal Cosmetics.pptx
 
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPTECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
 

Prapun B-Exam

  • 1. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion Capacity Analysis of Neurons with Descending Action Potential Thresholds Prapun Suksompong Electrical and Computer Engineering Cornell University, Ithaca, NY 14853 ps92@cornell.edu Final Examination for the Doctoral Degree (“B” Exam) July 24, 2008 Prapun Suksompong Capacity Analysis of Neurons
  • 2. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion Outline Introduction Sources of Variability for the ISIs and Derivation of the Threshold Information-Theoretic Analysis of IF Neuron Conclusion Prapun Suksompong Capacity Analysis of Neurons
  • 3. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Introduction Neuron Morphology Integrate-and-Fire Neurons Goal Sources of Variability for the ISIs and Derivation of the Threshold Information-Theoretic Analysis of IF Neuron Conclusion Prapun Suksompong Capacity Analysis of Neurons
  • 4. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Neuron Morphology A neuron is the basic working unit of the nervous system. Dendrite A typical neuron has three Nucleus Axon Hillock functionally distinct parts, called Axon dendrites, Axon Axon from Terminals soma, and another neuron Cell Body (Soma) Synapse axon. Myelin Sheath Node of Ranvier The junction between two Synaptic Vesicle Presynaptic neurons is called a synapse. Axom Terminal Synaptic Cleft Postsynaptic Dendrite Ion Channel Prapun Suksompong Capacity Analysis of Neurons
  • 5. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Action Potentials (Spikes) Looking at a synapse, we refer to the sending neuron as the presynaptic neuron and to the receiving neuron as the postsynaptic neuron. presynaptic postsynaptic synapse axon The neuronal signals consist of short electrical pulses called action potentials (APs) or spikes. A chain of APs emitted by a single neuron is called a spike train. Prapun Suksompong Capacity Analysis of Neurons
  • 6. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Quantal Synaptic Failure (QSF) presynaptic postsynaptic synapse axon Synaptic failure: It is possible that an AP fails to get “across” the synapse. We may model a synapse as a Z -channel. Spikes which successfully cross the synapse then propagate down to soma. Prapun Suksompong Capacity Analysis of Neurons
  • 7. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Integrate-and-Fire Neurons Assumption: ∼ 104 pre-synaptic neurons. True in cortex (higher brain functions). Prapun Suksompong Capacity Analysis of Neurons
  • 8. Introduction which are drawn here are in fact decreasing. We will return to t Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Integrate-and-Fire Neurons Descending Threshold Ascending Membrane Potential time Spike Train Spikes generated when the membrane potentials hit the time thresholds. • First jitter: Spike generation Descending thresholds. • Poisson approximation Prapun Suksompong Capacity Analysis of Neurons
  • 9. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion (Leaky) Integrate-and-Fire Model: LIF or IF Let τ1 , τ2 , τ3 , . . . be the sequence of time that the spikes arrive at the spike generating region. The membrane potential at time t is then X (t) = h (t, τm , Ym ) = Ym h(t − τm ). m m This is the “integrate” part of the integrate-and-fire neuron. X (t ) Ym is the weight for the mth spike due to propagation loss, synaptic strength, synaptic failure, etc. Yi + 2 h ( t − τ i + 2 ) Yi h ( t − τ i ) Yi +1h ( t − τ i +1 ) h is the shape function. τi τ i +1 τ i+2 time Prapun Suksompong Capacity Analysis of Neurons
  • 10. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron   Goal Conclusion Integrate-and-Fire Neurons (Con’t) Constant bombardment of spikes leads to increase in membrane potential. As soon as the membrane potential reaches a critical value or threshold, the neuron “fires” an time (t) action potential. Then, everything resets. Refractory period: The time after a AP is produced, during which it is impossible to generate another   Threshold AP. time (t) Set T (t) to be ∞ during this period. Prapun Suksompong Capacity Analysis of Neurons
  • 11. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Integrate-and-Fire Neurons (Summary) ∼ 104 pre-synaptic neurons. Descending Threshold Ascending Membrane Potential time Spike Train time • First jitter: Spike generation Descending thresholds. • Poisson approximation Prapun Suksompong Capacity Analysis of Neurons
  • 12. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Theoretical Approaches to Neuroscience We use IF model, but more biologically-realistic models exist (e.g. Hodgkin and Huxley [’52] model). 1. Too many parameters. Physical measurements “fundamentally disturb cell properties” 2. Provide less insight. Biological structures have evolved via natural selection to operate optimally. See the book Optima for Animals by R. McNeill Alexander. What is the best strength for a bone? At what speed should humans change from walking to running? Prapun Suksompong Capacity Analysis of Neurons
  • 13. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Information-Theoretic Optimization Application of information theory has already found success in many areas of neuroscience. Barlow’s “economy of impulses”[’59, ’69] Minimize redundancy. Linsker’s InfoMax principle [’88, ’89] Maximize the mutual information. Levy and Baxter’s energy-efficient coding [’96, ’02] Maximize mutual information per unit energy expended. Prapun Suksompong Capacity Analysis of Neurons
  • 14. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Motivation and Goal Integrate-and-Fire (IF) model is very popular. The threshold function is a crucial element of the IF model. Little amount of work exists on deriving the form of the threshold curve. In fact, using constant thresholding is also popular. This leads to large jitter in the spike timing and hence discourages the use of time coding. Goal: Find (1) an expression for threshold curve under biologically realistic constraints and (2) the optimal operating point of neuron under such threshold. Prapun Suksompong Capacity Analysis of Neurons
  • 15. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Introduction Sources of Variability for the ISIs and Derivation of the Threshold First Jitter: Spike generation Second and Third Jitters The Threshold Curve Information-Theoretic Analysis of IF Neuron Conclusion Prapun Suksompong Capacity Analysis of Neurons
  • 16. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Three sources of variability for inter-spike intervals 1. Spike generation 2. Spike propagation 3. Time-of-arrival estimation Prapun Suksompong Capacity Analysis of Neurons
  • 17. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion First jitter: Spike generation formula that governs how the membrane potential of this middle neuron rises given the combined incoming rate λ . λ1 λ2 ⊕ λ3 λ = λ1 + λ2 + λ3 + Now, of course, there is some jitter in the timing of the paper today. Large number of presynaptic neurons source of jitter comes from the fact that the in The first allows Poisson approximation for the superposed in fact some jitter in large. That assumption allows has process. presynaptic neurons are them. Now, you may recall th The membrane potential is governed by a filtered Poisson to a Poisson p .. the superposed spike trains … is close process. spike train coming out of a single neuron is not a Poiss a tractable formula that governs how the membrane po Prapun Suksompong given the combined incoming rate λ . Capacity Analysis of Neurons
  • 18. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Approximation for the first timing jitter For fixed λ, different realizations of the membrane potential correspond to different spiking times. Membrane potential X ( t ) Filtered Poisson σX approximation for amount of variation in vertical direction [Parzen’62]. Linear approximation Threshold T ( t ) for amount of σ time variation in horizontal Time direction. Prapun Suksompong Capacity Analysis of Neurons
  • 19. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Membrane potential X ( t ) σX Approximation for the first timing jitter (con’t) Threshold T ( t ) σ time Time H (τ ) T (τ ) c2 H2 (τ ) σtime (τ ) ≈ The figure here shows different realizations of the membrane potentials for a fixed combined incoming rate λ . Here, we see that the randomness from the Poisson arrivals T (τ ) h (τ ) − T (τ ) H (τ ) c1 H (τ ) causes fluctuation in the time that the membrane potentials hit the threshold. Under som linear approximation, we can relate the jitter in the vertical direction to the one in horizontal direction. This then gives us the formula for the magnitude of the timing jitte as a function of the spike time τ . h : shape function. Membrane potential X ( t ) Here, h is the shape function which describes how the membrane potential changes in e.g. exponential response to a single input spike. For the usual leaky integrate-and-fire model, this h star with some amplitude and then decay exponentially. σX h (t ) For conciseness, we define these two integrations which get used in the formula here. t H (t) = h (µ)dµ. 0 Threshold T ( t ) t σ time H2 (t) = h2 (µ)dµ. Time 0 Prapun Suksompong Capacity Analysis of Neurons
  • 20. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Approximation for the first timing jitter (con’t) H (τ ) T (τ ) c2 H2 (τ ) σtime (τ ) ≈ T (τ ) h (τ ) − T (τ ) H (τ ) c1 H (τ ) c1 and c2 are constants Membrane potential X ( t ) which depend on the σX distribution of the weight (Ym ) for each spike. Recall: X (t) = m Ym h(t − τm ). Threshold T ( t ) σ time Time Prapun Suksompong Capacity Analysis of Neurons
  • 21. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Jitter in rate estimation The only information contained in a Poisson process is its rate λ. Different λ’s ⇒ different spiking times τ ’s. Descending Threshold Spike times vary inversely with λ. PSP for Large λ T (τ ) λ(τ ) ≈ c1 H (τ ) PSP for Small λ Error in rate estimation: 1 T (τ ) c2 H2 (τ ) σλ (τ ) ≈ . Time c1 H (τ ) c1 H (τ ) However, this spike time has some jitter, so the $lambda$ estimation also have some error. We then go on and approximate this error: Prapun Suksompong Capacity Analysis of Neurons
  • 22. The second jitter is the randomness in the length of time a spike takes to propa Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivationanother neuron. It is on the order of 10 microseconds. The third jitt synapse on Second and Third Jitters time-of-arrival estimation error; that is, if this neuron tries to measure the inter Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion interval, it needs to find out what time a spike arrives. We borrow some formu Radar guys shown here because this is exactly the problem that they call the ra Second and Third Jitters over N0 here. Iterror depends on the signal-to-noise the shape ofis show ranging problem. The this Es also depends on the bandwidth for ratio which the acti potential. The amount of error here is about 10 microseconds as well. Propagation Time. Time-of-Arrival Estimation Error (radar ranging problem): −1 2Es 2 4π 2 f . N0 f 2 : Gabor-bandwidth. ES : Signal energy. N0 2 : Spectral height of the Noise. Small: < 10µs. Prapun Suksompong Capacity Analysis of Neurons
  • 23. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Deriving The Threshold Curves Recall: Deviation in time: H (τ ) T (τ ) c2 H2 (τ ) σtime (τ ) = . T (τ ) h (τ ) − T (τ ) H (τ ) c1 H (τ ) We consider the thresholds which 1) preserve timing jitter σtime (τ ) ≡ σtime,0 , or σtime (τ ) 2) preserve relative timing jitter τ ≡ σ%time,0 across spiking times (or spiking frequencies) of interest. Prapun Suksompong Capacity Analysis of Neurons
  • 24. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Deriving The Threshold Curves Recall: H(τ ) T (τ )c2 H2 (τ ) (a) Deviation in time: σtime (τ ) = T (τ )h(τ )−T (τ )H(τ ) c1 H(τ ) . 1 T (τ )c2 H2 (τ ) (b) Deviation in λ estimation: σλ (τ ) = c1 H(τ ) c1 H(τ ) . We consider the thresholds which 1) preserve timing jitter σtime (τ ) ≡ σtime,0 , or σtime (τ ) 2) preserve relative timing jitter τ ≡ σ%time,0 , or 3) preserve jitter in λ estimation σλ (τ ) ≡ σλ,0 , or σλ (τ ) 4) preserve relative jitter in λ estimation λ ≡ σ%λ,0 , or 5) preserve jitter in ln λ estimation across spiking times (or spiking frequencies) of interest. Prapun Suksompong Capacity Analysis of Neurons
  • 25. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Deriving The Threshold Curves (con’t) Constant timing jitter level: h (t) 1 c2 H2 (t) T (t) = T (t) − T (t) . H (t) σtime,0 c1 H (t) Constant relative-timing-jitter level: h (t) 1 c2 H2 (t) T (t) = T (t) − T (t) . H (t) tσ%time,0 c1 H (t) Preserve relative error in λ estimation: H2 (t) T (t) = c0 . H (t) Prapun Suksompong Capacity Analysis of Neurons
  • 26. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion The differential equations are Bernoulli equations of the form T (t) = T (t) P (t) − T (t)Q (t) . They can be reduced to linear equation by introducing v (t) = T (t) which gives 1 1 v (t) = P (t) v (t) − Q (t) . 2 2 Linear! The solution is t 1 v (t) = v (t0 ) φ (t, t0 ) − φ (t, τ ) Q (τ )dτ, 2 t0 t 1 2 P(τ )dτ where φ (t, s) = e s . Prapun Suksompong Capacity Analysis of Neurons
  • 27. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Comparison between derived thresholds Constant timing jitter Constant relative timing jitter Exponential Heavy-tail Linear 6 7 8 Time [ms] Prapun Suksompong Capacity Analysis of Neurons
  • 28. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Summary Analyze and quantify three sources of timing jitter Predict shape of threshold curves Constant timing jitter Constant relative timing jitter Exponential Heavy-tail Linear 6 7 8 Time [ms] Prapun Suksompong Capacity Analysis of Neurons
  • 29. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Introduction Sources of Variability for the ISIs and Derivation of the Threshold Information-Theoretic Analysis of IF Neuron OPT1: Maximization of Mutual Information OPT2: Mutual Information per Unit Energy Cost Rate Matching Conclusion Prapun Suksompong Capacity Analysis of Neurons
  • 30. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Conditional density Q(t|λ) = fτ |Λ (t|λ)   We have formula(s) for the threshold curve T (t). Assumption: λ stays constant during each ISI. Given Poisson input intensity λ, can find the conditional density Q(t|λ) = fτ |Λ (t|λ). τ = g (Λ)+jitter. Λ λ  τ   τ |Λ |λ Prapun Suksompong Capacity Analysis of Neurons
  • 31. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Optimization 1: Mutual Information OPT1: sup I (Λ; τ ) where fΛ,τ (Λ, τ ) I (Λ; τ ) = E log fΛ (Λ)fτ (τ ) and the supremum is taken over all possible fΛ (λ). Blahut-Arimoto Algorithm (BAA) Prapun Suksompong Capacity Analysis of Neurons
  • 32. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Exponential “Constant-Jitter” “Constant-Relative-Jitter” Threshold Threshold Threshold -3 -3 -3 x 10 x 10 x 10 6 6 6 4 4 4 f () f () f () 2 2 2 0 0 0 0 500 1000 1500 2000 0 500 1000 1500 2000 0 500 1000 1500 2000    0.8 0.8 0.8 0.6 0.6 0.6 f(t) f(t) f(t) 0.4 0.4 0.4 0.2 0.2 0.2 0 0 0 4 6 8 10 12 14 4 5 6 7 8 4 5 6 7 8 9 t t t C = 5.457 C = 4.931 C = 5.109 (a) (b) (c) Prapun Suksompong Capacity Analysis of Neurons
  • 33. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Capacity-achieving input densities look similar. −3 x 10 6 exponential constant jitter 5 constant relative jitter 4 fΛ(λ) 3 2 1 0 0 200 400 600 800 1000 1200 1400 1600 1800 2000 λ Prapun Suksompong Capacity Analysis of Neurons
  • 34. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Input-Intensity Density Approximation BAA does not provide any insight. Our simpler formula: σ0 (c1 H (t))3 fΛ (λ) ≈ d T (t) c2 H2 (t) t=g (λ) where g (λ) = E [τ |λ]. Prapun Suksompong Capacity Analysis of Neurons
  • 35. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Input-Intensity Density Approximation −3 BAA does not provide any 6 x 10 exponential insight. 5 constant jitter constant relative jitter approximation Our simpler formula: 4 fΛ(λ) (c1 H (t))3 3 σ0 fΛ (λ) ≈ d T (t) c2 H2 (t) 2 t=g (λ) 1 where g (λ) = E [τ |λ]. 0 0 200 400 600 800 1000 1200 1400 1600 1800 2000 λ Prapun Suksompong Capacity Analysis of Neurons
  • 36. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Input-Intensity Density Approximation BAA does not provide any exponential constant jitter insight. constant relative jitter approximation Our simpler formula: fΛ(λ) −3 10 σ0 (c1 H (t))3 fΛ (λ) ≈ d T (t) c2 H2 (t) t=g (λ) where g (λ) = E [τ |λ]. 1 10 10 2 λ 10 3 Prapun Suksompong Capacity Analysis of Neurons
  • 37. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Approximation Strategy Assume that g (Λ) is uniform and then find the corresponding fΛ . Exponential “Constant-Jitter” “Constant-Relative-Jitter” Threshold Threshold Threshold -3 -3 -3 x 10 x 10 x 10 6 6 6 4 4 4 f () f () f () 2 2 2 0 0 0 0 500 1000 1500 2000 0 500 1000 1500 2000 0 500 1000 1500 2000    0.8 0.8 0.8 0.6 0.6 0.6 f(t) f(t) f(t) 0.4 0.4 0.4 0.2 0.2 0.2 0 0 0 4 6 8 10 12 14 4 5 6 7 8 4 5 6 7 8 9 t t t C = 5.457 C = 4.931 C = 5.109 (a) (b) (c) Exponential “Constant-Jitter” “Constant-Relative-Jitter” Threshold Threshold Threshold Prapun Suksompong Capacity Analysis of Neurons
  • 38. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Approximation Strategy Assume that g (Λ) is uniform and then find the corresponding fΛ . ������������  ������ ������ ������������ ������ * Convolution For invertible function g , the pdf of Z = g (Λ) is given by d −1 1 fZ (z) = g (z) fΛ g −1 (z) = fΛ (λ) , dz |g (λ)| where z = g (λ). Prapun Suksompong Capacity Analysis of Neurons
  • 39. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion OPT2: Energy-Efficient Neuron Brains consume 20% of energy consumption for adults and 60% for infant [Laughlin and Sejnowski’03]. Suppose neuron spends 1 unit of energy per ms when it is idle, and e unit of energy per ms when AP is produced. e >> 1. If the time to the next spike is τ = t, the energy expended is bo (t) = 1 × (t − ∆) + e × ∆ = t + (e − 1)∆ = t + r . where ∆ is the time used to produce a spike. The value of r depends on the type of neurons under consideration. Prapun Suksompong Capacity Analysis of Neurons
  • 40. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Optimization 2: I/E OPT2: I (Λ; τ ) sup E [bo (τ )] where the supremum is taken over all possible fΛ (λ). I (Λ;τ ) Jimbo-Kunisawa algorithm (JKA) maximizes E[b(Λ)] . b is a function of input. Our bo is a function of output. We define b(λ) = E [bo (τ )|Λ = λ] and apply JKA. Because bo (τ ) = τ + r , we have b(λ) = E [τ |λ] + r = g (λ) + r . Prapun Suksompong Capacity Analysis of Neurons
  • 41. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Input-Intensity Density Approximation Can use the same technique as in OPT1 to do approximation of input-intensity density. In stead of uniform density, consider bounded exponential density of the form γ f (t; γ, α, β) = e −γt 1[α,β] (t) . e −γα − e −γβ Prapun Suksompong Capacity Analysis of Neurons
  • 42. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Input-Intensity Density Approximation Can use the same technique as in OPT1 to do approximation of input-intensity density. Result: (c1 H (t))3 fΛ (λ) ≈ σ0 f (t; γ, g (b), g (a)) , T (t) c2 H2 (t) t=g (λ) where f (t; γ, g (b), g (a)) is the bounded exponential pdf with support on the interval [g (b), g (a)] and parameter γ. Prapun Suksompong Capacity Analysis of Neurons
  • 43. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion −3 x 10 6 exponential exponential constant jitter constant jitter 5 constant relative jitter 0.5 constant relative jitter approximation 4 0.4 fΛ(λ) fτ(t) 3 0.3 2 0.2 1 0.1 0 0 0 500 1000 1500 2000 4 6 8 10 12 14 λ t [ms] exponential constant jitter constant relative jitter approximation fΛ(λ) −3 fτ(t) 10 −1 10 exponential constant jitter constant relative jitter 1 2 3 4 5 6 7 8 9 10 10 10 10 λ t [ms] Prapun Suksompong Capacity Analysis of Neurons
  • 44. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Free Parameters - Revisited Recall, for example, the differential equation that define our “constant-relative-jitter” threshold: h (t) 1 c2 H2 (t) T (t) = T (t) − T (t) . H (t) tσ%time,0 c1 H (t) There are a couple of parameters which we want to revisit. The constants c1 and c2 . Embedded in them is the effect of QSF. σtime,0 σ%time,0 = t0 . What value should we set σtime,0 to be? > 10µs. Prapun Suksompong Capacity Analysis of Neurons
  • 45. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion By scaling the unit of the voltage, we can make c1 = 1. The scaling makes 1 1 c2 ∝ = psuccess 1 − pfailure where pfailure is the QSF probability. pfailure depends on the type of neurons under consideration. Let σtime,0 = σ1 and play with it. Prapun Suksompong Capacity Analysis of Neurons
  • 46. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Average Rates From our optimization (either OPT1 or OPT2), we get the optimal input-intensity density fΛ (λ) and spiking-time density fτ (t) for each value of σ1 . ¯ Each value of σ1 gives average arrival rate λin and the average ¯ out . spiking rate λ ¯ λout = 1 . E[τ ] ¯ λin = E [Λ]? Prapun Suksompong Capacity Analysis of Neurons
  • 47. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Average Afferent Rate High value of Λ corresponds to low value of τ . Neuron experiences large Λ value for short amount of time. ¯ λin should be lower than E [Λ]. Let (Λi , τi ) be the pair of input-intensity and the length of the ISI associated with the ith spike. The number of input spikes during the ith ISI is a Poisson random variable Ni with mean Λi τi . The long-term average input rate is then k 1 k i=1 Ni k i=1 Ni E [Λ1 τ1 ] k = 1 k → . i=1 τi i=1 τi E [τ1 ] k ¯ E[Λτ ] λin = E[τ ] . Prapun Suksompong Capacity Analysis of Neurons
  • 48. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Rate Matching On average, our neuron in consideration is bombarded with an ¯ average input-intensity λin . Suppose our neuron is receiving input from n other neurons. Then, on average, each of the sending neurons fire at a rate 1¯ of n λin . Our neuron should also generate spikes at this rate. Therefore, we must have 1¯ ¯ λin = λout . n Prapun Suksompong Capacity Analysis of Neurons
  • 49. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Rate Matching (Con’t) Average spiking rate [spikes/s] 200 input 150 output 100 50 0 10 20 30 40 50 60 70 80 90 100 σ1 [microsec] 5.4 Capacity [bits] 5.2 5 4.8 10 20 30 40 50 60 70 80 90 100 σ1 [microsec] Prapun Suksompong Capacity Analysis of Neurons
  • 50. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Rate Matching (Con’t) Capacity value increases as we increase the noise level Average spiking rate [spikes/s] 200 σ1 . 150 input output 100 Larger value of σ1 implies 50 threshold decays slower This 0 10 15 20 25 30 35 40 45 50 55 60 σ1 [microsec] gives larger support for the spiking time. 5.8 Capacity [bits] 5.6 Noise is small. Effect of 5.4 increasing the support of the 5.2 10 15 20 25 30 35 40 45 50 55 60 output is stronger than the σ1 [microsec] effect of increased noise. Prapun Suksompong Capacity Analysis of Neurons
  • 51. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion 60 55 (c) max I/E, r = 100  50 Optimal rate [spikes per second] 45 40 (b) max I/E, r = 400  35 30 25 exponential constant relative jitter (a) max I  20 15 1 1.5 2 2.5 3 3.5 4 4.5 5 c2   Prapun Suksompong Capacity Analysis of Neurons
  • 52. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Rate Matching (Con’t) 60 55 (c) max I/E, r = 100  Firing rate decreases as the 50 Optimal rate [spikes per second] failure probability increases. 45 Smaller value of r 40 (b) max I/E, r = 400  35 corresponds to higher rate. 30 As r → ∞, the rate 25 exponential constant relative jitter (a) max I  converges to the max-I case. 20 15 1 1.5 2 2.5 3 3.5 4 4.5 5 c2   Prapun Suksompong Capacity Analysis of Neurons
  • 53. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Optimal Threshold: c2 = 5 and r = 400 800 700 exponential constant relative timing jitter 600 500 400 300 200 100 0 0 20 40 60 80 100 120 140 160 180 200 Time [ms] Prapun Suksompong Capacity Analysis of Neurons
  • 54. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion Introduction Sources of Variability for the ISIs and Derivation of the Threshold Information-Theoretic Analysis of IF Neuron Conclusion Prapun Suksompong Capacity Analysis of Neurons
  • 55. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion Contributions 1. Quantify the amount of timing jitter in neuron. 2. Construct threshold functions. 3. Provide optimal operating points for neurons which are close to experimentally observed values. Formulas to approximate the optimal input-intensity densities. Prapun Suksompong Capacity Analysis of Neurons
  • 56. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion References I P. Suksompong and T. Berger. Jitter Analysis of Timing Codes for Neurons with Descending Action Potential Thresholds. ISIT, 2006. P. Suksompong and T. Berger. Capacity Analysis of Neurons with Descending Action Potential Thresholds. In preparation for special issue of IEEE Tran. on Info. Theory, 2009. Prapun Suksompong Capacity Analysis of Neurons
  • 57. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion References II P. Suksompong and T. Berger. Energy-Efficient Neurons with Descending Action Potential Thresholds. In preparation for Journal of Comp. Neuroscience, 2009. T. Berger and W.B. Levy. Encoding of excitation via dynamic thresholding. Society for Neuroscience, 2004. W.B. Levy and R.A. Baxter. Energy-Efficient Neuronal Computation via Quantal Synaptic Failures. Journal of Neuroscience, 2002. Prapun Suksompong Capacity Analysis of Neurons
  • 58. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion References III W.B. Levy and R.A. Baxter. Energy Efficient Neural Codes Neural Computation, 1996. Patrick Crotty and William Levy. Biophysical limits on axonal transmission rates in axons. CNS, 2005. E. Parzen. Stochastic Processes. Holden Day, 1962. H. Vincent Poor. An introduction to signal detection and estimation (2nd ed.). Springer-Verlag New York, Inc., New York, NY, USA, 1994. Prapun Suksompong Capacity Analysis of Neurons
  • 59. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion References IV Dale Purve et al. Neuroscience (3rd ed.). Sinauer Associates Inc., Sunderland, MA USA, 1997. Prapun Suksompong Capacity Analysis of Neurons
  • 60. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion THE END Prapun Suksompong Capacity Analysis of Neurons