Stochastic Processes - part 4

Electronic noise
Three types of noise are commonly observed
Shot noise. Electric currents are not continuous but are
ultimately made up from large numbers of moving
charge carriers, typically electrons. Shot noise arises
from statistical fluctuations in the flow of charge
carriers: if a single bit of data is represented by 10,000
electrons, the magnitude of the fluctuations will
typically be about 1%. When looked at as a waveform
over time, shot noise has a flat frequency spectrum.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.1/123

Thermal (Johnson) noise. Even though an electric
current may have a definite overall direction, the
individual charge carriers within it will exhibit random
motions. In a material at nonzero temperature, the
energy of these motions and thus the intensity of the
thermal noise they produce is essentially proportional
to temperature. (At very low temperatures, quantum
mechanical fluctuations still yield random motion in
most materials.) Like shot noise, thermal noise has a
flat frequency spectrum.

Flicker (1/f) noise. Almost all electronic devices also
exhibit a third kind of noise, whose main characteristic
is that its spectrum is not flat, but instead goes roughly
like 1/f over a wide range of frequencies. Such a
spectrum implies the presence of large low-frequency
fluctuations, and indeed fluctuations are often seen
over timescales of many minutes or even hours. Unlike
the types of noise described above, this kind of noise
can be affected by details of the construction of
individual devices. Although seen since the 1920s its
origins remain somewhat mysterious

Shot noise in electronic devices consists of random
fluctuations of the electric current in an electrical
conductor, which are caused by the fact that the current is
carried by discrete charges (electrons).
Shot noise is to be distinguished from current fluctuations
in equilibrium, which happen without any applied voltage
and without any average current flowing. These equilibrium
current fluctuations are known as thermal noise.

h(t)
Z(t)
× × × ×
S(t)
Figure 1: Shot noise.

Given a set of Poisson points ti with average density λ and
a real system h(t)
S(t) =
X
i
h(t − ti)
is an SSS process known as shot noise.
S(t) = Z(t) ∗ h(t) =
Z ∞
−∞
h(α)Z(t − α)dα

E{S(t)} =
Z ∞
−∞
h(α)E{Z(t − α)}dα
= ηZ
Z ∞
−∞
h(α)dα = ηZH(0)
Z(t) =
X
i
δ(t − ti) =
d
dt
X(t)
z }| {
X
i
u(t − ti), Z(t) = X′
(t)
X(t) is a Poisson process.

RXX(t1, t2) = λ2
t1t2 + λ min(t1, t2)
RXZ(t1, t2) =
∂
∂t1
RXX(t1, t2) = λ2
t1 + λu(t1 − t2)
RZZ(t1, t2) = λ2
+ λδ(t1 − t2)
SZZ(ω) = 2πλ2
δ(ω) + λ
SSS(ω) = 2πλ2
|H(0)|2
δ(ω) + λ |H(ω)|2
RSS(τ) = λ2
|H(0)|2
+ λρ(τ), ρ(τ) = h(τ) ∗ h(−τ)
CSS(τ) = λρ(τ)

Markov process:
In probability theory, a Markov process is a stochastic
process characterized as follows: The state ck at time k is
one of a finite number in the range
{1, · · · , M}
Under the assumption that the process runs only from time
0 to time N and that the initial and final states are known,
the state sequence is then represented by a finite vector
C = (c0, · · · , cN ).

Let P(ck|c0, c1, · · · , c(k−1)) denote the probability (chance of
occurrence) of the state ck at time k conditioned on all states
up to time k − 1. Suppose a process was such that ck de-
pended only on the previous state ck−1 and was indepen-
dent of all previous states. This process would be known as
a first-order Markov process.

This means that the probability of being in state ck at time
k, given all states up to time k − 1 depends only on the
previous state, i.e. ck−1 at time k − 1:
P(ck|c0, c1, . . . , ck−1) = P(ck|ck−1).
For an nth-order Markov process,
P(ck|c0, c1, . . . , ck−1) = P(ck|ck−n, . . . , ck−1).

The underlying process is assumed to be a Markov
process with the following characteristics:
finite-state, this means that the number M is finite.
discrete-time, this means that going from one state to
other takes the same unit of time.
observed in memoryless noise, this means that the
sequence of observations depends probabilistically
only on the previous sequence transitions.

In probability theory, a stochastic process has the Markov
property if the conditional probability distribution of future
states of the process, given the present state, depends only
upon the current state, i.e. it is conditionally independent of
the past states (the path of the process) given the present
state. A process with the Markov property is usually called
a Markov process, and may be described as Markovian.

Mathematically, if X(t), t > 0, is a stochastic process, the
Markov property states that
Pr[X(t+h) = y|X(s) = x(s), s ≤ t] = Pr[X(t+h) = y|X(t) = x(t)], ∀h > 0
Markov processes are typically termed (time-)
homogeneous if
Pr[X(t + h) = y|X(t) = x(t)] = Pr[X(h) = y|X(0) = x(0)], ∀t, h > 0

and otherwise are termed (time-) inhomogeneous (or (time-
) nonhomogeneous). Homogeneous Markov processes,
usually being simpler than inhomogeneous ones, form the
most important class of Markov processes.

In some cases, apparently non-Markovian processes may
still have Markovian representations, constructed by
expanding the concept of the ’current’ and ’future’ states.
Let X be a non-Markovian process. Then we define a
process Y, such that each state of Y represents a
time-interval of states of X, i.e. mathematically
Y (t) = {X(s) : s ∈ [a(t), b(t)]}.

If Y has the Markov property, then it is a Markovian rep-
resentation of X. In this case, X is also called a second-
order Markov process. Higher-order Markov processes are
defined analogously.

An example of a non-Markovian process with a Markovian
representation is a moving average time series.
Xt = εt +
q
X
i=1
θiεt−i
where the θ1, · · · , θq are the parameters of the model and
the εt, εt−1, · · · are again, the error terms. A moving aver-
age model is essentially a finite impulse response filter with
some additional interpretation placed on it.

The most famous Markov processes are Markov chains,
but many other processes, including Brownian motion, are
Markovian.

Markov chain:
A collection of random variables {Xt} (where the index
runs through 0, 1, . . . ) having the property that, given the
present, the future is conditionally independent of the past.
In other words,
P{Xt = j|X0 = i0, X1 = i1, · · · , Xt−1 = i(t−1)} = P{Xt = j|Xt−1 = it−1}
If a Markov sequence of random variates Xn take the dis-
crete values {a1, · · · , aN }, then

P{xn = ain |xn−1 = ain−1 , · · · , x1 = ai1 } = P{xn = ain |xn−1 = ain−1 }
and the sequence Xn is called a Markov chain. A simple
random walk is an example of a Markov chain.

Example of Markov chains
The probabilities of weather conditions, given the weather
on the preceding day, can be represented by a transition
matrix:
P =

0.9 0.5
0.1 0.5

The matrix P represents the weather model in which a
sunny day is 90% likely to be followed by another sunny
day, and a rainy day is 50% likely to be followed by another
rainy day.

The columns can be labelled “sunny” and “rainy” respec-
tively, and the rows can be labelled in the same order. Pij
is the probability that, if a given day is of type j, it will be
followed by a day of type i. Notice that the columns of P
sum to 1. This is because P is a stochastic matrix.

Predicting the weather
The weather on day 0 is known to be sunny. This is
represented by a vector in which the “sunny” entry is
100%, and the “rainy” entry is 0%:
x(0)
=

1
0

The weather on day 1 can be predicted by:

x(1)
= Px(0)
=

0.9 0.5
0.1 0.5

1
0

=

0.9
0.1

Thus, there is an 90% chance that day 1 will also be sunny.

The weather on day 2 can be predicted in the same way:
x(2)
= Px(1)
= P2
x(0)
=

0.9 0.5
0.1 0.5
2
1
0

=

0.86
0.14

General rules for day n are:
x(n)
= Px(n−1)
x(n)
= Pn
x(0)

Steady state of the weather In this example, predictions for
the weather on more distant days are increasingly
inaccurate and tend towards a steady state vector. This
vector represents the probabilities of sunny and rainy
weather on all days, and is independent of the initial
weather.
The steady state vector is defined as:
q = lim
n→∞
x(n)
but only converges if P is a regular transition matrix (that
is, there is at least one Pn
with all non-zero entries).

Since the q is independent from initial conditions, it must be
unchanged when transformed by P. This makes it an
eigenvector (with eigenvalue 1), and means it can be
derived from P. For the weather example:
P =


0.9 0.5
0.1 0.5


Pq = q
(I − P)q = 0 ⇒

0.1 −0.5
−0.1 0.5

q =

0
0


q1 + q2 = 1, 0.1q1 − 0.5q2 = 0 ⇒
q1 = 0.833, q2 = 0.167
In conclusion, in the long term, 83% of days are sunny.

For homogeneous Markov chain:
The statistics of any order can be determined in terms of
the conditional PDF f(Xn|Xn−1) and f(Xn),
f(Xn|Xn−1, · · · , X0) = f(Xn|Xn−1)
and if Xn is stationary then f(Xn) and f(Xn|Xn−1) are
invariant to shift of origin. A Markov process is
homogeneous if the conditional PDF f(Xn|Xn−1) is
invariant to shift of the origin

Chapman-Kolmogorov Equation
f(xn|xs) =
Z ∞
−∞
f(xn|xr)f(xr|xs) dxr
which gives the transitional densities of a Markov sequence.
Here, n r s are any integers.

Continuous time-discrete state Markov chain (CTDSMC)
A CTDSMC is Markov process X(t) consisting of a family
of staircase functions(discrete states) with discontinuities at
random times tn.

× × × × × ×
aj
ai
Figure 2: CTDSMC
πij(t1, t2) = P{X(t2) = aj|X(t1) = ai}

×
t1
×
t2
× × × ×
tk
qn = X(t+
n ), n = 1, 2, · · ·
Figure 3: Imbedded sequence qn is Markov chain.

X(t) at these random points form a discrete-state Markov
sequence called the Markov chain imbedded in the
process X(t).
A discrete-state stochastic process is called semi-Markov if
it is not Markov but the imbedded sequence qn is Markov
chain.

Pi(t) = P{X(t) = ai}, state probabilities
πij(t1, t2) = P{X(t2) = aj|X(t1) = ai}, transition probabilities
X
j
πij(t1, t2) = 1,
X
i
Pi(t1)πij(t1, t2) = Pj(t2)

Discrete-time Markov chain is Markov chain where Xn has a
countable number of states ai. This kind of Markov process
is specified by:

state probabilities:
Pi(n) = P{Xn = ai}, i = 1, 2, · · ·
transition probabilities:
πij(n1, n2)P{Xn2 = aj|Xn1 = ai}
X
j
πij(n1, n2) = 1
P{Xn = aj} =
X
i
P{Xn2 = aj, Xn1 = ai}

Chapman-Kolmogorov Equation, discrete version
πij(n1, n3) =
X
r
πir(n1, n3)πrj(n2, n3)

If Xn is homogeneous, then the transition probabilities
depend only on m = n2 − n1 ⇒
πij(n1, n2) = πij(m) = P{Xn+m = aj|Xn = ai}
the Chapman-Kolmogorov equation will be:
πij(n1, n3) =
X
r
πir(n1, n3)πrj(n2, n3) ⇒

Πij(n + k) =
X
r
elements
z }| {
Πir(k)
| {z }
matrix
Πrj(n)
| {z }
matrix
Π(n + k) = Π(k)Π(n)
the right hand side is matrix at time n + k

For homogeneous discrete Markov chain
P{Xn2 = aj} =
X
i
P{Xk = ai}πij(k, n) ⇒
P(n) = P(0)Πn
Π =





π11 π12 · · · π1N
π21 π22 · · · π2N
.
.
.
.
.
. · · ·
.
.
.
πN1 πN2 · · · πNN





,
X
j
πij = 1
Πn
is the state transition matrix at the time instant n = 2.

Steady state probabilities:
P = PΠ,
N
X
i=1
pi = 1

Random walk with two reflecting walls.
Π =







0 1 0 0 0
.5 0 0 .5 0
0 0.5 0 0.5 0
0 0 0.5 0 0.5
0 0 0 1 0







Steady state probabilities:
P = PΠ

eig(AT
) provides the eigenvector corresponding to the
eigenvalue=1
P =

1
8
1
4
1
4
1
4
1
8


Continuous time Markov chain discrete state:
Stationary homogeneous transition probabilities:
πij(t) = P{X(t+s) = aj|X(s) = ai}, i, j = 0, 1, · · · , m, s, t 0
lim
t→0
πij(t) =

0, i 6= j
1, i = j
lim
t→∞
πij(t) = Pj =
steady state probability,
independent of initial state probability vector

Chapman-Kolomogorov equation:
πij(t) =
m
X
ℓ=0
πiℓ(ν)πℓj(t − ν), 0 6 ν 6 t
Π(τ + α) = Π(τ)Π(α)
Steady state probabilities satisfy:
Pm
j=0 Pj = 1
Pm
i=0 Piπij(t) = Pj
, j = 0, 1, · · · , m, t 0

λj = lim
∆t→0

1 − πjj(∆t)
∆t

= −
d
dt
πjj(t) |t=0
λj=rate(intensity) of transition
λij = lim
∆t→0

πij(∆t)
∆t

=
d
dt
πij(t) |t=0
λij=rate(intensity) of passage

Π′
(0+
) = Λ =





λ11 · · · λ1n
λ21 · · · λ2n
.
.
.
.
.
.
.
.
.
λn1 · · · λnn





X
j
πij(τ) = 1 ⇒
d
dτ
X
j
πij(τ) = 0 ⇒
X
j
λij = 0
λii +
X
i6=j
λij = 0 ⇒ µi = −λii

P{X(t + ∆t) = aj|X(t) = ai} =

1 − µi∆t, i = j
λij∆t, i 6= j
⇒
d
dα
{Π(τ + α) = Π(τ)Π(α)} |α=0
Π′
(τ) = Π(τ)Π′
(0) ⇒ Π′
(τ) = Π(τ)Λ, Π(0) = I
Π(τ) = eΛτ
X
i
Pi(t1)πij(t1, t2) = Pj(t2) ⇔ P(t)Π(τ) = P(t + τ)

if homogeneous Markov chain(homogeneity condition):
d
dτ
{P(t)Π(τ) = P(t + τ)} |τ=0 ⇒
P(t)Λ = P′
(t), P(0) = P0 ⇒
P(t) = P0eΛt
P(t)Λ = P′
(t), P(0) = P0 ⇒
sP(s) − P0 = P(s)Λ ⇒
P(s) = (sI − Λ)−1
P0 ⇔ P(t) = P0eΛt

Telegraph signal:
A
−A
λ12 = µ2
µ1 µ2
λ21 = µ1
A
−A
Figure 4: Telegraph signal.

The process takes on values A and −A
P′
(t) = P(t)

−µ1 µ1
µ2 −µ2

, µ1, µ2 0
P1(t) =
µ2
µ1 + µ2

1 − e−(µ1+µ2)t

+ P1(0)e−(µ1+µ2)t
P2(t) = 1 − P1(t)
E{X(t)} = A[P1(t) − P2(t)] = A[2P1(t) − 1]

E{X(t1)X(t2)} = RXX(t1, t2)
by using
Π′
(τ) = Π(τ)Λ
we have
RXX(t1, t2) = A2
[P{X(t + τ) = A, X(t) = A}
+P{X(t + τ) = −A, X(t) = −A}−
P{X(t + τ) = −A, X(t) = A}
−P{X(t + τ) = A, X(t) = −A}]

Then we conclude that the telegraph signal is a non-
stationary signal. But as t → ∞, E{X(t)} is constant and
E{X(t1)X(t2)} is only a function of time lag.

Caley-Hamilton theorem:
∆(A) = |A − λI| = (−λ)n
+
n−1
X
i=0
ciλi
= characteristic polynomial
∆(λ) = (−1)n
An
+
n−1
X
i=0
ciAi
A = MΛM−1
⇒ Ak
= MΛk
M−1
⇒
∆(A) = M

(−1)n
Λn
+
n−1
X
i=0
ciΛi
#
M−1

A =

3 1
1 2

∆(λ) = |A − λI| = (3 − λ)(2 − λ) − 1 = λ2
− 5λ + 5 ⇒
∆(A) = A2
− 5A + 5I =

0 0
0 0

∆(A) = (−1)n
An
+
n−1
X
i=0
ciAi
= 0 ⇒

(−1)n
An
+ · · · + c1A + c0I = 0 ⇒ (−1)n
An−1
+ · · · + c1I + c0A−1
= 0
A−1
= −1
c0
Pn−1
i=1 ciAi
A =

3 1
1 2

∆(A) = A2
− 5A + 5I = 0 ⇒ A−1
=
−1
5

−2 1
1 −3


P(x) = 3x4
+ 2x2
+ x + 1, P1(x) = x2
− 3
= P1(x)(3x2
+ 11) + R(x), R(x) = x + 34
This is true for matrices as well.
P(A) = ∆(A)Q(A) + R(A), ∆(A) = 0 ⇒
P(A) = R(A)

P(A) = A4
+ 3A3
+ 2A2
+ A + I, A =

3 1
1 2

∆(x) = x2
− 5x + 5
P(x) = ∆(x)(x2
+ 8x + 37) + (146x − 184) ⇒
P(A) = 146A − 184I

f(x) =
∞
X
k=0
γkxk
⇒ f(x) = ∆(x)
∞
X
k=0
βkxk
+ R(x)
R(x) = α0 + α1x + · · · + αn−1xn−1
In order to find {αi}, we note that ∆(λi) = 0 ⇒
R(λi) = f(λi)

A =

−3 1
0 −2

⇒
Determine sin A.
∆(A) = (−3 − λ)(−2 − λ)
R(x) = α0 + α1x
sin(−2) = α0 + α1(−2)
sin(−3) = α0 + α1(−3)
α0 = 3 sin(−2) − 2 sin(−3), α1 = sin(−2) − sin(−3)

sin A = α0I + α1A =

sin(−3) sin(−2) − sin(−3)
0 sin(−2)


However, for λi is a repeated root:
d
dλ
∆(λ) |λi
= 0
A =


0 1 0
0 0 1
27 −27 9


Determine eAt
|A − λI| = (3 − λ)3
= 0
eAt
= R(A) = α0 + α1A + λ2A2
⇒




e3t
= α0 + 3α1 + 9α2
te3t
= α1 + 2α2(3)
t2
e3t
= 2α2
eAt
=


1 − 3t + 4.5t2
t − 3t2
0.5t2
13.5t2
1 − 3t − 9t2
t + 1.5t2
27t + 40.5t2
−27t − 27t2
1 + 6t + 4.5t2

 e3t

Λ =

−µ1 µ1
µ2 −µ2

⇒ λ =

0
−(µ1 + µ2)
Π(τ) = eΛτ
= α0 + α1Λ
1 = α0, e−(µ1+µ2)τ
= α0 + (−µ1 − µ2)α1 ⇒
Π(τ) = I +

1−e−(µ1+µ2)τ
µ1+µ2

Λ

As another strategy,
Π(τ) =

π11(τ) π12(τ)
π11(τ) π22(τ)

Π′
(τ) = Π(τ)Λ, Π(0) = I ⇒
π11(τ) + π12(τ) = 1, π21(τ) + π22(τ) = 1
π′
11(τ) = −µ1π11(τ) + µ2π12(τ), π11(0) = 1
π′
11(τ) = −µ1π11(τ) + µ2 (1 − π11(τ))
Taking Laplace transform:
sπ11(s) − 1 = −µ1π11(s) +
µ2
s
− µ2π11(s)

π11(s) =
1
s + µ1 + µ2
+
µ2
s(s + µ1 + µ2)
⇒
π11(τ) = µ1
µ1+µ2
e−(µ1+µ2)τ
+ µ2
µ1+µ2
π′
22(τ) = µ1π21(τ) − µ2π22(τ), π22(0) = 1;
π22(τ) = µ2
µ1+µ2
e−(µ1+µ2)τ
+ µ1
µ1+µ2

For a discrete-time discrete-state Markov chain the
transition probabilities are as follows:
Π =


0 0.75 0.25
0.25 0 0.75
0.25 0.25 0.5


The steady state and transient probabilities are as follows.
P = PΠ,
3
X
i=1
pi = 1 ⇒
p1 =
1
5
, p2 =
7
25
, p3 =
13
25

For transient state probabilities:
P(n) = P(n − 1)Π ⇒
P(1) = P(0)Π, P(2) = P(1)Π, · · · , P(n) = P(0)Πn
The first strategy using Caley-Hamilton:
eig(Π) = {1, −0.25, −0.25}
Πn
= α0I + α1Π + α2Π2
, nΠn−1
= α1 + 2α2Π ⇒
1n
= α0 + α1 + α2
(−0.25)n
= α0 + (−0.25)α1 + (−0.25)2
α2
n(−0.25)n−1
= α1 + 2(−0.25)α2

α0 = 0.04 + 0.96

−1
4
n
+ 0.2n

−1
4
n−1
α1 = 0.32 − 0.32

−1
4
n
+ 0.6n

−1
4
n−1
α2 = 0.64 − 0.64

−1
4
n
− 0.8n

−1
4
n−1
P(n) = P(0)Πn
As for the second strategy, we use Z transform:

Z{P(n)} =
∞
X
n=0
P(n)zn
=
∞
X
n=0
P(0)Πn
zn
= P(0)(1 − zΠ)−1
Because |eig(Π)| 6 1 ⇒ (1 − zΠ)−1
exists.
I − zΠ =


1 −0.75z −0.25z
−0.25z 1 −0.75z
−0.25z −0.25z 1 − 0.5z



(I −zΠ)−1
=




1 − z/2 − 3z2
/16 3z/4 − 5z2
/16 z/4 + 9z2
/16
z/4 + z2
/16 1 − z/2 − z2
/16 3z/4 + z2
/16
z/4 + z2
/16 z/4 + 3z2
/16 1 − 3z2
/16




(1 − z)(1 + 0.25z)2
(I−zΠ)−1
=




5 7 13
5 7 13
5 7 13




25(1 − z)
+




0 −8 8
0 2 −2
0 2 −2




5(1 + z/4)2
+




20 33 −53
−5 8 −3
−5 −17 22




25(1 + z/4)

Πn
=
ergodic irreducible
z }| {


5 7 13
5 7 13
5 7 13


25
+
1
5
(n + 1)

−1
4
n


0 −8 8
0 2 −2
0 2 −2


+
1
25

−1
4
n


20 33 −53
−5 8 −3
−5 −17 22



If for n → ∞,
P(n) = P0Πn
has a limit which is independent of state of Markov chain,
then the Markov process is called ergodic and irreducible.
In general, for irreducible, ergodic Markov chain:
lim
n→∞
πij(n) = Pj
these steady state probabilities Pj satisfy
Pn
j=1 Pj = 1
Pj =
Pn
i=1 Piπij

Birth-Death process:
A birth is referred to as an arrival a death as a departure
from a physical system.
X(t) is the number of customers in the system at time
t.
States=0,1,2,· · ·,j,j+1,· · ·
S is the number of servers.
Pj(t) = P{X(t) = j}
Pj = lim
t→∞
Pj(t)
λn mean arrival rate given n customers are in the
system
µn mean service rate given n customers are in the
system AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.76/123

Birth-death process can be used to describe how X(t)
changes through time.
The basic assumption is Poisson arrival so that the
probability of more than one birth or death at the same
instant is zero.
Because of Poisson arrival, when X(t) = j, the PDF of the
time to the next birth(arrival) is exponential with parameter
λj.

P′
0(t) = −λ0P0(t) + µ1P1(t)
P′
j(t) = −(λj + µj)Pj(t) + λj−1Pj−1(t)
+µj+1Pj+1(t), j = 1, 2, · · ·
∞
X
j=0
Pj(t) = 1
In the steady state: P′
j(t) = 0

µ1P1 = λ0P0
λ0P0 + µ2P2 = (λ2 + µ2)P1
.
.
.
.
.
.
.
.
.
λj−2Pj−2 + µjPj = (λj−1 + µj−1)Pj−1
P1 =
λ0
µ2
P0, Pj+1 =
λj
µj+1
Pj =
Πj
k=0λk
Πj+1
k=1µk
P0
Cj =
Πj−1
k=0λk
Πj
k=1µk
, Pj = CjP0,
∞
X
j=0
Pj = 1 ⇒
P0 =
1
1 +
P∞
j=1 Cj

Specific scenarios:
λj = λ µ =mean service rate per busy server.
µj = sµ, j s
µj = jµ, j s

In queueing theory, the inter-arrival times (i.e. the times be-
tween customers entering the system) are often modeled
as exponentially distributed variables. The length of a pro-
cess that can be thought of as a sequence of several inde-
pendent tasks is better modeled by a variable following the
gamma distribution (which is a sum of several independent
exponentially distributed variables).

The exponential distribution is used to model Poisson pro-
cesses, which are situations in which an object initially in
state A can change to state B with constant probability per
unit time λ. The time at which the state actually changes is
described by an exponential random variable with parame-
ter λ. Therefore, the integral from 0 to T over PDF is the
probability that the object is in state B at time T.

An important property of the exponential distribution is that
it is memoryless. This means that if a random variable T is
exponentially distributed, its conditional probability obeys
P(T s + t|T t) = P(T s), ∀s, t 0.

This says that the conditional probability that we need to
wait, for example, more than another 10 seconds before
the first arrival, given that the first arrival has not yet hap-
pened after 30 seconds, is no different from the initial prob-
ability that we need to wait more than 10 seconds for the
first arrival. This is often misunderstood by students tak-
ing courses on probability: the fact that P(T 40|T 30) =
P(T 10) does not mean that the events T 40 and T 30
are independent.

To summarize: “memorylessness” of the probability
distribution of the waiting time T until the first arrival means
P(T 40|T 30) = P(T 10)
It does not mean
P(T 40|T 30) = P(T 40)

The arrival and service processes follow the following PDF:
fa(t) = λe−λt
, fs(t) = µe−µt
The inter-arrival and service times in a busy channel
produce rates λ and µ.
Mean inter-arrival time = 1
λ
Mean for busy channel to complete service = 1
µ

Expected number of customers in the queueing system:
L =
∞
X
j=0
jPj
Expected queue length:
Lq =
∞
X
j=0
jPj − s
W =Expected waiting time in the system. Wq =Expected
waiting time in the queue(excluding service time).
L = λW, Lq = λWq
W = Wq +
1
µ

If λj are not equal, λ̄ replaces λ
λ̄ =
∞
X
j=0
λjPj
System utilization factor:
ρ =
λ
sµ
ρ is the fraction of time that the servers are busy.

An important case s = 1:
We assume an unlimited queue length with exponential
inter-arrivals and
λ0 = λ1 = λ2 = · · · = λ
We also assume the service times will be independent with
exponential distribution, and
µ1 = µ2 = µ3 = · · · = µ ⇒
Cj =

λ
µ
j
= ρj
, j = 1, 2, · · ·

Pj = ρj
P0, P0 =
1
1 +
P∞
j=1 ρj
= 1 − ρ
The steady state probabilities:
Pj = (1 − ρ)ρj
, j = 1, 2, · · ·
Pj the probability that there are j customers in the system
follows a geometric distribution.

The exponential distribution may be viewed as a
continuous counterpart of the geometric distribution, which
describes the number of Bernoulli trials necessary for a
discrete process to change state. In contrast, the
exponential distribution describes the time for a continuous
process to change state.

Expected number of customers in the system:
L =
∞
X
j=0
jPj =
∞
X
j=0
j(1 − ρ)ρj
=
(1 − ρ)ρ
∞
X
j=0
d
dρ
ρj
=
ρ
1 − ρ
Expected queue length:
Lq =
∞
X
j=0
jPj − 1 = L − (1 − P0) =
λ2
µ(µ − λ)

Expected waiting time in the system:
W =
L
λ
=
ρ
λ(1 − ρ)
=
1
µ − λ
Expected waiting time in the queue:
Wq =
Lq
λ
=
λ
µ(µ − λ)

Queueing Theory Basics
A good understanding of the relationship between conges-
tion and delay is essential for designing effective congestion
control algorithms. Queuing Theory provides all the tools
needed for this analysis.

Communication Delays
Lets understand the different components of delay in a
messaging system. The total delay experienced by
messages can be classified into the following categories:
Processing Delay
Queuing Delay
Transmission Delay
Propagation Delay
Retransmission Delay

Processing Delay
This is the delay between the time of receipt of a
packet for transmission to the point of putting it into the
transmission queue.
On the receive end, it is the delay between the time of
reception of a packet in the receive queue to the point
of actual processing of the message.
This delay depends on the CPU speed and CPU load
in the system.

Queuing Delay
This is the delay between the point of entry of a packet
in the transmit queue to the actual point of
transmission of the message.
This delay depends on the load on the communication
link.

Transmission Delay
This is the delay between the transmission of first bit of
the packet to the transmission of the last bit.
This delay depends on the speed of the
communication link.

Propagation Delay
This is the delay between the point of transmission of
the last bit of the packet to the point of reception of last
bit of the packet at the other end.
This delay depends on the physical characteristics of
the communication link.

Retransmission Delay
This is the delay that results when a packet is lost and
has to be retransmitted.
This delay depends on the error rate on the link and
the protocol used for retransmissions.

We will be dealing primarily with queueing delay.
Little’s Theorem
Little’s theorem states that:
The average number of customers (N) can be determined
from the following equation:
N = λT
λ is the average customer arrival rate.
T is the average service time for a customer.

We will focus on an intuitive understanding of the result.
Consider the example of a restaurant where the customer
arrival rate (λ) doubles but the customers still spend the
same amount of time in the restaurant (T). This will double
the number of customers in the restaurant (N). By the same
logic if the customer arrival rate remains the same but the
customers service time doubles, this will also double the
total number of customers in the restaurant.

Queueing System Classification
With Little’s Theorem, we have developed some basic un-
derstanding of a queueing system. To further our under-
standing we will have to dig deeper into characteristics of a
queueing system that impact its performance.

For example, queueing requirements of a restaurant will
depend upon factors like:
How do customers arrive in the restaurant? Are
customer arrivals more during lunch and dinner time (a
regular restaurant)? Or is the customer traffic more
uniformly distributed (a cafe)?
How much time do customers spend in the restaurant?
Do customers typically leave the restaurant in a fixed
amount of time? Does the customer service time vary
with the type of customer?
How many tables does the restaurant have for
servicing customers?

The above three points correspond to the most important
characteristics of a queueing system. They are explained
under the following:
Arrival Process
Service Process
Number of Servers

Arrival Process
The probability density distribution that determines the
customer arrivals in the system.
In a messaging system, this refers to the message
arrival probability distribution.

Service Process
The probability density distribution that determines the
customer service times in the system.
In a messaging system, this refers to the message
transmission time distribution. Since message
transmission is directly proportional to the length of the
message, this parameter indirectly refers to the
message length distribution.

Number of customers
Number of servers available to service the customers.
In a messaging system, this refers to the number of
links between the source and destination nodes.

Based on the above characteristics, queueing systems can
be classified by the following convention:
A/S/n
Where A is the arrival process, S is the service process
and n is the number of servers. A and S can be any of the
following:
M (Markov) Exponential probability density
D (Deterministic) All customers have the same value
G (General) Any arbitrary probability distribution

Examples of queueing systems that can be defined with
this convention are:
1. M/M/1
2. M/D/n
3. G/G/n

M/M/1:
This is the simplest queueing system to analyze. Here the
arrival and service time are poisson process. The system
consists of only one server. This queueing system can be
applied to a wide variety of problems as any system with
a very large number of independent customers can be ap-
proximated as a Poisson process. Using a Poisson process
for service time however is not applicable in many applica-
tions and is only a crude approximation.

M/D/n:
Here the arrival process is poisson and the service time
distribution is deterministic. The system has n servers. (e.g.
a ticket booking counter with n cashiers.) Here the service
time can be assumed to be same for all customers).

G/G/n:
This is the most general queueing system where the ar-
rival and service time processes are both arbitrary. The
system has n servers. No analytical solution is known for
this queueing system.

M/M/1 Queueing System
M/M/1 refers to Poisson arrivals and service times with a
single server. This is the most widely used queueing sys-
tem in analysis as pretty much everything is known about
it. M/M/1 is a good approximation for a large number of
queueing systems.

Poisson Arrivals
M/M/1 queueing systems assume a Poisson arrival
process. This assumption is a very good approximation for
arrival process in real systems that meet the following
rules:
1. The number of customers in the system is very large.
2. Impact of a single customer on the performance of the
system is very small, i.e. a single customer consumes
a very small percentage of the system resources.
3. All customers are independent, i.e. their decision to
use the system are independent of other users.

Example: Cars on a Highway
As you can see these assumptions are fairly general, so
they apply to a large variety of systems. Lets consider the
example of cars entering a highway. Lets see if the above
rules are met.
1. Total number of cars driving on the highway is very
large.
2. A single car uses a very small percentage of the
highway resources.
3. Decision to enter the highway is independently made
by each car driver.

The above observations mean that assuming a Poisson ar-
rival process will be a good approximation of the car arrivals
on the highway. If any one of the three conditions is not met,
we cannot assume Poisson arrivals. For example, if a car
rally is being conducted on a highway, we cannot assume
that each car driver is independent of each other. In this
case all cars had a common reason to enter the highway
(start of the race).

Another Example: Telephony Arrivals
Consider arrival of telephone calls to a telephone
exchange. Putting our rules to test we find:
1. Total number of customers that are served by a
telephone exchange is very large.
2. A single telephone call takes a very small fraction of
the systems resources.
3. Decision to make a telephone call is independently
made by each customer.

Again, if all the rules are not met, we cannot assume tele-
phone arrivals are Poisson. If the telephone exchange is
a (Private Branch eXchange) (PBX) catering to a few sub-
scribers, the total number of customers is small, thus we
cannot assume that rule 1 and 2 apply. If rule 1 and 2 do
apply but telephone calls are being initiated due to some
disaster, calls cannot be considered independent of each
other. This violates rule 3.

Poisson Arrival Process
Pn(t) =
(λt)n
n!
e−λt
This equation describes the probability of seeing n arrivals
in a period from 0 to t. Where:
t is used to define the interval 0 to t
n is the total number of arrivals in the interval 0 to t.
λ is the total average arrival rate in arrivals/sec

Poisson Service times
In an M/M/1 queueing system we assume that service times
for customers are also exponentially distributed (i.e. gener-
ated by a Poisson process). Unfortunately, this assumption
is not as general as the arrival time distribution. But it could
still be a reasonable assumption when no other data is avail-
able about service times. Lets see a few examples:

Telephone Call Durations
Telephone call durations define the service time for
utilization of various resources in a telephone exchange.
Lets see if telephone call durations can be assumed to be
exponentially distributed.
1. Total number of customers that are served by a
telephone exchange is very large.
2. A single telephone call takes a very small fraction of
the systems resources.
3. Decision on how long to talk is independently made by
each customer

From these rules it appears that exponential call hold times
are a good fit. Intuitively, the probability of a customers mak-
ing a very long call is very small. There is a high probability
that a telephone call will be short. This matches with the
observation that most telephony traffic consists of short du-
ration calls. (The only problem with using the exponential
distribution is that, it predicts a high probability of extremely
short calls).

Stochastic Processes - part 4

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Stochastic Processes - part 4

Similar to Stochastic Processes - part 4 (20)

Recently uploaded

Recently uploaded (20)

Stochastic Processes - part 4