55. 1.3.1 生物電位與活化電位 (8)
• 我們可以用如圖 1.8 所示的電路來模擬
位於軸突的細胞膜。
• 其中 C 代表細胞膜的電容性， g 、 g
m k Cl
和 gNa 代表離子進出入細胞膜的難易度
，
• g 與 gNa 是用可變電阻來說明細胞膜的
k
特性，亦即對鉀與鈉離子來說，進出細胞
膜的難易度是會改變的
57. 1.3.1 生物電位與活化電位 (9)
• Vm 代表軸突的細胞膜電位差，而 EK 、 ECl 、
和 ENa 代表由 Nernst 方程式所推導出來的
細胞膜電位 [1] ，所謂的 Nernst 方程式定
義如下： RT Ion
Ek = ( ) ln( o
)
zF Ioni
其中， R 是氣體常數， T 為絕對溫度， z
為價電子數， F 是法拉第常數， Iono 是細胞
膜外的離子濃度， Ioni 是細胞膜內的離子濃度。
66. 1.4 類神經元的模型 (4)
我們可以用以下的數學式子來描述類神經元的輸入輸出關係：
p
u j = ∑ w ji xi (1.3)
i =1
y j = ϕ (u j − θ j ) (1.4)
其中
w ji 代表第 i 維輸入至第 j 個類神經元的鍵結值；
θ j 代表這個類神經元的閥值；
x = ( x1 ,, x p )T 代表 p 維的輸入；
u j 代表第 j 個類神經元所獲得的整體輸入量，
其物理意義是代表位於軸突丘的細胞膜電位；
ϕ ( ⋅) 代表活化函數；
y j 則代表了類神經元的輸出值，也就是脈衝頻率。
67. 1.4 類神經元的模型 (5)
如果我們用 w j 0 代表 θ j，則上述式子可改寫為：
p
(
v j = ∑ w ji xi = wT x
i =0
j ) (1.5)
及
y j = ϕ (v j ) (1.6)
w j = [ w j 0 , w j1 ,, w jp ]T
其中 和
。 x = [ −1, x1 , x2 ,, x p ]T
68. 1.4 類神經元的模型 (6)
所用的活化函數型式，常見的有以下四種型式：
• 嚴格限制函數 (hard limiter or threshold function) ：
1 if v ≥ 0
ϕ (v ) =
0 if v < 0
圖 1.12 ：嚴格限制函數。
• 區域線性函數 (piecewise linear function) ：
1 if v > v1
ϕ (v) = cv if v 2 ≤ v ≤ v1
0 if v < v 2
圖 1.13 ：區域線性函數。
79. 1.7 類神經網路的學習規則
(3)
我們以數學式來描述通用型的學習規則
w ji ( n + 1) = w ji ( n ) + ∆w ji ( n )
其中
w ji (n ) 及 w ji ( n + 1) 分別代表原先的及調整後的鍵結
值；
∆w ji (n )
代表此類神經元受到刺激後，為了達成學習
效果，所必須採取的改變量。
∆w ji (n ) 此改變量 ，通常是 (1) 當時的輸入xi (n ) 、
(2) 原先的鍵結值w ji (n ) 、及 (3) 期望的輸出值
(desired output) di ( 若屬於非監督式學習，則無此項 )
的某種函數關係 。
80. 1.7.1 Hebbian 學習規則
• 神經心理學家 (neuropsychologist) Hebb 在他的
一本書中寫著 [7]
當神經元 A 的軸突與神經元 B 之距離，近到足以激發它的地
步時，若神經元 A 重複地或持續地扮演激發神經元 B 的角色，
則某種增長現象或新陳代謝的改變，會發生在其中之一或兩個神
經元的細胞上，以至於神經元 A 能否激發神經元 B 的有效性會
被提高。
• 因此我們得到以下的學習規則：
w ji ( n + 1) = w ji ( n ) + F ( y j ( n ), xi ( n ) ) (1.14)
這種 Hebbian 學習規則屬於前饋 (feedforward) 式的非監督
學習規則。以下是最常使用的型式：
w ji ( n + 1) = w ji ( n ) + ηy j ( n ) xi ( n ) (1.15)
81. 1.7.2 錯誤更正法則 (1)
• 錯誤更正法則的基本概念是，若類神經元的真實輸出值
y j (n ) 與期望的目標值 d j (n ) 不同時，則兩者之差，定
義為誤差信號 ：
e j (n ) = d j (n) − y j (n)
• 我們可以選擇一特定的「代價函數」 (cost function) 來
反應出誤差信號的物理量；
• 錯誤更正法則的終極目標，就是調整鍵結值使得代價函
數值越來越小，亦即使類神經元的真實輸出值，越接近
目標值越好，一般都採用梯度坡降法 (gradient decent
method) 來搜尋一組鍵結值，使得代價函數達到最小。
82. 1.7.2 錯誤更正法則 (2)
一、 Windrow-Hoff 學習法
代價函數定義為：
E = ∑ e j (n ) = ∑ ( d j (n ) − v j (n ) ) 2
1
j 2 j
(1.18)
1
( T
= ∑ d j (n) − w j (n) x(n)
2 j
2
)
因此根據梯度坡降法可得：
∂E
∆ w j ( n ) = −η
∂ w j (n )
( )
= η d j ( n ) − wT ( n ) x ( n ) x ( n )
j (1.19)
= η ( d j (n ) − v j (n ) ) x(n )
此學習規則，有時候亦被稱為最小均方演算法 (least square error
algorithm) 。
83. 1.7.2 錯誤更正法則 (3)
二、 Delta 學習法
使用此種學習法的類神經網路，其活化函數都是採用連續且可微分
的函數型式，而代價函數則定義為：
E = ∑ e j (n ) = ∑ ( d j (n ) − y j (n ) ) 2
1
(1.20)
j 2 j
因此根據梯度坡降法可得：
∂E
∆ w j ( n ) = −η (1.21)
∂ w j (n )
= η ( d j (n) − o j (n) )ϕ ' ( v j ( n) ) x(n)
實際上，若 ϕ ( v j (n) ) = v j (n) 時，則 Widrow-Hoff 學習可視為
Delta 學習法的一項特例。
84. 1.7.3 競爭式學習法
• 競爭式學習法有時又稱為「贏者全拿」 (winner-take-
all) 學習法。
步驟一：得勝者之篩選
假設在此網路中有 K 個類神經元，如果
wT ( n ) x ( n ) =
k max wT ( n ) x ( n )
j
(1.22)
j =1, 2,, K
那麼第 k 個類神經元為得勝者。
步驟二：鍵結值之調整
η ( x ( n ) − w j ( n ) ) if j = k
∆ w j (n ) = (1.23)
0 if j ≠ k
108. 網路演算法
• 訓練範例的輸入處理單元的輸入值 {X} ，計
算隱藏層隱藏處理單元的輸出值 {H} 如下：
H k = f (net k ) = f (∑ Wik X i − θ k )
• 隱藏層隱藏處理單元的輸出值 {H} ，計算輸
出層處理單元的推論輸出值 {Y} 如下：
Y j = f (net j ) = f (∑ Wkj H k − θ j )
109. 網路演算法
• 誤差函數 :
E = (1 / 2)∑(T j − Y j ) 2
• 最陡坡降法 (the gradient steepest descent method):
∆W = −η ∂E
∂W
: 學習速率，控制每次權值修改的步幅
η
117. Architecture of DSNN
Hidden
Neurons
Input
Output
Surface
Surface
Ld
Virtual 3-D Cube Space
118. Architecture of DSNN
Input Hidden Output
Layer Layer Layer
N1
wX1,1 w 2,1 w1,Y1
N2 w1,4
wX1,2 w 2,4
X1 N4 w4,Y1 Y1
N3
X2 Y2
N5
N6
...
...
N8
N7
Xx wXx,1 N9 Yy
w9,n wn,Yy
Nn
w Xx,n
119. The Wavelet-based Neural Network Classifier
Disturbance
Waveform
Estimate Amplitude &
Subtract Disturbance Waveform by the
Estimated Perfect Waveform
Detection of Amplitude Irregular
Disturbances
Wavelet Transforms
Detection of Impulsive Transient
Disturbances
Dynamic Structural Neural Networks
&
Detection of Harmonic Distortion and
Voltage Flicker
Output Final Result of Detection
120. The Wavelet
• This work utilizes the hierarchical wavelet
transform technique to extracting the time
and frequency information by the
Daubechies wavelet transform with the 16-
coefficient filter.
The four-scale hierarchical decomposition of G0(n).
121. The Neural Network Classifier
• The detection and extraction of the features from
the wavelet transform is then fed into the DSNN
for identifying the types of PQ variations.
• The inputs of the DSNN are the standard
derivations of the wavelet transform coefficients of
each level of hierarchical wavelet transform.
• The outputs of the DSNN are the types of
disturbances along with its critical value.
122. Wavelet Transform
• Let f(t) denotes the original time domain signal.
The continuous wavelet transform is defined as
follow: t −b
1 ∞
CWT f (a, b) =
a ∫
−∞
f (t )ψ dt
a
where ψ(t) represents the mother wavelet, a is the
scale parameter, and b is the time-shift parameter.
123. Wavelet Transform
• The mother wavelet ψ(t) is a compact support function that
must satisfies the following condition:
∞
∫−∞
ψ (t )dt = 0
• In order to satisfying the equation above, a wavelet is
constructed so that it has a higher order of vanishing
moments. A wavelet that has vanishing moments of order N
if
∞
∫
−∞
t pψ (t )dt = 0 for p = 0, 1, …, N-1
124. Architecture of DSNN
• The distinct features of the DSNN:
tune itself and adjust its learning capacity.
• The structure of the hidden layer of the
network must be reconfigurable during the
training process.
125. Architecture of DSNN
• The length of the edge of the virtual 3-D cube space is
defined as follows:
Ld = ρ × (10 × N )
where N is the total initial number of neurons of the network,
(10×N)3 is the space used for deploying the initial neurons, and
ρ is the space reserve factor for preserving extra space to place the new
generating neurons.
• Typically, ρ is predetermined within
an interval from 1.5 to 3, or the
interval can be set according to
experiments.
126. Model of Neurons
Model of an input Input Hidden
vector feeds into the Neuron Neuron
hidden neurons Input Vector yi yo
i w io o
bo
Hidden Hidden
Neuron i Neuron j
Model of signals yi yj
propagation between i wij o
two hidden neurons.
bj
127. Model of Neurons
The output of the hidden neuron is given by
yo (n) = ϕo ∑ wio (n) ⋅ yi ( n) + bo (n)
i∈C
where yo is the output of neuron o,
C denotes the index of the input neurons,
n is the iteration number of the process,
wio is the synaptic weight between neuron i and neuron o,
yi is the input of neuron i,
bo is the bias of neuron o,
φo is the activation function.
128. Supervised Training of Output
Neurons
The output error is defined by following
eo (n) = d o (n) − yo (n)
where eo is the error of output neuron o,
do is the target value of output neuron o, and
yo is the actual output value of output neuron
o.
129. Supervised Training of Output
Neurons
• The correction Δwio(n) can be calculated by:
∆wio (n) = l ⋅ η ⋅ eo (n) ⋅ y o (n)
1 if ∆ yo ( n ) > 0
l=
−1 if ∆ yo ( n ) < 0
where Δwio(n) is the weighting correction value of
the connection from original terminal neuron i to
destination terminal neuron o.
η is the learning rate,
l is the refine direction indicator used for deciding
the direction for weighting tuning.
130. Supervised Training of Output
Neurons
• The correction Δbo(n) is defined as:
∆bo ( n) = η ⋅ eo (n)
where Δbo is the bias correction value of the output neuron
o.
• The weighting and bias are adjusted by following
formulas: w (n + 1) = w (n) + ∆w
io io io
bo (n + 1) = bo (n) + ∆bo
where wio(n+1) and bo(n+1) are the refined weighting and
bias of output neuron o.
131. g j ( n) = η ⋅ si ( n) ⋅ y i ( n)
Supervised Training of Hidden
Neurons
• The updating the hidden neuron:
g i (n) = η ⋅ eo (n) ⋅ yo (n)
where gi(n) is the turning momentum of the hidden neurons to
the output neurons.
• The momentum of the hidden neuron i is defined as:
g i ( n)
si (n) = g j (n) = η ⋅ si (n) ⋅ yi (n)
Ci
where si(n) is the momentum of the hidden neuron i,
gj(n) is the turning momentum of the hidden neuron j
connected to the hidden neuron i,
132. Supervised Training of Hidden
Neurons
• The correction weighting is
∆ w ji (n) = l ⋅ η ⋅ si (n) ⋅ yi (n)
1 if ∆ yi (n) > 0
l=
− 1 if ∆ yi (n) < 0
where Δwji(n) is the weighting correction value of the connection from
original neuron j to destination terminal neuron i.
• The correction to bi(n) is
∆bi (n) = η ⋅ si ( n)
where Δbi is the bias correction value of the hidden neuron i.
133. Supervised Training of Hidden
Neurons
• The function of tuning indicator for backward
neurons is described as below.
g j (n) = η ⋅ si (n) | yi (n) |
where gj(n) is the tuning indicator for hidden
neuron j that connected to hidden neuron i.
134. Flow chart of tuning of weighting and
bias of the output neuron
do Target
Output
Neuron Vector
yo
wi,o o Error
bo eo
Δwi,o
Tw Δyo
Delta Delay
Δb o
Tb
135. Dynamic Structure
• creating new neurons and neural connections.
• The restructuring algorithm can produce or prune
neurons and the connections between the neurons
in an unsupervised manner.
y1 Grow Direction
N1
1
N2 Wnf1
2
y3 Wnf2 yn
N3 Nn
Wnf3
yk 3
Nk
136. Dynamic Structure
• The correction of the coordinate of the free connectors can
be formulated as follow:
gj
∆ ( x fn , y fn , z fn ) = ∑ D ⋅
L
( x , y ,z )
j j j
j j
1 if attraction
D=
− 1 if repulsion
where Δ(xfn,yfn,zfn) is the correction of coordinate of the free
connector, Lj is the distance between the free connector and
the scanned neuron, and (xj,yj,zj) is the coordinate of the
scanned neuron.
137. Creating New Neurons
• The probability P of a new neuron being created is
given by:
N max_ h − N h
P = ∑ ei ⋅
N
i max_ h
where ei is the error of the output neuron i,
Nh is the current number of the hidden neurons in the
middle layer.
Nmax_h is the maximum number of neurons that can be
created in the virtual cube space.
138. Block Diagram of the DSNN
1 to 4 scale Wavelet Coefficients
Input Disturbance Disturbance Standard D1 D2 D3 D4 S4
Types and Waveform Derivation
Conditions
Impulse Impulsive Impulsive Transient
Yes
Detector Transient? Disturbance
RMS Voltage
Calculation No
Impulsive Transient Filter
Estimate the Amplitude
Estimated
of the Fundamental
Amplitude
System Frequency 1 to 4 scale Wavelet Coefficients
Standard
Derivation
Generator D1 D2 D3 D4 S4
Perfect Waveform Sag? Sag Disturbance or
Swell? Yes Swell Disturbance or
with the Estimated Interrupt? Interrupt Disturbance
Amplitude
Neural
No Dynamic Structural
Weighting
and Bias Neural Networks
Waveform
Subtraction
Harmonic Distortions
Daubechies-8 Harmonic?
Wavelet Yes and/or
Wavelet Flocker?
Transform Voltage Flicker
Coefficients
No
End
139. Amplitude Estimator
• The estimating RMS value of voltages can be calculated by
the following equation:
M
∑ ( f (t ) )
2
RMS = t =1
M
where f(t) represents the value of the voltage sampled from the
disturbance waveform, and
M is the total amount of sampling points.
• In order to reduce the computational complexity, the RMS
value of f(t) can be approximated by
M
∑ f (t )
RMS A = t =1
M
140. Amplitude Estimator
• Then the amplitude of the fundamental voltage can be
predicted as
AmpEst = 2 × RMS
AmpEst_A = 1.5725 × RMS A
where AmpEst is the estimated amplitude of the fundamental voltage
obtained from the RMS value, and
AmpEst_A is the approximately estimated amplitude of the fundamental
voltage obtained from the approximately RMS value RMSA.
141. Wavelet Transform
• According to the estimated amplitude AmpEst produced by
the amplitude estimator, a perfect sinusoidal waveform
with the amplitude of AmpEst can be generated.
• And, subtract the generated perfect sinusoidal waveform
from the original measured waveform we have the
disturbance signal. Then, the wavelet transform is applied
to the extracted disturbance signal for analysis.
142. Wavelet Transform
• The disturbance features reside in four scales of the
decomposed high-pass and low-pass signals.
• The first scale of high-pass signal is most sensitive than
other scales of decomposed signals because it contains the
signals with high frequency band.
• Therefore, it is employed for extracting the features of the
impulsive transient disturbance within the disturbance
waveform.
143. Feature Extraction of Impulsive
Transient
An example of impulsive transient disturbance.
results of wavelet analysis in high-pass band and low-pass band, respectively.
144. Feature Extraction of Impulsive
Transient
• The values of mean and standard derivation of the signal in
high-pass band (D1) are calculated as follows to identify the
impulse disturbance.
M /2 M /2
∑ D (t ) ∑ ( D1 (t ) − µ )
2
1
µ1 = t =1
ρ1 = t =1
M /2 M /2
where μ1 and ρ1 are the mean and standard derivation of the signal in
high-pass band (D1), respectively.
• The impulsive transient disturbance event is identified
according to the following rule:
∀ t , ∋ D1 (t ) ≥ µ + 1.25ρ
145. Impulsive Transient Removal
• However, the impulsive transient disturbance may contain
multiple frequency components, which could make the
decomposed signals contain irregular disturbance.
• Hence, the impulsive transient components must be
removed from all scales of the decomposed signals, after
the impulsive transient disturbance has been identified.
• Then, the values of mean and standard derivation on each
scale of the decomposed signals D1, D2, D3, D4 and S4 are
calculated again for identifying other disturbance.
• This procedure can prevent the following DSNN classifier
misclassifying.
146. Example of hybrid of
Harmonic and Flicker
Example waveform of combining several harmonic distortions and voltage flicker
Decomposed signals D1, D2, D3, D4 and S4 form the 4-scale wavelet transform
147. Generating Waveform Data
(Training/Testing Dataset)
Number of
Condition Name Disturbances Options
Included
Single Disturbance
1 all type of PQ disturbances
Waveform
One is randomly chosen from
Dual Disturbances Type A, B, or C,
2
Waveform the other is randomly chosen from
Type D, E, or F.
One of them is randomly chosen from
Multiple Disturbances Type A, Type B or Type C and
2~4 the others are randomly chosen from
Waveform
Type D, Type E or Type F
148. Types of PQ Disturbances
Types Name RMS (pu) Duration
A Momentary Swell Disturbance 1.1~1.4 30 cycles~3 sec
B Momentary Sag Disturbance 0.1~0.9 30 cycles~3 sec
Momentary Interrupt
C Disturbance
<0.1 0.5 cycles~3 sec
Impulsive Transient Microseconds to
D Disturbance milliseconds
E Harmonic Distortion 0~0.2
F Voltage Flicker 0.001~0.07
149. Multiple PQ Disturbances
• From the field measurements, usually there existed
multiple types of disturbances in a PQ event.
• Recognizing a waveform that consists of multiple
disturbances is far more complex than that consists of
single disturbance.
• This work develops a new method that is capable of
recognizing several typical types of disturbances existing in
a measured waveform and identifying their critical value.
150. Multiple PQ Disturbances
Hybrid of voltage flicker and impulsive transient disturbance.
Hybrid of momentary sag disturbance and voltage flicker.
Hybrid of momentary sag disturbance and high-frequency harmonic distortions.
151. Examples
Single Disturbance
Dual Disturbances
Multiple Disturbances
152. Experimental Results
(Parameters)
• This section presents the classification results of 6
types of disturbances under 3 kinds of conditions.
• The sampling rate of the voltage waveform is 30
points/per cycle, the fundamental frequency is 60
Hz and the amplitude is 1 pu.
• The parameters of the proposed DSNN are that
space preservation factor ρ is set as 1.5, the number
of initially generated neurons is 50, and active
function φo of neurons is the hyperbolic tangent
function.
153. Experimental Results
(Parameters)
• The disturbance waveforms are randomly
generated according to the definition of IEEE Std.
1159.
• The minimum amplitudes of harmonic distortions
and voltage flicker are both 0.01 pu.
• There are 3000 randomly generated waveforms for
three kinds of PQ variations. 250 waveforms of
each kind of PQ variations and 100 normal
waveforms are utilized for training the DSNN.
