2. Introduction
• Motivated by the possibility of creating an artificial
computing network similar to the brain and nerve cells in
our body.
• These networks are computing systems composed of a
number of highly interconnected layers of simple neuron-
like processing elements.
• The entire network collectively performs computations,
with the knowledge represented as distributed patterns
of activity all over processing elements.
• The collective activities result in a high degree of
parallelism, which enables the network to solve complex
problems.
• The distributed representation leads to greater fault
tolerance and to graceful degradation when problems
3. • They have the capability of simulating non-linear
patterns.
• Their advantage relies on the fact that they demand less
time for development than traditional mathematical
models.
4. ANN Architecture
There are three layers:-
1. Input Layer:- The first layer of an ANN that receives the
input information in the form of various texts, numbers,
audio files, image pixels, etc.
2. Hidden Layer:- In the middle of the ANN model are
the hidden layers. There can be a single hidden layer or
multiple hidden layers. These hidden layers perform
various types of mathematical computation on the input
data and recognize the patterns that are part of.
3. Output Layer:- In the output layer, we obtain the result
that we obtain through rigorous computations performed
by the middle layer.
5.
6. Types of ANN
1. Feedforward ANN:-
• The flow of information takes place only in one direction.
• no feedback loops
• mostly used in supervised learning for instances such
as classification, image recognition etc.
• used in cases where the data is not sequential in nature.
1. Feedback ANN:-
• the feedback loops are a part of it.
• Such type of neural networks are mainly for memory
retention such as in the case of recurrent neural
networks.
• These types of networks are most suited for areas where
7.
8. Back-Propagation
• Back-propagation is used to train the neural network of
the chain rule method.
• After each feed-forward passes through a network,
this algorithm does the backward pass to adjust the
model's parameters based on weights and biases.
• It is a process in which the internal parameters to the
network, the weighing factors W, and bias B, are
adjusted.
• The bias is an adjusting parameter, which reduces the
error in the system. Values of these parameters are
calculated using multiple-variable optimization
algorithms.
9. • the change that has to be made to the weighing factors
and bias is calculated using the derivative vector D and
the input data to that layer according to the following
rule:
Wnew = Wold + lrDvT
Bnew = Bold + lrD
where Ir is the learning rate.
10. Back-Propagation Pseudo-code
• Initialize the weights and offsets.
• Set all of them to low random values. Present inputs and
desired outputs. This is done by presenting a continuous-
valued input vector and specifying the desired outputs. If the
network is used as a classifier, all desired outputs are set to 1.
The input could be new on each turn or one could use a cyclic
pattern to train.
• Calculate the actual outputs using the sigmoidal non-linearity.
• Adapt weights using a recursive algorithm starting at the
output nodes and working back.
• Adjust the weights using the formula
Wij(t + 1) = Wij(t) +ηδjxt’
where Wij is the weight from node i to node j at time t, η is the
gain term, and δj is the error term for node j. If node j is an output
node, then
δ = y (1- y )(d - y )
11. where dj is the desired output of nodej and yj is the actual
output. If node j is an internal hidden node, then
where k is the number of overall nodes in the layers above
node j. If a momentum term α is added, the network
sometimes becomes faster and the weight changes are
smoothed by:-
• Repeat Step 2
• Stop
12. Network Training
1. Supervised learning :-
• An input stimulus is applied to the network, which results
in an output response.
• This is compared with the desired target response and
an error signal is generated.
• The learning in back-propagation networks is
supervised.
2. Unsupervised learning: -
• During training, the network receives different input
excitations and arbitrarily organizes the patterns into
categories.
• When a stimulus is later applied, the network indicates
the class to which it belongs and an entirely new class of
stimuli is generated.
13. 3. Reinforced learning :-
• In this case, the network indicates whether the output is
matching with the target or not-a pass or fail indication.
In other words, the generated signal is binary. This kind
of learning is used in applications such as fault
diagnosis.
14. Modes of Training
• Pattern mode :- Consider a training set having N
patterns. The first pattern is presented to the network,
and the whole sequence of forward and backward
computations is performed, resulting in weight
adjustment. Then the second pattern is presented and
weights updated and so on until the Nth pattern.
• Batch mode:- Here, weight updating is done after the
presentation of one full epoch. One complete
presentation of the entire training set is called an epoch.
