1. Ch01_Introduction to Pattern Recognition
(modified from 패턴인식개론/한학용)

2. Contents
 01_ Philosophical Debates on AI
 02_ Pattern Recognition (PR)
 03_ Features and Patterns
 04_ Components of PR and Design Cycle
 05_ Category of PR and Classifiers
 06_ Performance Evaluation of PR Algorithms
 07_ Approaches of PR and its Application Areas
 08_ Example of PR Applications
2

3. 01_Philosophical Debates on AI
 Questions
 Is computer merely a calculating machine?
 Can computer think and understand languages like human?
3

4. 01_Philosophical Debates on AI
 Positive opinions on the possibility of AI
A.M. Turing(1912~1954) Imitation Game
 Negative opinions on the possibility of AI
John Searle(1932~ ) Chinese Room Arguments
4

5. 02_Definition of Pattern Recognition
 What is PR?
 An area of AI that deals with the problems to make computable machines
(Turing Machines) to recognize certain objects
Cognitive
Science
AI
PR
5

6. 03_Features and Patterns
 What is feature?
 Discernible aspects, qualities, characteristics that a certain object has
 What is pattern?
 A set of traits or features of individual objects
6

7. 03_Features and Patterns
 Easy features and difficult features
 Categories of patterns
7

8. 04_Components of PR and Design Cycle
 Components of PR System and its Process
8

9. 04_ Components of PR and Design Cycle
 Design steps of PR system
 Step 1 : Data gathering
 Most time-consuming tedious process in PR tasks
 Necessary step to ensure stable PR performance
 For stable performance, we need to consider how many samples are needed before
the gathering.
 Step 2 : Feature selection
 Essential part regarding PR system’s performance
 We need to decide what features to choose through sufficient prior analysis on the
object patters.
 Step 3 : Model selection
 To decide what approach (model and algorithm) is to be constructed and applied
 Need prior knowledge on the features
 Need to set up parameters for the model according to the approach
9

10. 04_ Components of PR and Design Cycle
 Step 4 : Learning
 Using the feature sets extracted from the collected data and chosen models, the
learning algorithm generates or fills up the model (or hypothesis, classifier)
 According to the methods, there are supervised learning, unsupervised learning and
reinforcement learning.
 Step 5 : Recognition
 Given a new feature set, the generated hypothesis decide a class or category that
the feature set belongs to.
10

11. 05_ Category of PR and Classifiers
 Categories of problems
 Classification
 In classification problem, the system needs to output one label in a set of finite
number of labels.
 Regression
 Generalized version of classification
 Through regression, the PR system will return a real value score (usually between 0
and 1)
 Clustering
 The problem of organizing a small number of multiple groups from a certain set
 The output of clustering system is a set of pairs (example and its class).
 The clustering can be processed in an hierarchical manner such as in phylogenetic
tree.
 Description
 The problem of expressing an object using a set of a prototype or primitive terms
11

12. 05_ Category of PR and Classifiers
 Classifier
 Most classification task in PR is done by classifiers
 Classification is to partitioning a feature space composed of feature vectors into
decision regions of nominal classes.
 We call the boundaries of the regions as decision boundaries
 Classification of a feature vector x is to decide what decision region the feature
vector belongs to, and to assign x to the class that represents the region
12

13. 05_ Category of PR and Classifiers
 Classifier can be represented as a set of discriminant functions
 ∀j=i , if 𝑔 𝑖 𝑥 > 𝑔 𝑗 𝑥 , then we decide that the feature vector 𝑥 ∈ class 𝜔 𝑖
13

14. 06_ Performance Evaluation of PR Algorithms
 Confusion Matrix
Actual Positive Actual Negative
Predicted Positive TP FP
Predicted Negative FN TN
TP
 Recall rate =
TP+FN
TP
 Precision =
TP+FP
TP
 True Positive Rate (TPR) =
TP+FN
FP
 False Positive Rate (FPR) =
FP+TN
14

15. 06_ Performance Evaluation of PR Algorithms
 Receiver operating characteristic (ROC) Curve
15

16. 06_ Performance Evaluation of PR Algorithms
 AUROC
 Ares under the region of ROC Curve
 Closer the curve to top-left corner, more accurate the recognition algorithm
 The performance can be evaluated by the amount of AUROC
16

17. 07_ Approaches of PR and its Application Areas
 Approaches of PR
 Template matching
 Oldest and easiest
 First, prepare the template for the object to compare.
 Normalize the pattern to recognize for matching it with the template.
 And calculate similarity value such as cross-correlation or distance to perform the
recognition
 Most important task is to prepare the most general template that explains all the samples
in a certain category.
 Fast running time, but weak in variation of features
 Statistical approaches
 Decide the class of unknown pattern bases on decision boundaries of pattern sets.
 Each of the pattern sets represent a certain class.
 The statistical model of the patterns is a probability density function 𝑃 𝑥|𝑐 𝑖 .
 Learning is a process of creating a probability density function and calculating its
parameters for each class
 Neural networks
 Model the relation of connection and integration of the biological neurons
 Calculate the response process of neural network for input stimulus
 Classify patterns based on the responses
17

18. 07_ Approaches of PR and its Application Areas
 Knowledge of the patterns is stored as weights that represent the connection
strength of synapse.
 Learning is performed similar to biological ways, but the learning process is not a
serial algorithm.
 The learned knowledge is considered as a black box.
 Minimal need for prior knowledge.
 With sufficient number of neurons, theoretically any complicated decision
boundaries can be constructed, so this approach is very attractive.
 Structural approaches
 Instead of quantitative features, we consider the relationship among the basic
prototypes what construct the pattern.
 Examples: Character, Fingerprint, Chromosome
18

19. 07_ Approaches of PR and its Application Areas
 Approaches of PR
19

20. 07_ Approaches of PR and its Application Areas
 Applications of PR
 Character recognition
 Convert a scanned text image into character codes which can be edited in a
computer
 Mail classification, Handwriting recognition, Check and banknote recognition,
License plate recognition
 Biological recognition and human behavioral pattern recognition
 Voice recognition, fingerprint recognition, face recognition, DNA mapping, walking
pattern analysis and classification, utterance habit analysis and classification
 Diagnostic systems
 Car malfunction, medical diagnostics, EEG, ECG signal analysis and classification, X-
Ray image pattern recognition
20

21. 07_ Approaches of PR and its Application Areas
 Prediction system
 Weather forecasting based on satellite data, earthquake pattern analysis and
earthquake prediction, stock price prediction, etc.
 Security and military area
 Intrusion detection based on network traffic pattern analysis, security screening
system, search and attack of terrorist camp and targets using satellite terrain
image analysis, radar signal classification, Identification Friend or Foe (IFF)
21

22. 07_ Approaches of PR and its Application Areas
Related Areas Application Areas
•Adaptive signal processing •Image
•Machine learning processing/segmentation
•Artificial Neural networks •Computer Vision
•Robotics and Vision •Speech recognition
•Cognitive science •Automatic target recognition
•Mathematical Statistics •Optical character recognition
•Nonlinear optimization •Seismic Analysis
•Exploratory Data analysis •Man-machine interaction
•Fuzzy and Genetic System •Bio recognition (fingerprint,
•Detection and Estimation vein, iris)
Theory •Industrial inspection
•Formal language •Financial forecast
•Structural modeling •Medical analysis
•Biological cybernetics •ECG signal analysis
•Computational neuroscience
22

23. 08_ Example of PR Applications
 Simple English character recognition system
 feature V : # of vertical lines
 feature H : # of horizontal lines
 feature O : # of slopes
 feature C : # of curves
Feature
Character
V H O C
L 1 1 0 0
P 1 0 0 1
O 0 0 0 1
E 1 3 0 0
Q 0 0 1 1
23

24. 08_ Example of PR Applications
 Automatic fish classification (Sea Bass or Salmon)
 A: Conveyor belt for fish
 B: Conveyor belt for classified fish
 C : Robot arm for grabbing fish
 D: Machine vision system with CCD camera
 E : Computer that analyze fish image and control the robot arm
24

25. 08_ Example of PR Applications
 Automatic fish classification
 Assume that fish is either salmon or sea bass
 Using machine vision system for acquiring new fish image
 Normalize the intensities of new fish image using image processing algorithm
 Segment fish from the background in the image processing process
 Using the prior knowledge that sea bass is bigger than salmon, extract features in
the image to measure the length of the new fish
 From the training samples of the two fish categories, calculate the distribution of
the length, and decide the threshold of decision boundary that minimize the
classification error
Accuracy : 60%
25

26. 08_ Example of PR Applications
 Adding features for enhancing recognition rate
 The accuracy should be over 95% for stable pattern recognition system
 We find that average intensity level is a good feature.
26

27. 08_ Example of PR Applications
 Enhancing the recognition rate
 We generate 2 dimensional feature vector with length and average intensity.
 Using a simple linear discriminant function, we enhance the recognition rate.
Accuracy : 95.7%
27

28. 08_ Example of PR Applications
 Cost vs. Classification Rate
 To minimize the cost, we adjust the decision boundary
28

29. 08_Example of PR Applications
 Generalization problem
 Using neural network, the performance can be enhanced to 99.9975%
 Is this a good result?
29