1. The document describes a proposed fuzzy rule-based classifier system called Fuzzy-UCS that is derived from the Michigan-style learning classifier system UCS.
2. Fuzzy-UCS uses fuzzy rule representations with linguistic terms and membership functions instead of interval-based rules. This aims to improve interpretability over UCS while maintaining similar performance.
3. An experimental methodology is outlined to evaluate Fuzzy-UCS's performance compared to UCS, other machine learning techniques, and other fuzzy learners on classification tasks.
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
JAEM'2007: Aprendizaje Supervisado de Reglas Difusas mediante un Sistema Clasificador Evolutivo Estilo Michigan
1. Aprendizaje Supervisado de Reglas
Difusas mediante un Sistema
Clasificador Evolutivo Estilo Michigan
Albert Orriols-Puig1,2
Orriols Puig
Jorge Casillas2
Ester Bernadó-Mansilla1
1Grup de Recerca en Sistemes Intel·ligents
Enginyeria i Arquitectura La Salle, Universitat Ramon Llull
2Dept. de Ciencias de la Computación e IA
Universidad de Granada
2. Motivation
Michigan-style LCSs for supervised learning. Eg. UCS
– Evolve online highly accurate models
– Competitive to the most-used machine learning techniques
• (Bernadó et al, 03; Wilson, 02; Bacardit & Butz, 04; Butz, 06; Orriols & Bernadó, 07)
Main weakness: Intepretability of the rule sets
– Continuous attributes represented with intervals: [ i, ui] . Semantic-
p [l
free variables
– Number of rules or classifiers
• Reduction schemes
(Wilson, 02; Fu & Davis, 02; Dixon et al., 03, Orriols & Bernadó, 2005)
Enginyeria i Arquitectura la Salle Slide 2
GRSI
3. Motivation
Jorge’s Proposal:
– Let’s “fuzzify” UCS
fuzzify”
• Change the rule representation to fuzzy rules
Framework on Michigan-style Learning Fuzzy-Classifier
Systems (LFCS)
– (Valenzuela-Radón, 91 & 98)
– (Parodi & Bonelli, 93)
– (Furuhashi, Nakaoka & Uchikawa, 94)
– (Velasco, 98)
– (Ishibuchi, Nakashima & Murata, 99 & 05): First LFCS for pattern classification
– (Casillas, Carse & Bull, 07) Fuzzy-XCS
Enginyeria i Arquitectura la Salle Slide 3
GRSI
4. Aim
Propose Fuzzy-UCS
– Accuracy based Michigan-style LFCS
Accuracy-based Michigan style
– Supervised learning scheme
– Derived from UCS (Bernadó & Garrell, 2003)
• Introduction of a linguistic fuzzy representation
• Modification of all operators that deal with rules
– We expect:
• Achieve similar performance than UCS
• Higher interpretability
– Plus new opportunities:
• Mine in uncertain environments
Enginyeria i Arquitectura la Salle Slide 4
GRSI
5. Outline
1. Description of Fuzzy-UCS
1D ii fF UCS
2.
2 Experimental Methodology
3. Results
4. Conclusions
Enginyeria i Arquitectura la Salle Slide 5
GRSI
6. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of UCS
p 3. Results
4. Conclusions
4C li
Michigan-style LCS’s (Holland, 1975):
– Derived from XCS (Wilson 1995) a reinforcement learning
(Wilson, 1995),
method.
– Designed specifically for supervised learning
Rule representation:
– C ti
Continuous variables represented as i t
intervals: [li, ui]
i bl td l
– Eg:
IF x1 Є [l1, u1] ^ x2 Є [l2, u2] … ^ xn Є[ln, nn] THEN class1
– Matching instance e: for all ei: li ≤ ei ≤ ui
– Set of parameters: Accuracy, Fitness, Numerosity, Experience, Correct set
size
Enginyeria i Arquitectura la Salle Slide 6
GRSI
7. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of UCS
p 3. Results
4. Conclusions
4C li
Stream of
Environment examples
Match Set
M t h S t [M]
Problem instance
P bl it
+
output class acc F num cs ts exp
1C A
acc F num cs ts exp
3C A
Population [P] acc F num cs ts exp
5C A
acc F num cs ts exp
6C A
…
acc F num cs ts exp
1C A
acc F num cs ts exp
2C A
acc F num cs ts exp
3C A
correct set
acc F num cs ts exp
4C A Classifier
generation
acc F num cs ts exp
5C A
Parameters
Match set
acc F num cs ts exp
6C A
Update
generation
…
Correct Set [C]
3 C A acc F num cs ts exp
Deletion # Correct
Selection, Reproduction,
acc =
6 C A acc F num cs ts exp
mutation
Experience
p
…
If there are no classfiers in
Genetic Algorithm Fitness = accν
[C], covering is triggered
Enginyeria i Arquitectura la Salle Slide 7
GRSI
8. Description of Fuzzy-UCS
p y
Describe the different components
1. Rule representation and matching
2. Learning interaction
3. Discovery component
3 Di t
4. Fuzzy-UCS in test mode
Enginyeria i Arquitectura la Salle Slide 8
GRSI
9. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of Fuzzy-UCS
p y 3. Results
4C li
4. Conclusions
Rule representation
– Linguistic fuzzy rules
– E.g.: IF x1 is A1 and x2 is A2 … and xn is An THEN class1
Disjunction of linguistic
fuzzy terms
– All variables share th same semantics
i bl h the ti
– Example: Ai = {small, medium, large}
IF x1 is small and x2 is medium or large THEN class1
– Codification:
IF [100 | 011] THEN class1
Enginyeria i Arquitectura la Salle Slide 9
GRSI
10. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of Fuzzy-UCS
p y 3. Results
4C li
4. Conclusions
How do we know if a given input is small, medium or large?
g p , g
– Each linguistic term defined by a membership function
Belongs to medium with a degree of 0 8
0.8
Belongs to small with a degree of 0 2
0.2
ei
Attribute value Triangular-shaped
membership functions
Enginyeria i Arquitectura la Salle Slide 10
GRSI
11. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of Fuzzy-UCS
p y 3. Results
4C li
4. Conclusions
Matching degree uAk(e)
gg () [,]
[0,1]
k: IF x1 is small and x2 is medium or large THEN class1
Example: (e1, e2)
0.8
08
0.2 0.2
e1 e2
T-conorm: bounded sum
max ( 1, 0.8 + 0.2) = 1
T-norm: product
uAk(e) = 1 * 0.2 = 0.2
Enginyeria i Arquitectura la Salle Slide 11
GRSI
12. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of Fuzzy-UCS
p y 3. Results
4. Conclusions
4C li
The role of matching changes:
• UCS: A rule matches or not an example (binary function)
• Fuzzy-UCS: A rule matches an example with a certain degree
Enginyeria i Arquitectura la Salle Slide 12
GRSI
13. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of Fuzzy-UCS
p y 3. Results
4. Conclusions
4C li
Each classifier has the following parameters:
1.
1 Weight per class wj:
• Soundness with which the rule predicts the class j.
• The class value is dynamic and corresponds to the class j with higher wj
2. Fitness:
• Quality of the rule
3. Other parameters directly inherited from UCS:
• numerosity
• Experience
Enginyeria i Arquitectura la Salle Slide 13
GRSI
14. Description of Fuzzy-UCS
p y
Describe the different components
1. Rule representation and matching
2. Learning interaction
3. Discovery component
3 Di t
4. Fuzzy-UCS in test mode
Enginyeria i Arquitectura la Salle Slide 14
GRSI
15. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of Fuzzy-UCS
p y 3. Results
4C li
4. Conclusions
Learning interaction:
– The environment provides an example e and its class c
– Match set creation: all classifiers that match with uAk(x) > 0
– Correct set creation: all classifiers that advocate c
– Covering: if there is not a classifier that maximally matches e
• Create the classifier that match the input example with maximum
degree.
• Generalize the condition with probability P#
For each variable:
A1 A2 A3
Enginyeria i Arquitectura la Salle Slide 15
GRSI
16. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of Fuzzy-UCS
p y 3. Results
4. Conclusions
4C li
Parameters’ Update
– Experience:
– Sum of correct matching per class j cmj:
Enginyeria i Arquitectura la Salle Slide 16
GRSI
17. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of Fuzzy-UCS
p y 3. Results
4. Conclusions
4C li
Parameters’ Update
– Use cm to update of the weights per each class:
• Rule that only matches instances of class c:
• wc = 1
• For all the other classes j: wj = 0
• Rule that matches instances o a c asses
u e t at atc es sta ces of all classes:
• All weights wi ranging [0, 1]
– Calculate the fitness
Pressuring toward rules that
correctly match instances of
only one class
Enginyeria i Arquitectura la Salle Slide 17
GRSI
18. Description of Fuzzy-UCS
p y
Describe the different components
1. Rule representation and matching
2. Learning interaction
3. Discovery component
3 Di t
4. Fuzzy-UCS in test mode
Enginyeria i Arquitectura la Salle Slide 18
GRSI
19. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of Fuzzy-UCS
p y 3. Results
4. Conclusions
4C li
Discovery component
– Steady state niched GA
Steady-state
– Roulette wheel selection
Instances that have a higher
g
matching degree have more
opportunities of being selected
Enginyeria i Arquitectura la Salle Slide 19
GRSI
20. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of Fuzzy-UCS
p y 3. Results
4. Conclusions
4C li
Discovery component
– Crossover and mutation applied on the antecedent
• 2 point crossover
IF [100 | 011] THEN class1
IF [101 | 100] THEN class1
• Mutation:
– Expansion
p IF [101 | 011] THEN class1
[ ]
IF [100 | 011] THEN class1
[ ]
– Contraction IF [100 | 001] THEN class1
IF [100 | 011] THEN class1
– Shift IF [010 | 011] THEN class1
IF [100 | 011] THEN class1
Enginyeria i Arquitectura la Salle Slide 20
GRSI
21. Description of Fuzzy-UCS
p y
Describe the different components
1. Rule representation and matching
2. Learning interaction
3. Discovery component
3 Di t
4. Fuzzy-UCS in test mode
Enginyeria i Arquitectura la Salle Slide 21
GRSI
22. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Description of Fuzzy-UCS
p y 3. Results
4. Conclusions
4C li
Class inference of a test example e
– Combining the information of all rules yields better results than
taking a single rule for reasoning (Cordon et al. 1998)
• Inference:
– All experienced rules vote for the class they predict as: uAk(e) · Fk
– The most voted class is returned.
Enginyeria i Arquitectura la Salle Slide 22
GRSI
23. Outline
1. Description of Fuzzy-UCS
1D ii fF UCS
2.
2 Experimental Methodology
3. Results
4. Conclusions
Enginyeria i Arquitectura la Salle Slide 23
GRSI
24. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Experimental Methodology
p gy 3. Results
4. Conclusions
4C li
Evaluating Fuzzy-UCS’ performance
– Compare Fuzzy-UCS accuracy to:
Fuzzy-UCS’
• Three non-fuzzy learners: UCS, SMO, and C4.5
• Two fuzzy learners: Fuzzy LogitBoost and Fuzzy GP
– Default configuration for all methods
–F
Fuzzy-UCS configuration:
UCS fi ti
iter = 100,000, N = 6400, F0 = 0.99, v=10, {θGA, θdel, θsub} = 50,
x =0.8, u 0.04, P#=0.6
0.8, u=0.04, 0.6
– Fuzzy learners: 5 linguistic labels per variable
– 10 fold cross-validation
10-fold cross validation
– Averages over 10 runs
Enginyeria i Arquitectura la Salle Slide 24
GRSI
26. Outline
1. Description of Fuzzy-UCS
1D ii fF UCS
2.
2 Experimental Methodology
3. Results
4. Conclusions
Enginyeria i Arquitectura la Salle Slide 26
GRSI
27. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Results 3. Results
4. Conclusions
4C li
• 1st objective: Competitive in terms of performance
Enginyeria i Arquitectura la Salle Slide 27
GRSI
28. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Results 3. Results
4. Conclusions
4C li
• 2nd objective: Improve the interpretability
Example of rules evolved by UCS for iris
Example of rules evolved by Fuzzy-UCS for iris
– Linguistic terms: {XS, S, M, L, XL}
Enginyeria i Arquitectura la Salle Slide 28
GRSI
29. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Further work 3. Results
4C li
4. Conclusions
Still large rule-sets!
Fuzzy-UCS
Fuzzy UCS UCS
2769 4494
annealing
1212 2177
balance
bl
1440 2961
bupa
2799 3359
glass
3574 2977
heart-c
2415 3735
heart-s
480 1039
iris
3130 2334
wbcd
3686 3685
wine
773 1291
zoo
Solution: New inference schemes
Enginyeria i Arquitectura la Salle Slide 29
GRSI
30. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Further work 3. Results
4C li
4. Conclusions
Still large rule-sets!
Fuzzy-UCS
y
Fuzzy-UCS
Fuzzy UCS UCS
best rule
36 2769 4494
annealing
75 1212 2177
balance
bl
39 1440 2961
bupa
36 2799 3359
glass
46 3574 2977
heart-c
62 2415 3735
heart-s
7 480 1039
iris
28 3130 2334
wbcd
26 3686 3685
wine
10 773 1291
zoo
Solution: New inference schemes
Enginyeria i Arquitectura la Salle Slide 30
GRSI
31. Outline
1. Description of Fuzzy-UCS
1D ii fF UCS
2.
2 Experimental Methodology
3. Results
4. Conclusions
Enginyeria i Arquitectura la Salle Slide 31
GRSI
32. 1. Description of Fuzzy-UCS
2. Experimental Methodology
Conclusions and Further Work 3. Results
4. Conclusions
4C li
Conclusions
– We proposed a Michigan-style LFCS for supervised learning
– Competitive with respect to:
• Some of the most-used machine learners: UCS, SMO, and C4.5
• Recent proposals of Fuzzy-learners: Fuzzy LogitBoost and Fuzzy GP
– Improvement in terms of interpretability with respect to UCS
Further work
– Evolve more reduced populations
– Enhance the comparison with new real-world problems
– Compare to other LFCS
– Exploit the incremental learning approach to dig large datasets
Enginyeria i Arquitectura la Salle Slide 32
GRSI
33. Aprendizaje Supervisado de Reglas
Difusas mediante un Sistema
Clasificador Evolutivo Estilo Michigan
Albert Orriols-Puig1,2
Orriols Puig
Jorge Casillas2
Ester Bernadó-Mansilla1
1Grup de Recerca en Sistemes Intel·ligents
Enginyeria i Arquitectura La Salle, Universitat Ramon Llull
2Dept. de Ciencias de la Computación e IA
Universidad of Granada