NK Landscapes, Problem Difficulty, and Hybrid Evolutionary Algorithms

NK Landscapes, Problem Diﬃculty, and Hybrid
Evolutionary Algorithms

Martin Pelikan
Missouri Estimation of Distribution Algorithms Laboratory (MEDAL)
University of Missouri, St. Louis, MO
http://medal.cs.umsl.edu/
pelikan@cs.umsl.edu

Download MEDAL Report No. 2010001
http://medal.cs.umsl.edu/files/2010001.pdf

Martin Pelikan NK Landscapes, Problem Diﬃculty, and Hybrid EAs

Motivation

Problem difficulty measures
Important for understanding and estimating problem difficulty.
Useful in designing, chosing and setting up optimization
algorithms.

Little work on problem difficulty of large sets of instances
Most past work considers isolated instances.
Little work on analyzing difficulty of large sets of instances.

This study
Consider
Two classes of NK landscapes.
Several popular problem difficulty measures.
Analyze difficulty measures on large sets of problem instances.


Outline

1. Problem diﬃculty measures.
Fitness distance correlation.
Correlation length.
Escape rate.
Distance of local and global optima.

2. NK landscapes.

3. Experiments.

4. Conclusions and future work.


Fitness Landscapes

Components of fitness landscape
Set S of admissible solutions.
Fitness function that assigns a real value to each solution.
Distance measure that defines distance between any two
solutions in S.

Problem vs. operators
Set of solutions and fitness function depend on the problem.
Distance measure depends on the operators (sometimes also
on problem).

Example distance measures
Hamming distance for binary strings (num. mismatched bits).
Euclidean distance metric for continuous vectors.

Fitness Distance Correlation (FDC): Intuition
Loose definition
FDC measures correlation between fitness and distance to global optimum.

Simple functions
Being closer to a global optimum leads to higher fitness.
Climbing directly to the optimum is rather easy.

Difficult functions
Being closer to a global optimum may lead to lower fitness.
Climbing directly to the optimum is not that easy.


Fitness Distance Correlation (FDC): Deﬁnition

Basic setup
Set of n candidate solutions x1 , x2 , . . . , xn .
Fitness values: F = {f1 , f2 , . . . , fn }.
Distances from closest global optimum: D = {d1 , d2 , . . . , dn }.
Can also focus only on local optima (useful for hybrid EAs).
Important statistics
¯
Average of F is f ; standard deviation of F is σF .
¯
Average of D is d; standard deviation of D is σD .
Covariance of F and D
n
1 ¯ ¯
cF D = (fi − f )(di − d)·
n i=1

Fitness distance correlation (FDC)
cF D
r=
σF σD
Range of values for FDC is [−1, 1].
For maximization, small FDC values indicate simpler problems.

Correlation Length: Intuition

Loose definition
Correlation length measures ruggedness of the landscape.

Simple functions
Solutions close to each other have similar fitness values.
Fitness differences increase slowly with distance.

Difficult functions
Solutions close to each other have different fitness values.
Fitness differences increase fast with distance.


Correlation Length: Deﬁnition

Basic setup
Random walk of n steps x1 , x2 , . . . , xn .
xi−1 and xi are neighbors.
Fitness values: F = {f1 , f2 , . . . , fn }.

Autocorrelation function for gap s
m−s
1 ¯ ¯
ρ(s) = 2 (m − s)
ft − f ft+s − f ,
σF t=1

Correlation length
1
l=−
ln |ρ(1)|

Smaller correlation lengths indicates harder problems.


Escape Rate
Intuition
Problem difficulty relates to ease with which we can escape local optima.

Escape rate for s steps
Start in an arbitrary local optimum xl .
Generate n solutions at distance s from xl .
Climb (steepest ascent) to nearest local optimum from each of them.
Escape rate is percentage of starting points that end up in a different local
optimum.
Repeat the above for multiple local optima.

Comments
Escape rate increases with number of steps s.
The larger escape rates may mean an easier problem because
it is easier to escape local optima.
The larger escape rates may mean increased ruggedness,
which indicates greater difficulty.


Distance of Local and Global Optima

Intuition
Problem diﬃculty is related to distance of global and local
optima.

Deﬁnition
Distance of local and global optima is the average distance of
a local optimum to its closest global optimum.

Some thoughts
Short distances indicate an easier problem, because it is easy
to climb near the global optimum.
Long distances may indicate decreased ruggedness, and
therefore an easier problem.


NK Landscape

NK landscape
Proposed by Kauffman (1989).
Model of rugged landscape and popular test function.
An NK landscape is defined by
Number of bits, n.
Number of neighbors per bit, k.
Set of k neighbors Π(Xi ) for i-th bit, Xi .
Subfunction fi defining contribution of Xi and Π(Xi ).
The objective function fnk to maximize is then defined as
n−1
fnk (X0 , X1 , . . . , Xn−1 ) = fi (Xi , Π(Xi )).
i=0


NK Landscape

Exmaple for n = 9 and k = 2:


Types of NK Landscapes
Unrestricted NK landscapes
Neighbors are chosen arbitrarily.
Each subfunction is look-up table from uniform distr. over [0, 1).
NP-complete problem.

Nearest-neighbor NK landscapes
Neighbors are in subsequent positions (neighborhoods wrap around).
Each subfunction is look-up table from uniform distr. over [0, 1).
Solvable in polynomial time using dynamic programming.

Parameters
Number of bits n ranges from 20 to 100.
Neighborhood size k ranges from 2 to 6.
For each n and k, we generate 104 random instances.


Hybrid hBOA

Hybrid hierarchical Bayesian optimization algorithm (hBOA)
Build and sample a Bayes net instead of crossover/mutation.
Model structure and parameters adapt to problem landscape.
Use niching to preserve diversity.
Use single-bit-ﬂip hill climbing for all candidate solutions.

Current Selected Bayesian New
population population network population


Focus of Experiments

What we already know from prior work
Effects of n and k on hBOA performance.
Effects of neighborhood type on hBOA performance.

Main areas of focus of this study
Effects of n and k on problem-difficulty measures.
Problem-difficulty measures and performance for fixed n and k.
Effects of neighborhood type on problem-difficulty measures.
Effects of escape rate on performance.


Eﬀects of k
Standard NK landscapes (n = 36)

Nearest-neighbor NK landscapes (n = 100)


Table 1: Some of the measures of problem difficulty for instances of varying problem size. Standard
Effects of n for Standard NK Landscapes
deviations are shown in brackets.
(a) Standard NK landscapes for k = 2 and k = 6.
DHC steps until fitness dist. corr. avg. dist. of
n k opt. (hBOA+DHC) fitness dist. corr. for local optima corr. length local from global
20 2 147.8 (88.6) −0.3202 (0.105) −0.6862 (0.18) 5.30 (0.41) 14.92 (0.63)
28 2 355.9 (195.9) −0.3177 (0.090) −0.6624 (0.17) 7.63 (0.50) 20.91 (0.74)
36 2 666.3 (340.2) −0.3181 (0.079) −0.6542 (0.16) 9.97 (0.56) 26.91 (0.83)
44 2 1105.0 (515.4) −0.3145 (0.072) −0.6424 (0.15) 12.29 (0.61) 32.88 (0.91)
52 2 1633.6 (727.7) −0.3145 (0.068) −0.6384 (0.14) 14.62 (0.67) 38.88 (0.98)
20 6 563.2 (308.9) −0.0496 (0.050) −0.2086 (0.11) 2.30 (0.05) 15.83 (0.30)
28 6 1916.5 (1148.3) −0.0491 (0.043) −0.1605 (0.09) 3.44 (0.06) 22.28 (0.32)
36 6 5583.8 (3940.5) −0.0489 (0.038) −0.1355 (0.08) 4.58 (0.07) 28.72 (0.33)

(b) Nearest-neighbor NK landscapes for k = 2 and k = 6.
Fitness distance correlation
DHC steps until opt. fitness dist. corr. avg. dist. of
n k (hBOA+DHC) distance correlation agrees with results. length
Fitness fitness dist. corr. for local optima corr. local from global
20 2 The differences are very small.−0.7271 (0.151)
173.0 (93.0) −0.3428 (0.104) 5.31 (0.39) 14.96 (0.64)
40 2 873.4 (286.7) −0.3412 (0.074) −0.7180 (0.107) 11.13 (0.56) 29.89 (0.91)
60 2 Focusing on local optima improves the measure. (0.68)
1881.4 (435.9) −0.3416 (0.061) −0.7151 (0.087) 16.95 44.85 (1.11)
80 2 Correlation length −0.3416 (0.054) −0.7143 (0.076) 22.76 (0.80)
3055.0 (691.9) 59.79 (1.29)
100 2 4436.2 (1019.5) −0.3423 (0.049) −0.7134 (0.067) 28.59 (0.88) 74.75 (1.44)
20 6 Correlation length indicates opposite (0.122)
634.3 (350.1) −0.0494 (0.051) −0.2518 behavior as expected.
2.30 (0.06) 15.80 (0.33)
40 6 5747.2 (3770.7)provide a good indicator in(0.091)case.5.14 (0.08)
Does not −0.0486 (0.036) −0.2430 this 31.68 (0.45)
60 6 16569.5 (10933.3) −0.0481 (0.030) −0.2416 (0.075) 7.97 (0.10) 47.53 (0.54)
80 6 Distance of optima
34485.9 (23700.1) −0.0488 (0.026) −0.2419 (0.065) 10.80 (0.11) 63.37 (0.62)
100 6 60774.8 (42442.8) −0.0486 (0.024) −0.2431 (0.058) 13.63 (0.13) 79.21 (0.69)
Distance of local and global optima increase with n.

5.4.2Pelikan
Martin Measures and actual performance NK Landscapes, Problem Difficulty, and Hybrid EAs

28 6 1916.5 (1148.3) −0.0491 (0.043) −0.1605 (0.09) 3.44 (0.06) 22.28 (0.32)
Effects of 5583.8 (3940.5) −0.0489 (0.038) −0.1355 (0.08) 4.58 (0.07)
36 6
n for Nearest-Neighbor NK Landscapes 28.72 (0.33)

(b) Nearest-neighbor NK landscapes for k = 2 and k = 6.
DHC steps until opt. fitness dist. corr. avg. dist. of
n k (hBOA+DHC) fitness dist. corr. for local optima corr. length local from global
20 2 173.0 (93.0) −0.3428 (0.104) −0.7271 (0.151) 5.31 (0.39) 14.96 (0.64)
40 2 873.4 (286.7) −0.3412 (0.074) −0.7180 (0.107) 11.13 (0.56) 29.89 (0.91)
60 2 1881.4 (435.9) −0.3416 (0.061) −0.7151 (0.087) 16.95 (0.68) 44.85 (1.11)
80 2 3055.0 (691.9) −0.3416 (0.054) −0.7143 (0.076) 22.76 (0.80) 59.79 (1.29)
100 2 4436.2 (1019.5) −0.3423 (0.049) −0.7134 (0.067) 28.59 (0.88) 74.75 (1.44)
20 6 634.3 (350.1) −0.0494 (0.051) −0.2518 (0.122) 2.30 (0.06) 15.80 (0.33)
40 6 5747.2 (3770.7) −0.0486 (0.036) −0.2430 (0.091) 5.14 (0.08) 31.68 (0.45)
60 6 16569.5 (10933.3) −0.0481 (0.030) −0.2416 (0.075) 7.97 (0.10) 47.53 (0.54)
80 6 34485.9 (23700.1) −0.0488 (0.026) −0.2419 (0.065) 10.80 (0.11) 63.37 (0.62)
100 6 60774.8 (42442.8) −0.0486 (0.024) −0.2431 (0.058) 13.63 (0.13) 79.21 (0.69)

5.4.2 Measures and actual correlation agrees with results.
Fitness distance performance
The previous subsection focusedare the influence of n and k on the performance of hBOA+DHC and
The differences on very small.
the measures of problem on local optima improves the measure.between the measures of problem
Focusing difficulty. But what is the relationship
Correlation length
difficulty and the actual performance of hBOA+DHC for fixed values of n and k? Furthermore, it is
well known that for NK landscapesindicates opposite behavior as expected.
Correlation length and other difficult classes of additively decomposable problems,
performance ofDoes not provide a good indicator in this one problem instance to another. Will
metaheuristics varies substantially from case.
these variations be captured by the considered measures of problem difficulty?
Distance of optima
The relationship between the measures of problem difficulty and the actual computational
Distance of local and global optima increase with n.
requirements of hBOA+DHC is illustrated in tables 2 and 3. Only the results for several values
of n and k are shown; the results for other values of n and k were qualitatively similar. The
results show that the correlation length and both versions ofProblem Difficulty, and Hybrid EAs
Martin Pelikan NK Landscapes, the fitness distance correlation are

50% easiest 422.4 (123.6) −0.3411 (0.076) −0.7081 (0.141) 10.0599 (0.553) 27.0721 (0.835)
all instances 666.3 (340.2) −0.3181 (0.079) −0.6542 (0.160) 9.9729 (0.555) 26.9069 (0.831)
Measures vs. Performance for Fixed n, k (Standard)
50% hardest
25% hardest
910.1
1119.1
(311.8)
(323.4)
−0.2952
−0.2819
(0.076)
(0.075)
−0.6004
−0.5635
(0.160)
(0.161)
9.8859
9.8345
(0.543)
(0.547)
26.7417
26.6541
(0.793)
(0.781)
10% hardest 1391.1 (358.7) −0.2695 (0.076) −0.5259 (0.158) 9.8102 (0.570) 26.5672 (0.757)

StandardStandardlandscapesn (n = = 6. k = 6)
(b) NK NK landscapes, = 36, k 36,
desc. of DHC steps until fitn. dist. corr. avg. dist. of
instances opt. (hBOA+DHC) fitn. dist. corr. for local optima corr. length local from global
10% easiest 1785.8 (315.3) −0.0737 (0.035) −0.2113 (0.070) 4.5874 (0.069) 28.9966 (0.370)
25% easiest 2279.8 (488.9) −0.0675 (0.035) −0.1909 (0.068) 4.5836 (0.067) 28.8979 (0.356)
50% easiest 3034.4 (882.5) −0.0604 (0.036) −0.1682 (0.069) 4.5803 (0.067) 28.8242 (0.340)
all instances 5583.8 (3940.5) −0.0489 (0.038) −0.1355 (0.075) 4.5752 (0.067) 28.7155 (0.332)
50% hardest 8133.3 (4156.7) −0.0374 (0.035) −0.1029 (0.067) 4.5701 (0.066) 28.6067 (0.285)
25% hardest 10738.5 (4534.7) −0.0316 (0.036) −0.0868 (0.066) 4.5699 (0.066) 28.5572 (0.272)
10% hardest 14542.1 (5101.1) −0.0242 (0.036) −0.0716 (0.064) 4.5652 (0.066) 28.5074 (0.258)

Provides good indication of problem difficulty.
Table 3: Some of the measures of problem difficulty for instances of varying difficulty. Nearest-
neighbor NK landscapes are considered with n = 100, and kbased on performance. of each
Differences smaller than expected = 2 or k = 6. The difficulty
instance is measured by the number of DHC steps until optimum using hBOA+DHC. Standard
deviations Correlation length
are shown in brackets.
Correlation length agreeslandscapes, n = 100, k = 2.
(a) Nearest-neighbor NK
with the results.
desc. of DHC steps until fitn. dist. corr. avg. dist. of lo-
instances Differences are almost negligible optima corr. length
opt. (hBOA+DHC) fitn. dist. corr. for local though. cal from global
10% easiest 3330.9 (163.9) (0.045) −0.7594 (0.054) 28.9495 (0.878) 75.2075 (1.370)
Distance of optima
25% easiest 3550.2 (217.0)
−0.3763
−0.3657 (0.045) −0.7466 (0.058) 28.8338 (0.878) 75.1053 (1.395)
50% easiest 3758.6 (265.2) (0.046) −0.7364 (0.058) 28.7432 (0.873) 74.9838 (1.427)
Greater distances observed for simpler problems.
−0.3578
all instances 4436.2 (1019.5) −0.3423 (0.049) −0.7134 (0.067) 28.5919 (0.885) 74.7529 (1.440)
50% hardest Indicates that ruggedness is important.
5113.8 (1044.2) −0.3269 (0.047) −0.6905 (0.067) 28.4405 (0.871) 74.5221 (1.415)
25% hardest 5805.5 (1089.4) −0.3178 (0.047) −0.6728 (0.069) 28.3831 (0.854) 74.3572 (1.390)
10% hardest 6767.6 (1152.3) −0.3115 (0.048) −0.6632 (0.069) 28.3347 (0.865) 74.3459 (1.372)

50% easiest 3758.6 (265.2) −0.3578 (0.046) −0.7364 (0.058) 28.7432 (0.873) 74.9838 (1.427)
all instances 4436.2 (1019.5) −0.3423 (0.049) −0.7134 (0.067) 28.5919 (0.885) 74.7529 (1.440)
Measures vs. Performance for Fixed n, k (Nearest Neighb.)
50% hardest
25% hardest
5113.8 (1044.2)
5805.5 (1089.4)
−0.3269
−0.3178
(0.047)
(0.047)
−0.6905
−0.6728
(0.067)
(0.069)
28.4405
28.3831
(0.871)
(0.854)
74.5221
74.3572
(1.415)
(1.390)
10% hardest 6767.6 (1152.3) −0.3115 (0.048) −0.6632 (0.069) 28.3347 (0.865) 74.3459 (1.372)

Nearest-neighbor NK NK landscapes, n =(n ==100, k = 6)
(b) Nearest-neighbor landscapes 100, k 6.
desc. of DHC steps until fitn. dist. corr. avg. dist. of lo-
instances opt. (hBOA+DHC) fitn. dist. corr. for local optima corr. length cal from global
10% easiest 21364.1 (2929.9) −0.0601 (0.023) −0.2962 (0.050) 13.6468 (0.135) 79.6971 (0.720)
25% easiest 26787.7 (5261.6) −0.0564 (0.023) −0.2778 (0.053) 13.6432 (0.131) 79.5491 (0.706)
50% easiest 34276.6 (8833.1) −0.0529 (0.023) −0.2626 (0.055) 13.6395 (0.132) 79.4021 (0.687)
all instances 60774.8 (42442.8) −0.0486 (0.024) −0.2431 (0.058) 13.6344 (0.131) 79.2125 (0.690)
50% hardest 87272.9 (46049.2) −0.0444 (0.023) −0.2237 (0.055) 13.6294 (0.130) 79.0228 (0.638)
25% hardest 114418.9 (52085.3) −0.0430 (0.023) −0.2163 (0.056) 13.6274 (0.131) 78.9531 (0.638)
10% hardest 154912.8 (62794.1) −0.0418 (0.024) −0.2107 (0.057) 13.6243 (0.130) 78.9277 (0.633)

10
Provides good indication of problem difficulty.
Correlation length
Correlation length agrees with the results.
Differences are almost negligible though.
Distance of optima
Greater distances observed for simpler problems.
Indicates that ruggedness is important.


Escape Rate for Nearest-Neighbor Instances (n = 100)

1
0.9
Escape rate 0.8
0.7
0.6
0.5 k=6
0.4 k=5
0.3 k=4
0.2 k=3
0.1 k=2
0
0 5 10 15 20 25
Number of steps

3: Escape rate for nearest-neighbor NK landscapes of n = 100 and k = 6.
Escape rate increases with k.
Increased k leads to more ruggedness and tougher problems.
shows thatEscape rate increases with the number of steps.difficulty correlate with th
in most cases the measures of problem
requirements of hybrid hBOA. Most measures capture the effects of n a
lty, and they provide input on problem difficulty Problem Difficulty, and Hybrid EAs k. Non
Martin Pelikan NK Landscapes, even for fixed n and

25% hardest 114418.93 (52085.301) −0.04304 (0.0231) −0.21626 (0.0558) 13.62740 (0.1308)
10% hardest 154912.82 (62794.146) −0.04179 (0.0236) −0.21073 (0.0571) 13.62429 (0.1300)
Neighborhood Type
Table 4: Comparison of the results for the standard NK landscapes (std) and for the n
landscapes (nn). In both cases, n = 36 and k varies from 2 to 6. Standard deviations are
DHC steps fit. dist. corr. fit. dist. corr. (local) corr. length d
k
std nn std nn std nn std nn
2 666 697 −0.32 (0.08) −0.34 (0.08) −0.65 (0.16) −0.72 (0.11) 9.973 (0.555) 9.962 (0.527) 13
3 1029 983 −0.20 (0.07) −0.21 (0.07) −0.50 (0.16) −0.60 (0.13) 7.912 (0.342) 7.911 (0.333) 11
4 1629 1437 −0.13 (0.06) −0.13 (0.06) −0.35 (0.13) −0.48 (0.13) 6.450 (0.200) 6.446 (0.202) 9
5 2846 2397 −0.08 (0.05) −0.08 (0.05) −0.23 (0.10) −0.35 (0.11) 5.379 (0.116) 5.379 (0.123) 8
6 5584 4255 −0.05 (0.04) −0.05 (0.04) −0.14 (0.08) −0.24 (0.09) 4.575 (0.067) 4.576 (0.073) 7

Fitness distance correlation for arbitrary solutions
[13] T. Jones and S. Forrest. Fitness distance correlation as a decomposable problems. Parallel
measure of problem arbitrary solutions, differences are negligible. pages 788–797, 2006.
For difficulty for genetic algorithms. Proc. Nature,
of the Int. Conf. on Genetic Algorithms (ICGA-95), pages [23] M. Pelikan, K. Sastry, M. V. Butz
184–192, 1995.Situation doesn’t seem to change with k. Analysis of estimation of distribut
[14] S. Kauffman. Adaptation on rugged fitness landscapes. In
Fitness distance correlation for local optima genetic algorithms onConf.landsca
D. L. Stein, editor, Lecture Notes in the Sciences of Evolutionary Comp.
NK
(GEC
Complexity, pages 527–618.indicates that 1989.
Correctly Addison Wesley, nearest-neighbor instances are easier.
1033–1040, 2008.
[15] S. Kauffman. The origins of Order: Self-organization and [24] M. Pelikan, K. Sastry, D. E. Gold
selection in evolution. Oxfordincrease with k (as expected). M. Hauschild. Performance of evo
Differences University Press, 1993.
[16] P. Merz. Advanced fitness landscape analysis and the NK landscapes with nearest neigh
Correlation length
performance of memetic algorithms. Evolutionary tunable overlap. Genetic and Evo
Computation, 12(3):303–325, are negligible.
Differences 2004. (GECCO-2009), pages 851–858, 2
[17] P. Merz and B. Freisleben. Greedy and local search [25] K. Sastry. Evaluation-relaxation s
heuristics for unconstraineddoesn’t seem to change with k.
Situation binary quadratic programming. evolutionary algorithms. Master’s
Journal of Heuristics, 8(2):197–213, 2002. Illinois at Urbana-Champaign, De
[18] M. Pelikan. Hierarchical Bayesian optimization algorithm: Engineering, Urbana, IL, 2001.
Toward a new generation of evolutionary algorithms. [26] P. F. Stadler. Landscapes and the
Springer, 2005. Journal of Mathematical Chemist

Conclusions and Future Work
Conclusions
Different measures focus on different facets of difficulty.
Most success with fitness distance correlation for local optima.
Differences in measures usually very small.
Need better measures to make these results useful in practice.
Most measures depend on information that is typically unavailable.

Future work
Compare to other existing measures (signal/noise, scaling).
Look at other optimization algorithms (e.g. GAs and ESs).
Define measures that do not depend on problem-specific
information.
Define measures that are more practically useful
Tune algorithm to problem (parameters, operators).
Choose best optimization algorithm.
Drive design of new optimization algorithms.


Acknowledgments

Acknowledgments
NSF; NSF CAREER grant ECS-0547013.
University of Missouri; High Performance Computing
Collaboratory sponsored by Information Technology Services;
Research Award; Research Board.


NK Landscapes, Problem Difficulty, and Hybrid Evolutionary Algorithms

Recomendados

Recomendados

Más contenido relacionado

Más de Martin Pelikan

Más de Martin Pelikan (15)

Último

Último (20)

NK Landscapes, Problem Difficulty, and Hybrid Evolutionary Algorithms