SlideShare una empresa de Scribd logo
1 de 54
Descargar para leer sin conexión
Hyper-heuristics: Past Present and Future
The University of Nottingham




                                                               Graham Kendall
                                                                               gxk@cs.nott.ac.uk


                                          Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Contents
The University of Nottingham




                               Past
                               • A selection of early work


                               Present
                               • Current State of the Art
                                                                                                                       Albert Einstein
                               Future                                                                                     1879 - 1955
                               • Potential Research Directions for the Future
                                                                                                            “We can't solve problems by using the
                                                                                                             same kind of thinking we used when
                                                                                                                      we created them.”


                                                             Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Contents
The University of Nottingham




                               Past
                               • A selection of early work


                               Present
                               • Current State of the Art
                                                                                                                       Albert Einstein
                               Future                                                                                     1879 - 1955
                               • Potential Research Directions for the Future
                                                                                                            “We can't solve problems by using the
                                                                                                             same kind of thinking we used when
                                                                                                                      we created them.”


                                                             Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
The University of Nottingham




                               Fisher H. and Thompson G.L. (1963) Probabilistic Learning
                               Combinations of Local Job-shop Scheduling Rules. In Muth J.F. and
                               Thompson G.L. (eds) Industrial Scheduling, Prentice Hall Inc., New
                               Jersey, 225-251

                               Based on (I assume)

                               Fisher H. and Thompson G.L. (1961) Probabilistic Learning
                               Combinations of Local Job-shop Scheduling Rules. In Factory
                               Scheduling Conference, Carnegie Institute of Technology




                                                                    Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Good
                                                                 Facility Order Matrix
The University of Nottingham



                                  Number
                                       1            3(1)       1(3)      2(6)      4(7)        6(3)       5(6)
                                       2            2(8)       3(5)     5(10)     6(10)       1(10)       4(4)
                                       3            3(5)       4(4)      6(8)      1(9)        2(1)       5(7)
                                       4            2(5)       1(5)      3(5)      4(3)        5(8)       6(9)
                                       5            3(9)       2(3)      5(5)      6(4)        1(3)       4(1)
                                       6            2(3)       4(3)      6(9)     1(10)        5(4)       3(1)
                                                6 x 6*6 Test Problem (times in brackets)



                               “The number of feasible active schedules is, by a conservative estimate, well over
                                       a million, so their complete enumeration is out of the question.”


                               • Also 10 (jobs) x 10 (operations) and 20 (jobs)
                                 x 5 (operations) problems


                                                                           Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Good
                                                      Facility Order Matrix
The University of Nottingham



                               Number
                                 1         3(1)     1(3)     2(6)      4(7)        6(3)       5(6)
                                 2         2(8)     3(5)    5(10)     6(10)       1(10)       4(4)
                                 3         3(5)     4(4)     6(8)      1(9)        2(1)       5(7)
                                 4         2(5)     1(5)     3(5)      4(3)        5(8)       6(9)
                                 5         3(9)     2(3)     5(5)      6(4)        1(3)       4(1)
                                 6         2(3)     4(3)     6(9)     1(10)        5(4)       3(1)
                                        6 x 6*6 Test Problem (times in brackets)




                                                                                                             Job 3, 1, 2, 5, 4, 6


                                                               Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Good
                                                       Facility Order Matrix
The University of Nottingham



                                Number
                                   1        3(1)     1(3)     2(6)      4(7)        6(3)       5(6)
                                   2        2(8)     3(5)    5(10)     6(10)       1(10)       4(4)
                                   3        3(5)     4(4)     6(8)      1(9)        2(1)       5(7)
                                   4        2(5)     1(5)     3(5)      4(3)        5(8)       6(9)
                                   5        3(9)     2(3)     5(5)      6(4)        1(3)       4(1)
                                   6        2(3)     4(3)     6(9)     1(10)        5(4)       3(1)
                                         6 x 6*6 Test Problem (times in brackets)

                               • Two Rules
                                  • SIO: Shortest Imminent Operation (“First on,
                                    First Off”)
                                  • LRT: Longest Remaining Time
                               • Only require knowledge of “your”
                                 machine

                                                                Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
• Monte Carlo: 58 time Units
The University of Nottingham



                               • SIO:         67 time units
                               • LRT:         61 time units
                               • Optimal:     55 time units


                               • SIO should be used initially (get the
                                 machines to start work) and LRT later
                                 (work on the longest jobs)
                               • Why not combine the two heuristics?
                               • Four learning models, rewarding good
                                 heuristic selection



                                                     Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Remarks
The University of Nottingham




                               • Not sure about reproducibility (e.g.
                                 reward/punishment functions)
                               • An unbiased random combination of
                                 scheduling rules is better than any of them
                                 taken separately
                               • “Learning is possible, but there is a question as
                                 to whether learning is desirable given the
                                 effectiveness of the random combination”
                               • “It is not clear what is being learnt as the
                                 original conjecture was not strongly
                                 supported”
                               • “It is likely that combinations of 5-10 rules
                                 would out-perform humans”



                                                          Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
The University of Nottingham




                               Fang H-L., Ross P. and Corne D. (1993) A Promising genetic
                               Algorithm Approach to Job-Shop Scheduling, Reschecduling, and
                               Open-Shop Scheduling Problems. In Forrest S. (ed) Fifth International
                               Conference on Genetic Algorithms, Morgan Kaufmann, San Mateo,
                               375-383




                                                                     Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Representation
The University of Nottingham




                               • For a j x m problem, a string represents j x m
                                 chunks.
                               • The chunk is atomic from a GA perspective.
                               • The chunks abc means to put the first
                                 untackled task of the ath uncompleted job into
                                 the earliest place it will fit in the developing
                                 schedule, then put the bth uncompleted job into
                                 ….
                               • A schedule builder decodes the chromosome.
                               • Fairly standard GA e.g. population size of 500,
                                 rank based selection, elitism, 300 generations,
                                 crossover rate 0.6, adaptive mutation rate




                                                          Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Other Remarks
The University of Nottingham




                               • Considered Job-Shop Scheduling and Open-
                                 Shop Scheduling

                               • Experimented with different GA parameters

                               • Results compared favourably with best known
                                 or optimal




                                                        Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
The University of Nottingham




                               Denzinger J. and Fuchs M. (1997) High Performance ATP Systems by
                               Combining Several AI Methods. In proceedings of the Fifteenth
                               International Joint Conference on Artificial Intelligence (IJCAI 97),
                               102-107




                                                                     Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Remarks
The University of Nottingham




                               • The first paper to use the term Hyper-heuristic

                               • Used in the context of an automated theorem
                                 prover

                               • A hyper-heuristic stores all the information
                                 necessary to reproduce a certain part of the
                                 proof and is used instead of a single heuristic




                                                           Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
The University of Nottingham




                               O’Grady P.J. and Harrison (1985) A General Search Sequencing Rule
                               for Job Shop Sequencing. International Journal of Production
                               Research, 23(5), 961-973




                                                                   Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Remarks
The University of Nottingham




                               Pi = (Ai x Ti) + (Bi x Si)

                               where
                               Pi the priority index for job i at its current stage
                               Ai a 1 x m coefficient vector for job i
                               Ti a m x 1 vector which contains the remaining
                                  operation times for job i in process order
                               Bi the due date priority coefficient for job i
                               Si the due date slack for job i
                               m the maximum number of processing stages
                                  for jobs 1 to i




                                                            Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Remarks
                               Pi = (Ai x Ti) + (Bi x Si)
The University of Nottingham




                               where
                               Pi the priority index for job i at its current stage
                               Ai a 1 x m coefficient vector for job i
                               Ti a m x 1 vector which contains the remaining operation
                                  times for job i in process order
                               Bi the due date priority coefficient for job i
                               Si the due date slack for job i
                               m the maximum number of processing stages for jobs 1 to i




                               A = (1,0,0,0,0,…,0), B = 0
                               Shortest Imminent Operation Time

                               A = (0,0,0,0,0,…,0), B = 1
                               Due Date Sequencing

                                                                Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Remarks
                               Pi = (Ai x Ti) + (Bi x Si)
The University of Nottingham




                               where
                               Pi the priority index for job i at its current stage
                               Ai a 1 x m coefficient vector for job i
                               Ti a m x 1 vector which contains the remaining operation
                                  times for job i in process order
                               Bi the due date priority coefficient for job I
                               Si the due date slack for job i
                               m the maximum number of processing stages for jobs 1 to i




                               A search is performed over Ai and Bi in order to
                               cause changes in the processing sequences.




                                                                Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
The University of Nottingham




                               Norenkov I. P. and Goodman E D. (1997) Solving Scheduling
                               Problems via Evolutionary Methods for Rule Sequence Optimization.
                               In proceedings of the 2nd World Conference on Soft Computing
                               (WSC2)




                                                                   Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Remarks
The University of Nottingham




                               • Similar in idea to Fang, Ross and Corne (1994)

                               • The allele at the ith position is the heuristic to
                                 be applied at the ith step of the scheduling
                                 process.

                               • Comparison with using eight single heuristics
                                 and the Heuristic Combination Method (HCM)
                                 was found to be superior.




                                                            Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Other (Selected) Papers
The University of Nottingham




                               • Crowston W.B., Glover F., Thompson G.L. and
                                 Trawick J.D. (1963) Probabilistic and Parameter
                                 Learning Combinations of Local Job Shop
                                 Scheduling Rules. ONR Research Memorandum,
                                 GSIA, Carnegie Mellon University

                               • Storer R.H., Wu S.D. and Vaccari R. (1992) New
                                 Search Spaces for Sequencing Problems with
                                 Application to Job Shop Scheduling. Management
                                 Science, 38(10), 1495-1509

                               • Battiti R. (1996) Reactive Search: Toward Self
                                 Tuning Heuristics. In Rayward-Smith R.J., Osman
                                 I.H., Reeves C.R. and Smith G.D. (eds) Modern
                                 Heuristics Search methods, John Wiley, 61-83



                                                           Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Contents
The University of Nottingham




                               Past
                               • A selection of early work


                               Present (Heuristics to Choose Heuristics)
                               • Current State of the Art
                                                                                                                       Albert Einstein
                               Future                                                                                     1879 - 1955
                               • Potential Research Directions for the Future
                                                                                                            “We can't solve problems by using the
                                                                                                             same kind of thinking we used when
                                                                                                                      we created them.”


                                                             Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Heuristics to Choose Heuristics
The University of Nottingham




                                          Hyper-heuristic

                                             Data flow

                                           Domain Barrier


                                             Data flow


                                   Set of low level heuristics
                                     H1     H2                 Hn
                                                 ……
                                     Evaluation Function



                                                    Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Choice Function
The University of Nottingham



                               • f1 + f2 + f3
                               • f1 = How well has each
                                 heuristic performed
                               • f2 = How well have pairs of
                                 heuristics performed
                               • f3 = Time since last called




                                                   Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Tabu Search
The University of Nottingham




                               • Low level heuristics compete
                                 with each other
                               • Recent heuristics are made tabu
                               • Rank low level heuristics based
                                 on their estimated performance
                                 potential




                                                  Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Case Based Heuristic Selection
The University of Nottingham




                               • Find heuristics that worked well
                                 in previous similar problem
                                 solving situations
                               • Features discovered in similarity
                                 measure – key research issue




                                                  Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Adaptive Ordering Strategies
The University of Nottingham




                               • Based on Squeaky Wheel
                                 Optimisation
                               • Consider constructive heuristics
                                 as orderings
                               • Adapt the ordering by a heuristic
                                 modifier according to the penalty
                                 imposed by certain features
                               • Generative


                                                  Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Contents
The University of Nottingham




                               Past
                               • A selection of early work


                               Present (Generating Heuristics)
                               • Current State of the Art


                               Future
                               • Potential Research Directions for the Future




                                                             Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Generating heuristics
The University of Nottingham



                                                                                                              Hyper-heuristic
                               • Rather than supply a set of low
                                 level heuristics, generate the                                                      Data flow
                                 heuristics automatically
                                                                                                                Domain Barrier

                               • Heuristics could be one off                                                        Data flow
                                 (disposal) heuristics or could be
                                 applicable to many problem
                                 instances                                                         Set of low level heuristics
                                                                                                         H1        H2                        Hn
                                                                                                                          ……
                                                                                                         Evaluation Function




                                                   Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Generating heuristics
The University of Nottingham




                               Burke E. K., Hyde M. and Kendall G. Evolving Bin Packing
                               Heuristics With Genetic Programming. In Proceedings of the 9th
                               International Conference on Problem Parallel Solving from Nature
                               (PPSN 2006), pp 860-869, LNCS 4193, Reykjavik, Iceland, 9-13
                               Sepetmber 2006




                                                                    Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Generating heuristics
The University of Nottingham




                               • Evolves a control program that
                                 decides whether to put a given
                                 piece into a given bin
                               • First-fit heuristic evolved from
                                 Genetic Programming without
                                 human input on benchmark
                                 instances
                               For each piece, p, not yet packed
                                 For each bin, i
                                   output = evaluate(p, fullness of i, capacity of i)
                                   if (output > 0)
                                      place piece p in bin i
                                      break
                                   fi
                                 End For
                               End For

                                                            Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Contents
The University of Nottingham




                               Past
                               • A selection of early work


                               Present
                               • Current State of the Art
                                                                                                                       Albert Einstein
                               Future                                                                                     1879 - 1955
                               • Potential Research Directions for the Future
                                                                                                            “We can't solve problems by using the
                                                                                                             same kind of thinking we used when
                                                                                                                      we created them.”


                                                             Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Results on Standard Datasets
The University of Nottingham




                               •Many early papers investigated JSSP.
                               There is an opportunity to investigate if
                               the current state of the art is able to beat
                               these and set new benchmarks
                               •Why not apply hyper-heuristics to more
                               current benchmarks (TSP, VRP, QAP
                               etc.).




                                                        Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Benchmark datasets
The University of Nottingham




                               •We need to add to resources such as
                               OR-LIB so that we are able to compare
                               hyper-heuristic approaches.
                               •We need to have access to benchmarks
                               that are understandable, perceived as fair
                               and which are not open to many
                               interpretations.




                                                      Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Comparison against benchmarks
The University of Nottingham




                               •Using the “good enough, soon enough,
                               cheap enough” mantra we don’t claim to
                               be competitive with bespoke solutions,
                               but we are interested if we can beat best
                               known solutions.
                               •Why are some hyper-heuristics better
                               than others – and on what class of
                               problems?
                               •Robustness vs quality and how do we
                               measure that?


                                                      Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Ant Algorithm based Hyper-heuristics
The University of Nottingham




                               •Ant algorithms draw their inspiration
                               from the way ants forage for food.
                               •Two major elements to an ant
                               algorithm.
                                  •Pheromone values
                                  •Heuristic values




                                                      Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Ant Algorithm based hyper-heuristics
The University of Nottingham




                                     Trail             Visibility
                                   Intensity

                                                  Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Ant Algorithm based hyper-heuristics
The University of Nottingham




                                  Heuristic            Visibility
                                  Synergy

                                                  Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
“Good enough, soon enough, cheap
                                           enough”
The University of Nottingham




                               •What does this actually mean?
                               •Will the scientific community accept
                               that this is a fair way to compare results?




                                                                                                         Different Evaluations




                                                      Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
“Good enough, soon enough, cheap
                                           enough”
The University of Nottingham




                               •How do we know if a solution is “good
                               enough”?
                                  •User feedback?
                                  •Within a given value of best known
                                  solution?
                                  •We get bored running the                                               Not Good Enough!
                                  algorithm?
                                  •The cost of accepting the solution is
                                  acceptable?
                                  •Two evaluation mechanisms?
                                                     Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
“Good enough, soon enough, cheap
                                           enough”
The University of Nottingham




                               •How do we know if a solution is “soon
                               enough”?
                                  •Meet a critical deadline?
                                  •Run as long as we can?
                                  •Can be embedded in a realtime
                                  system?                                                                       Soon Enough!




                                                     Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
“Good enough, soon enough, cheap
                                           enough”
The University of Nottingham




                               •How do we know if a solution is
                               “cheap enough”?
                                  •Can be embedded in “off-the-shelf”
                                  software?
                                  •Development costs are significantly
                                  lower writing a bespoke system?
                                                                                                             Cheap Enough!
                                  •Can be run on a standard PC, rather
                                  than requiring specialised hardware?



                                                    Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Comparing Hyper-heuristics
The University of Nottingham




                               •How can we compare different hyper-
                               heuristics so that reviewers have a way
                               of fairly judging new contributions


                               •What do we mean by “One hyper-
                               heuristic is better than another”?




                                                     Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Anti-heuristics
The University of Nottingham




                               •There is/has been a significant amount
                               of research investigating how we can
                               “choose which heuristic to select at each
                               decision point”
                               •There could also be some benefit in
                               investigating hyper-heuristics that are
                               obviously bad and seeing if the hyper-
                               heuristic is able to learn/adapt not to use
                               them




                                                       Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Minimal Heuristics
The University of Nottingham




                               •Many of the hyper-heuristic papers
                               effectively say “choose a set of low level
                               heuristics…”
                               •But, can we define a minimal set of
                               heuristics that operate well across
                               different problems (e.g. add, delete and
                               swap)?




                                                      Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Evolve heuristics
The University of Nottingham




                               •We can ignore “choose a set of low level
                               heuristics…” if we can generate our own
                               set of human competitive heuristics
                               •We have utilised genetic programming
                               and adaptive constructive heuristics but
                               there remains lots of scope for further
                               investigation.




                                                      Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Co-evolution
The University of Nottingham




                               •Heuristics compete for survival
                               •Similarities with genetic algorithms etc.,
                               but there is a wide scope of possible
                               research in this area.




                                                                                                                       Arthur Samuel
                                                                                                                         1901 – 1990
                                                                                                                       An AI Pioneer

                                                      Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Hybridisations
The University of Nottingham




                               •Is there anything to be gained from
                               hybridising various methodologies?
                               •There has been success with exact
                               methods and meta-heuristics
                               •What about hybridising hyper-heuristics
                               with meta-heuristics, exact approaches,
                               user interaction etc?




                                                     Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
User interaction
The University of Nottingham




                               •How can users interact with hyper-
                               heuristics?
                                  •Introduce/delete heuristics as the
                                  search progresses?
                                  •Prohibit some areas of the search
                                  space?
                                  •Provide a time/quality trade off?




                                                     Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Framework
The University of Nottingham




                               •There is a large learning curve and high
                               buy-in to develop a hyper-heuristic
                               •Tools such as GA-LIB help the
                               community to utilise the tools and to
                               carry out research
                               •But, what should this framework enable
                               you to do? Choose heuristics, generate
                               heuristics?




                                                      Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
A unifying theory
The University of Nottingham




                               •What is the formal relationship between
                               heuristics, meta-heuristics and hyper-
                               heuristics (and even exact methods)?
                                                                                                             Stephen Hawking
                                                                                                                         1942 -




                                                     Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
A unifying theory
The University of Nottingham




                               •Can we analyse the landscape of the
                               different search methodologies?
                               •Can we move between different search                                         Stephen Hawking
                               spaces during the search?
                                                                                                                         1942 -




                                                     Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
A unifying theory
The University of Nottingham




                               •Can we offer convergence guarantees?
                               •Can we offer guarantees of solution
                               quality and/or robustness?
                                                                                                             Stephen Hawking
                                                                                                                         1942 -




                                                     Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
Questions/Discussion
The University of Nottingham




                                          Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)

Más contenido relacionado

Destacado

5B_1_Neogeography for the rural urban classification of england and wales
5B_1_Neogeography for the rural urban classification of england and wales5B_1_Neogeography for the rural urban classification of england and wales
5B_1_Neogeography for the rural urban classification of england and walesGISRUK conference
 
True trends by_truecar_april-2010 v2
True trends by_truecar_april-2010 v2True trends by_truecar_april-2010 v2
True trends by_truecar_april-2010 v2TrueCar
 
Ref Share – Sharing Your Ref Works Database
Ref Share – Sharing Your Ref Works DatabaseRef Share – Sharing Your Ref Works Database
Ref Share – Sharing Your Ref Works DatabaseUCT
 
CV-Online efektyvaus darbo skelbimo receptas
CV-Online efektyvaus darbo skelbimo receptasCV-Online efektyvaus darbo skelbimo receptas
CV-Online efektyvaus darbo skelbimo receptasRita Karavaitiene
 
Steelcraft jaguar
Steelcraft jaguarSteelcraft jaguar
Steelcraft jaguarsteelcraft
 
4 music contents page
4 music  contents page4 music  contents page
4 music contents pageowenbailey25
 
S-E-Tom_Mitchell-19448-431-1020366feb2016
S-E-Tom_Mitchell-19448-431-1020366feb2016S-E-Tom_Mitchell-19448-431-1020366feb2016
S-E-Tom_Mitchell-19448-431-1020366feb2016Tom Mitchell
 

Destacado (11)

5B_1_Neogeography for the rural urban classification of england and wales
5B_1_Neogeography for the rural urban classification of england and wales5B_1_Neogeography for the rural urban classification of england and wales
5B_1_Neogeography for the rural urban classification of england and wales
 
True trends by_truecar_april-2010 v2
True trends by_truecar_april-2010 v2True trends by_truecar_april-2010 v2
True trends by_truecar_april-2010 v2
 
Ref Share – Sharing Your Ref Works Database
Ref Share – Sharing Your Ref Works DatabaseRef Share – Sharing Your Ref Works Database
Ref Share – Sharing Your Ref Works Database
 
3B_4_Rate-my-place
3B_4_Rate-my-place3B_4_Rate-my-place
3B_4_Rate-my-place
 
CSR live project
CSR live projectCSR live project
CSR live project
 
CV-Online efektyvaus darbo skelbimo receptas
CV-Online efektyvaus darbo skelbimo receptasCV-Online efektyvaus darbo skelbimo receptas
CV-Online efektyvaus darbo skelbimo receptas
 
Steelcraft jaguar
Steelcraft jaguarSteelcraft jaguar
Steelcraft jaguar
 
EXPOSICIÓN nº 4
EXPOSICIÓN nº 4EXPOSICIÓN nº 4
EXPOSICIÓN nº 4
 
4 music contents page
4 music  contents page4 music  contents page
4 music contents page
 
Songs for 2nd service
Songs for 2nd serviceSongs for 2nd service
Songs for 2nd service
 
S-E-Tom_Mitchell-19448-431-1020366feb2016
S-E-Tom_Mitchell-19448-431-1020366feb2016S-E-Tom_Mitchell-19448-431-1020366feb2016
S-E-Tom_Mitchell-19448-431-1020366feb2016
 

Hyperheuritics: Past, Present and Future

  • 1. Hyper-heuristics: Past Present and Future The University of Nottingham Graham Kendall gxk@cs.nott.ac.uk Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 2. Contents The University of Nottingham Past • A selection of early work Present • Current State of the Art Albert Einstein Future 1879 - 1955 • Potential Research Directions for the Future “We can't solve problems by using the same kind of thinking we used when we created them.” Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 3. Contents The University of Nottingham Past • A selection of early work Present • Current State of the Art Albert Einstein Future 1879 - 1955 • Potential Research Directions for the Future “We can't solve problems by using the same kind of thinking we used when we created them.” Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 4. The University of Nottingham Fisher H. and Thompson G.L. (1963) Probabilistic Learning Combinations of Local Job-shop Scheduling Rules. In Muth J.F. and Thompson G.L. (eds) Industrial Scheduling, Prentice Hall Inc., New Jersey, 225-251 Based on (I assume) Fisher H. and Thompson G.L. (1961) Probabilistic Learning Combinations of Local Job-shop Scheduling Rules. In Factory Scheduling Conference, Carnegie Institute of Technology Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 5. Good Facility Order Matrix The University of Nottingham Number 1 3(1) 1(3) 2(6) 4(7) 6(3) 5(6) 2 2(8) 3(5) 5(10) 6(10) 1(10) 4(4) 3 3(5) 4(4) 6(8) 1(9) 2(1) 5(7) 4 2(5) 1(5) 3(5) 4(3) 5(8) 6(9) 5 3(9) 2(3) 5(5) 6(4) 1(3) 4(1) 6 2(3) 4(3) 6(9) 1(10) 5(4) 3(1) 6 x 6*6 Test Problem (times in brackets) “The number of feasible active schedules is, by a conservative estimate, well over a million, so their complete enumeration is out of the question.” • Also 10 (jobs) x 10 (operations) and 20 (jobs) x 5 (operations) problems Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 6. Good Facility Order Matrix The University of Nottingham Number 1 3(1) 1(3) 2(6) 4(7) 6(3) 5(6) 2 2(8) 3(5) 5(10) 6(10) 1(10) 4(4) 3 3(5) 4(4) 6(8) 1(9) 2(1) 5(7) 4 2(5) 1(5) 3(5) 4(3) 5(8) 6(9) 5 3(9) 2(3) 5(5) 6(4) 1(3) 4(1) 6 2(3) 4(3) 6(9) 1(10) 5(4) 3(1) 6 x 6*6 Test Problem (times in brackets) Job 3, 1, 2, 5, 4, 6 Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 7. Good Facility Order Matrix The University of Nottingham Number 1 3(1) 1(3) 2(6) 4(7) 6(3) 5(6) 2 2(8) 3(5) 5(10) 6(10) 1(10) 4(4) 3 3(5) 4(4) 6(8) 1(9) 2(1) 5(7) 4 2(5) 1(5) 3(5) 4(3) 5(8) 6(9) 5 3(9) 2(3) 5(5) 6(4) 1(3) 4(1) 6 2(3) 4(3) 6(9) 1(10) 5(4) 3(1) 6 x 6*6 Test Problem (times in brackets) • Two Rules • SIO: Shortest Imminent Operation (“First on, First Off”) • LRT: Longest Remaining Time • Only require knowledge of “your” machine Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 8. • Monte Carlo: 58 time Units The University of Nottingham • SIO: 67 time units • LRT: 61 time units • Optimal: 55 time units • SIO should be used initially (get the machines to start work) and LRT later (work on the longest jobs) • Why not combine the two heuristics? • Four learning models, rewarding good heuristic selection Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 9. Remarks The University of Nottingham • Not sure about reproducibility (e.g. reward/punishment functions) • An unbiased random combination of scheduling rules is better than any of them taken separately • “Learning is possible, but there is a question as to whether learning is desirable given the effectiveness of the random combination” • “It is not clear what is being learnt as the original conjecture was not strongly supported” • “It is likely that combinations of 5-10 rules would out-perform humans” Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 10. The University of Nottingham Fang H-L., Ross P. and Corne D. (1993) A Promising genetic Algorithm Approach to Job-Shop Scheduling, Reschecduling, and Open-Shop Scheduling Problems. In Forrest S. (ed) Fifth International Conference on Genetic Algorithms, Morgan Kaufmann, San Mateo, 375-383 Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 11. Representation The University of Nottingham • For a j x m problem, a string represents j x m chunks. • The chunk is atomic from a GA perspective. • The chunks abc means to put the first untackled task of the ath uncompleted job into the earliest place it will fit in the developing schedule, then put the bth uncompleted job into …. • A schedule builder decodes the chromosome. • Fairly standard GA e.g. population size of 500, rank based selection, elitism, 300 generations, crossover rate 0.6, adaptive mutation rate Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 12. Other Remarks The University of Nottingham • Considered Job-Shop Scheduling and Open- Shop Scheduling • Experimented with different GA parameters • Results compared favourably with best known or optimal Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 13. The University of Nottingham Denzinger J. and Fuchs M. (1997) High Performance ATP Systems by Combining Several AI Methods. In proceedings of the Fifteenth International Joint Conference on Artificial Intelligence (IJCAI 97), 102-107 Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 14. Remarks The University of Nottingham • The first paper to use the term Hyper-heuristic • Used in the context of an automated theorem prover • A hyper-heuristic stores all the information necessary to reproduce a certain part of the proof and is used instead of a single heuristic Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 15. The University of Nottingham O’Grady P.J. and Harrison (1985) A General Search Sequencing Rule for Job Shop Sequencing. International Journal of Production Research, 23(5), 961-973 Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 16. Remarks The University of Nottingham Pi = (Ai x Ti) + (Bi x Si) where Pi the priority index for job i at its current stage Ai a 1 x m coefficient vector for job i Ti a m x 1 vector which contains the remaining operation times for job i in process order Bi the due date priority coefficient for job i Si the due date slack for job i m the maximum number of processing stages for jobs 1 to i Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 17. Remarks Pi = (Ai x Ti) + (Bi x Si) The University of Nottingham where Pi the priority index for job i at its current stage Ai a 1 x m coefficient vector for job i Ti a m x 1 vector which contains the remaining operation times for job i in process order Bi the due date priority coefficient for job i Si the due date slack for job i m the maximum number of processing stages for jobs 1 to i A = (1,0,0,0,0,…,0), B = 0 Shortest Imminent Operation Time A = (0,0,0,0,0,…,0), B = 1 Due Date Sequencing Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 18. Remarks Pi = (Ai x Ti) + (Bi x Si) The University of Nottingham where Pi the priority index for job i at its current stage Ai a 1 x m coefficient vector for job i Ti a m x 1 vector which contains the remaining operation times for job i in process order Bi the due date priority coefficient for job I Si the due date slack for job i m the maximum number of processing stages for jobs 1 to i A search is performed over Ai and Bi in order to cause changes in the processing sequences. Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 19. The University of Nottingham Norenkov I. P. and Goodman E D. (1997) Solving Scheduling Problems via Evolutionary Methods for Rule Sequence Optimization. In proceedings of the 2nd World Conference on Soft Computing (WSC2) Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 20. Remarks The University of Nottingham • Similar in idea to Fang, Ross and Corne (1994) • The allele at the ith position is the heuristic to be applied at the ith step of the scheduling process. • Comparison with using eight single heuristics and the Heuristic Combination Method (HCM) was found to be superior. Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 21. Other (Selected) Papers The University of Nottingham • Crowston W.B., Glover F., Thompson G.L. and Trawick J.D. (1963) Probabilistic and Parameter Learning Combinations of Local Job Shop Scheduling Rules. ONR Research Memorandum, GSIA, Carnegie Mellon University • Storer R.H., Wu S.D. and Vaccari R. (1992) New Search Spaces for Sequencing Problems with Application to Job Shop Scheduling. Management Science, 38(10), 1495-1509 • Battiti R. (1996) Reactive Search: Toward Self Tuning Heuristics. In Rayward-Smith R.J., Osman I.H., Reeves C.R. and Smith G.D. (eds) Modern Heuristics Search methods, John Wiley, 61-83 Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 22. Contents The University of Nottingham Past • A selection of early work Present (Heuristics to Choose Heuristics) • Current State of the Art Albert Einstein Future 1879 - 1955 • Potential Research Directions for the Future “We can't solve problems by using the same kind of thinking we used when we created them.” Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 23. Heuristics to Choose Heuristics The University of Nottingham Hyper-heuristic Data flow Domain Barrier Data flow Set of low level heuristics H1 H2 Hn …… Evaluation Function Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 24. Choice Function The University of Nottingham • f1 + f2 + f3 • f1 = How well has each heuristic performed • f2 = How well have pairs of heuristics performed • f3 = Time since last called Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 25. Tabu Search The University of Nottingham • Low level heuristics compete with each other • Recent heuristics are made tabu • Rank low level heuristics based on their estimated performance potential Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 26. Case Based Heuristic Selection The University of Nottingham • Find heuristics that worked well in previous similar problem solving situations • Features discovered in similarity measure – key research issue Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 27. Adaptive Ordering Strategies The University of Nottingham • Based on Squeaky Wheel Optimisation • Consider constructive heuristics as orderings • Adapt the ordering by a heuristic modifier according to the penalty imposed by certain features • Generative Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 28. Contents The University of Nottingham Past • A selection of early work Present (Generating Heuristics) • Current State of the Art Future • Potential Research Directions for the Future Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 29. Generating heuristics The University of Nottingham Hyper-heuristic • Rather than supply a set of low level heuristics, generate the Data flow heuristics automatically Domain Barrier • Heuristics could be one off Data flow (disposal) heuristics or could be applicable to many problem instances Set of low level heuristics H1 H2 Hn …… Evaluation Function Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 30. Generating heuristics The University of Nottingham Burke E. K., Hyde M. and Kendall G. Evolving Bin Packing Heuristics With Genetic Programming. In Proceedings of the 9th International Conference on Problem Parallel Solving from Nature (PPSN 2006), pp 860-869, LNCS 4193, Reykjavik, Iceland, 9-13 Sepetmber 2006 Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 31. Generating heuristics The University of Nottingham • Evolves a control program that decides whether to put a given piece into a given bin • First-fit heuristic evolved from Genetic Programming without human input on benchmark instances For each piece, p, not yet packed For each bin, i output = evaluate(p, fullness of i, capacity of i) if (output > 0) place piece p in bin i break fi End For End For Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 32. Contents The University of Nottingham Past • A selection of early work Present • Current State of the Art Albert Einstein Future 1879 - 1955 • Potential Research Directions for the Future “We can't solve problems by using the same kind of thinking we used when we created them.” Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 33. Results on Standard Datasets The University of Nottingham •Many early papers investigated JSSP. There is an opportunity to investigate if the current state of the art is able to beat these and set new benchmarks •Why not apply hyper-heuristics to more current benchmarks (TSP, VRP, QAP etc.). Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 34. Benchmark datasets The University of Nottingham •We need to add to resources such as OR-LIB so that we are able to compare hyper-heuristic approaches. •We need to have access to benchmarks that are understandable, perceived as fair and which are not open to many interpretations. Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 35. Comparison against benchmarks The University of Nottingham •Using the “good enough, soon enough, cheap enough” mantra we don’t claim to be competitive with bespoke solutions, but we are interested if we can beat best known solutions. •Why are some hyper-heuristics better than others – and on what class of problems? •Robustness vs quality and how do we measure that? Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 36. Ant Algorithm based Hyper-heuristics The University of Nottingham •Ant algorithms draw their inspiration from the way ants forage for food. •Two major elements to an ant algorithm. •Pheromone values •Heuristic values Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 37. Ant Algorithm based hyper-heuristics The University of Nottingham Trail Visibility Intensity Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 38. Ant Algorithm based hyper-heuristics The University of Nottingham Heuristic Visibility Synergy Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 39. “Good enough, soon enough, cheap enough” The University of Nottingham •What does this actually mean? •Will the scientific community accept that this is a fair way to compare results? Different Evaluations Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 40. “Good enough, soon enough, cheap enough” The University of Nottingham •How do we know if a solution is “good enough”? •User feedback? •Within a given value of best known solution? •We get bored running the Not Good Enough! algorithm? •The cost of accepting the solution is acceptable? •Two evaluation mechanisms? Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 41. “Good enough, soon enough, cheap enough” The University of Nottingham •How do we know if a solution is “soon enough”? •Meet a critical deadline? •Run as long as we can? •Can be embedded in a realtime system? Soon Enough! Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 42. “Good enough, soon enough, cheap enough” The University of Nottingham •How do we know if a solution is “cheap enough”? •Can be embedded in “off-the-shelf” software? •Development costs are significantly lower writing a bespoke system? Cheap Enough! •Can be run on a standard PC, rather than requiring specialised hardware? Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 43. Comparing Hyper-heuristics The University of Nottingham •How can we compare different hyper- heuristics so that reviewers have a way of fairly judging new contributions •What do we mean by “One hyper- heuristic is better than another”? Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 44. Anti-heuristics The University of Nottingham •There is/has been a significant amount of research investigating how we can “choose which heuristic to select at each decision point” •There could also be some benefit in investigating hyper-heuristics that are obviously bad and seeing if the hyper- heuristic is able to learn/adapt not to use them Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 45. Minimal Heuristics The University of Nottingham •Many of the hyper-heuristic papers effectively say “choose a set of low level heuristics…” •But, can we define a minimal set of heuristics that operate well across different problems (e.g. add, delete and swap)? Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 46. Evolve heuristics The University of Nottingham •We can ignore “choose a set of low level heuristics…” if we can generate our own set of human competitive heuristics •We have utilised genetic programming and adaptive constructive heuristics but there remains lots of scope for further investigation. Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 47. Co-evolution The University of Nottingham •Heuristics compete for survival •Similarities with genetic algorithms etc., but there is a wide scope of possible research in this area. Arthur Samuel 1901 – 1990 An AI Pioneer Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 48. Hybridisations The University of Nottingham •Is there anything to be gained from hybridising various methodologies? •There has been success with exact methods and meta-heuristics •What about hybridising hyper-heuristics with meta-heuristics, exact approaches, user interaction etc? Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 49. User interaction The University of Nottingham •How can users interact with hyper- heuristics? •Introduce/delete heuristics as the search progresses? •Prohibit some areas of the search space? •Provide a time/quality trade off? Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 50. Framework The University of Nottingham •There is a large learning curve and high buy-in to develop a hyper-heuristic •Tools such as GA-LIB help the community to utilise the tools and to carry out research •But, what should this framework enable you to do? Choose heuristics, generate heuristics? Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 51. A unifying theory The University of Nottingham •What is the formal relationship between heuristics, meta-heuristics and hyper- heuristics (and even exact methods)? Stephen Hawking 1942 - Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 52. A unifying theory The University of Nottingham •Can we analyse the landscape of the different search methodologies? •Can we move between different search Stephen Hawking spaces during the search? 1942 - Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 53. A unifying theory The University of Nottingham •Can we offer convergence guarantees? •Can we offer guarantees of solution quality and/or robustness? Stephen Hawking 1942 - Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)
  • 54. Questions/Discussion The University of Nottingham Graham Kendall, Hyper-heuristics: Past, Present and Future (uploaded to Slideshare.com : 25th April 2010)