Las cosas que voy a investigar aquí.

1. Combining several things (new and improved)
Pablo García Sánchez - University of Granada
(but now in Vrije Universiteit Amsterdam)
pgarcia@atc.ugr.es

2. Outline
• What I am investigating in Granada
• EvAg and MultiKulti
• Evolutionary Robotics
• μ+1 online
• μ+1 online Vs. EvAg (Robert-Jan)
• Global Parent Selector or HybridEvag (Berend-Luis)
• μ+EvAg
• EDEA (?)
• Some ideas about MultiKulti + EvAg

3. My Work
• SOAP
• BPEL
• WSDL
• JBI
• OSGi
• XPATH
• XSLT
• OMG!
• WTF?

4. My Work
• SOAP
• BPEL
But in my spare time...
• WSDL
- XSLT Evolution
• JBI
- Service Oriented Evolutionary Algorithms
• OSGi
• XPATH
• XSLT
• OMG!
• WTF?

5. EvAg

6. EvAg Pseudo-code
Sactual←Initialize Agent
loop
Sols←LocalSelection(NeighborsEvAg)
Snew←Recombination(Sols,Pc)
Snew’←Mutation(Snew,Pm)
Evaluate(Snew’)
if Snew better than Sactual then
Sactual←Snew’
end if
end loop

7. Newscast
• Is the population structure used by Juanlu
• It is the cache of Neighbours. Each entry has:
• Reference to the node
• Time-stamp of the entry creation
• Agent identifier
• Specific application data (?)
• Every cycle EvAgi initiates cache exchange
• Selects randomly a neighbour EvAgj of its cache
• Aggregation of caches. JuanLu uses:
• the c freshest items. i and j share the same cache

8. Newscast pseudocode
• Active mode:
loop
EvAgj←Uniformily random selected node from Cachei
Send Cachei to EvAgj
Receive Cachej from EvAgj
Cachei←Aggregate(Cachei,Cachej)
endloop
• Pasive mode
loop
Wait Cachej from EvAgj
Receive Cachei to EvAgj
Cachei←Aggregate(Cachei,Cachej)
endloop

9. EvAg (Results)
• Island model: results are sensitive to the number of islands→vulnerability in
dynamic environments (like P2P)
• Outperforms tuned GAs
• Less links (no panmictic)
• C newest items of the both caches are kept (TTL). c<<N

10. MultiKulti

11. MultiKulti (results)
• MK policies outperform the classic policies (send the best or the random, that
never reach the optimum)
• Specially with low number of individuals but larger number of islands
• It is (a bit) better to send the consensus as a representation
• It is better to send the most different of an elite as response
• Problem: difficult to caracterize a population using a string

12. Evolutionary Robotics
• Taxonomy:
• When? On-Line vs. off-line
• Where? On-board vs. Off-board
• How? Encapsulated vs. distributed (and hybrid!)

13. μ+1 online
• Local population
• No migration
• Mantains the μ fittest ones
• Sometimes re-evaluates

14. EvAgRobot (Robert-Jan - EA2011 October)
• Added genotype exchanges using newscast. Difference: last active solutions
in caches, not only connections

15. Comparison μ+1 vs. EvAg
• Sensibility to parameter tuning: μ+1 is reliable. EvAg is reliable when the
number of robots increases
• EvAg Panmictic vs EvAg newscast: the newscast performance is worst, but in
a panmictic case a large number of peers could be difficult to manage in real-
case scenarios
• Doubt: in newscast is it the topology fixed? Only active solutions are
exchanged among neighbours or also the connections? (so a peer can
exchange its new cache to new neighbours)

16. HybridEvag (Berend-Luis Approach)

17. Source code: PopulationStructure.cpp
PopulationMember pop; //HivemindInstance or NewscastInstance
Controller[] activeSolutions //although localPopulation is a better name
Controller getNextCandidate(){
currentTime++
pop->update(currentTime)
if(reevaluate) return c else...
parentA and B = BinaryTournament(pop)
crossover, mutation...
return new candidate
}
SetFitness(Controller candidate, double fitness){
if (reevaluated) candidate->fitness = (fitness + oldfitness)/2
else
replace the worst of activeSolutions if candidate is better
if (accepted || reevaluated)
pop->setNews(acceptedCandidate, pos, currentTime)
}

18. HiveMind.cpp
static vector <vector <newsitems>> allItems;
update() // not implemented
setNews(Controller c, pos, currenttime){
stores the c in the static (shared) vector (of accepted solutions)
}

19. NewscastInstance.cpp
update(int currentTime){
delete older than “currentTime - itemTTL” from myCache
neighbour = select random peer of myCache
otherCache = neighbour->exchange(myCache, currentTime) //caches
CLONING the objects (no shared data)
}
setNews(Candidate c, int currentTime){
stores the last accepted candidate (not the best!) in the cache with my
timestamp
}
BEFORE SEND: 1) Remove older items of my cache 2) Remove duplicates
AFTER RECEIVE: 1) Mantain the cacheSize freshest solution of each node.
DOUBT: this timestamp is different!

20. μ+EvAg
• Combining previous approaches in a parameter-based algorithm with two
parameters:
•μ
• rangeRate (parent selection global or local) (Note: no information about
newscast, or TTL, is given in the Robert’s thesis)
• Change from one to another easily
• Allows a whole parameter tuning

21. EDEA (Georgios)
• One solution per robot
• Two kinds of mating for each robot:
• Eagerly: sends genome to the partner without asking
• Selectively: Binary Tournament from other peers and accepted if fitness is
quite good enough: if(otherF/myF*Sc <prob())
• In GECCO‘11→panmictic→Random from all robots (experiments with an
small set)

22. Differences
Algorithm Population Migration Ellitism Type
μ+1 Local population None Replace the worst Centralized
EvAg 1 Solution
Selection from actual
Replace if better Distributed
neighbors
(Juanlu)
EvAg 1 Active Solution +
Cache swapping
Replace the active
Distributed??
Cache solution if better
(Robert)
Global (panmictic) Replace the worst (local
HybridEvAg Local population
parent selection survivor selection)
Hybrid
Previous algorithms combined using two parameters: μ (population) and
μ+Evag rangeRate (parent selection global or local)
Hybrid
Sends eagerly, receives
New solution can be
EDEA One solution selectively (panmictic in
worst
Distributed
GECCO’11)

23. Differences
Few Local genomes Many Few Local genomes Many
Never mU+1 Never
Genome Genome
exchange exchange
Always Always EvAg
Few Local genomes Many Few Local genomes Many
Never Never
Genome Genome
exchange exchange mu+EvAg
Always Hybrid Evag Always

24. Differences (Population)
• μ+1: Local population of μ individuals
• Juanlu EvAg: 1 Best Solution + Selection from the neighbours. Topology:
newscast
• Robert EvAg: 1 Active Solution + cache of other (less diverssity)→Finally,
behaves like a whole population
• HybridEvag: local population becomes global population (panmictic) for
parent selector (more diversity: not only active solutions are used) and local
population for replacement
• All algorithms are special cases of μ+EvAg (Robert thesis)→(few or many
local genomes AND never or always genome exchange from the global pool)

25. Differences (Parent Selection)
• μ+1: from the local population
• JuanLu EvAg: selected from current neighbours (open)
• Robert EvAg: binary tournament of active solutions (obtained from the cache)
• Hybrid EvAg Approach: binary tournament of ALL populations (not the active
ones)

26. Differences (Ellitism)
• mu+1: Replace the worst in the local population
• EvAg (Robert and Juanlu): Replace the actual solution if the new is better,
otherwise is dropped
• HybridEvag: replaces the worst individual if better

27. Ideas (I)
• Distributed Islands with MultiKulti but without EvAg (using local populations)
• Which structure is better? (ring, P2P, grid, random...)
• Which migration policy is better? (best, elite...)
• Compare:
• All islands with the tuned parameter settings (homogeneous)
• All island with random parameter settings (heterogeneous)
• What about robot failures?

28. Ideas (II) Combining EvAg and Multikulti?
• Combine MultiKulti with EvAg (could be difficult)
• EvAg: keeps one individual + exchange of several
• MultiKulti: keeps several individuals + exchange of one
• So, using EvAg...
• Binary Tournaments from the caches but giving priority to the most different
instead the most fitted?
• Sending the most different individuals in cache exchange/aggregating
ordered by differences, instead timestamp
• Problems:
• Lost of information (is not the EvAg philosophy)
• Caches not too large