7. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
ANT COLONY OPTIMIZATION
A valuable technique for mathematical optimization.
Takes inspiration from swarming behavior of birds, animals
or insects.
Useful for discrete and continuous optimization problems.
In telecommunications: Routing and load balancing.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
8. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
BIOLOGICAL INSPIRATION
Inception – early 90’s.
Proposed by Kennedy and Eberhardt.
Social psychologist and electrical engineer.
Based on observation of bird flocks searching for corn.
Birds are social animals.
Birds are also driven by the goal of community survival
rather than being focused on survival of the individuals.
Bird’s foraging behavior: How birds swarm together as they
search for food.
Convergence: How the whole swarm converges to good
corn fields (optimum solutions)
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
9. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
SWARMING BEHAVIOR OF BIRDS
When searching for food birds:
As a single bird finds a good corn source.
Other birds try to converge to it so that they can also grab
some food.
Which other birds: They are the neighboring birds.
Who are the neighbors: Neighborhood functions.
Birds drift together probabilistically, meaning sometimes
they move closer and sometimes they lurch away.
Benefit: Better exploration of corn fields for good corn.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
10. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
BENEFITS
Indirect communication between birds enables them to
converge to better food sources.
Random probabilistic search enables them to find better,
globally optimal, food sources as opposed to substandard,
locally optimal, ones.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
11. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
BASIC IDEAS
A number of simple entities – particles – are placed in the
search space of some problem or function.
Each particle evaluates the objective function at its current
location.
Each particle determines its movement through the search
space by:
1 Combining some aspect of the history of its own current
and best locations with those of one or more members of
the swarm.
2 And with some random perturbations.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
12. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
THE ALGORITHM
The next iteration takes place when all particles have been
moved.
Eventually the swarm as a whole is likely to move close to
an optimum of the fitness function.
Like a flock of birds foraging for food.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
13. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
DATA STRUCTURES
Each individual is composed of three D-dimensional
vectors.
D is the dimensionality of the search space.
The vectors are: The current position xi , the previous best
position pi, and the velocity vi .
The current position xi can be considered as a set of
coordinates describing a point in space.
On each iteration of the algorithm, the current position is
evaluated as a problem solution.
If that position is better than any that has been found so far,
then the coordinates are stored in the second vector, pi.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
14. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
USING DATA STRUCTURES
The value of the best function result so far is stored in a
variable that can be called pbesti (for previous best)., for
comparison on latter iterations.
The objective, of course, is to keep finding better positions
and updating pi and pbesti.
New points are chosen by adding vi coordinates to xi , and
the algorithm operates by adjusting vi .
vi is effectively the step size.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
15. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
WHY SWARMING IS IMPORTANT?
The particle swarm is more than just a collection of
particles.
A particle itself has almost no power to solve any problem.
Progress occurs only when particles interact.
Problem solving is a population-wide phenomenon.
It emerges from the individual behaviors of the particles
through their interactions.
In any case, populations are organized according to some
sort of communication structure or topology.
This is often thought of as a social network.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
16. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
SWARM TOPOLOGY
The topology typically consists of bidirectional edges
connecting pairs of particles.
So that if j is in i’s neighborhood, i is also in j’s.
Each particle communicates with some other particles.
And is affected by the best point found by any member of
its topological neighborhood.
This is just the vector pi for that best neighbor.
We denote this with pg.
The potential kinds of population ""social networks" are
hugely varied.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
17. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
SWARM TOPOLOGY
In practice certain types have been used more frequently.
Velocity is iteratively adjusted to allow the particles to
oscillate.
Topologies can be static and dynamic depending on the
problem.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
18. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
THE ALGORITHM – PSEUDOCODE I
1 Initialize a population array of particles with random
positions and velocities and D-dimensions in the search
space.
2 Begin loop:
3 For each particle, evaluate the desired optimization fitness
function in D variables.
4 Compare particleÕs fitness evaluation with its pbesti. If
current value is better than pbesti , then set pbesti equal to
the current value, and pi equal to the current location xi in
D-dimensional space.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
19. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
THE ALGORITHM – PSEUDOCODE II
5 Identify the particle in the neighborhood with the best
success so far, and assign its index to the variable g.
6 Change the velocity and position of the particle.
vi ← vi + U(0, φ1) ⊗ (pi − xi ) + U(0, φ2) ⊗ (pg − xi ) (1)
xi ← xi + vi (2)
7 If a criterion is met (usually a sufficiently good fitness or a
maximum number of iterations), exit loop.
8 End loop
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
20. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
THE ALGORITHM – PSEUDOCODE III
Where:
U(0, φ1) represents a vector of random numbers uniformly
distributed in [0, φi ] which is randomly generated at each
iteration and for each particle.
⊗ is component-wise multiplication.
In the original version of PSO, each component of vi is
kept within the range [−Vmax , +Vmax ]
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
22. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
APPLICATIONS
Image and video analysis.
Design and restructuring of electricity networks and load
dispatching.
Control applications.
Applications in electronics and electromagnetics.
Antenna design.
Power generation and power systems.
Scheduling.
Design applications.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
23. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
APPLICATIONS
Design and optimization of communication networks.
Biological, medical and pharmaceutical.
Clustering, classification and data mining.
Fuzzy and neuro-fuzzy systems and control.
Signal processing.
Neural networks.
Combinatorial optimization problems.
Robotics.
Prediction and forecasting.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
24. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
APPLICATIONS
Modeling.
Detection and diagnosis of faults and recovery from them.
Sensors and sensor networks.
Applications in computer graphics and visualization.
Design or optimization of engines and electrical motors.
Applications in metallurgy.
Music generation and games.
Security and military applications.
Finance and economics.
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications
25. Introduction
Biological Inspiration
The Algorithm
Applications
Conclusions
CONCLUSIONS
A great algorithm.
Bio-inspiration is the key.
Emulation of real bird swarming behavior..
Easy to comprehend.
Many variants.
Many applications.
Problem formulation is the real trick.
Inspiration (reference): Particle Swarm Optimization,
Riccardo Poli, James Kennedy and Tim Blackwell
Muhammad Adil Raja Particle Swarm Optimization: Algorithm and Applications