This document provides an overview of swarm robotics. It begins with examples of decentralized control and self-organization in natural swarms like ants and bees. It then discusses how swarm robotics takes inspiration from these systems, using local control methods, local communication, and self-organization to complete collective tasks without centralized control. The rest of the document focuses on a proposed system for gesture recognition to allow human control of swarm robots. It describes hand detection, feature extraction, and hardware implementation using three foot-bot robots. It concludes with potential applications of swarm robotics and areas for future work.
2. PRESENTATION OVERVIEW
Introduction To Gestures
Design Of The Proposed System
Hand Detection
Hardware Implementation
Conclusion / Future Work
4. Natural swarms
Decentralised – no-one in control
Individuals are simple and autonomous
Local communication and control
Cooperative behaviours emerge through self-
organisation
e.g. repairing damage to nest, foraging for food,
caring for brood
10. Robots
• Collective task completion
• No need for overly complex
algorithms
• Adaptable to changing environment
11. Swarm robotics
Inspired by self-organisation of social insects
Using local methods of control and communication
Local control: autonomous operation
Local communication: avoids bottlenecks
Scalable – new robots can be added, or fail without need for
recalibration
Simplicity – cheap, expendable robots
Self-organisation
Decentralisation
13. Introduction To Gestures
Gestures can originate from any body
motion.
Commonly from face/hand.
Gesture recognition-understand human
body language.
Help human to interact with machines
without any mechanical devices.
14. Disadvantages of centralised
control and communication.
Central control: failure of controller implies
failure of whole system
Robot to robot communication becomes very
complex as number of robots increases.
Communication bottlenecks
Adding new robots means changing the
communication and control system
15. Design Of The Proposed System
Hand detection
Feature extraction
Gesture recognition
Goal directed navigation of swarm
robots.
16. Hand Detection
Hand detection
– Detection of hand in an image, background objects are
avoided for feature extraction.
– Skin color is the key component.
– Detecting skin and non-skin.
– Detecting image pixels and regions that contains skin-
tone color.
– Background is controlled.
– Appearance depends on illumination conditions.
Two phases
– Training phase
– Detection phase
17. Training Phase
Three steps
– Collecting a database of skin patches from different
images
– Choosing a suitable color space
– Learning the parameters of skin classifier
18. Detection Phase
Two steps
Converting the image into some color space that was
used in training phase.
Classifying each pixel using the skin classifier to either
a skin or non-skin.
RGB color space
Skin classifier
Variety of classification techniques
Any pixel which color falls inside the skin color class
boundary is labeled as skin.
19. Feature Extraction
Feature-An interesting part of an image.
No exact definition.
Depends on the problem.
Transforming the input data into set of features.
Result is a feature vector.
Features extracted are invariant to image scaling,
rotation and less affected to changes in
illumination.
SIFT feature extraction.
25. Applications of swarm approach
Some tasks are particularly suited to group of expendable
simple robots
e.g. - cleaning up toxic waste
- exploring an unknown planet
- pushing large objects
- surveillance and other military applications
26. Conclusion
Hand detection and feature extraction removes
noise from the image.
System performance and accuracy will
increase.
Swarm robots movement can be controlled
through gestures.
Future Work
Speech recognition.
28. Satellite
Maintenance
The Future?
Medical
Interacting
Chips in
Mundane Objects
Cleaning Ship
Hulls
Pipe
Inspection
Pest Eradication
M
iniaturization
EngineMaintenance
Telecommunications
Self-Assem
bling
Robots
Job Scheduling
Vehicle
Routing
Data
Clustering
Distributed
M
ail
System
s
O
ptim
al
Resource
Allocation
Combinatorial
Optimization
29. 1. C.C.Wang, K.C.Wang.: Hand Posture Recognition Using Adaboost with SIFT For Human
Robot Interaction, in Robotics: Viable Robotic Service to Human, Springer-2009.
2. M. Kolsch and M. Turk.: Robust hand detection, in IEEE International Conference on
Automatic Face and Gesture Recognition, 2004.
3. Cristina Manresa, Javier Varona, Ramon Mas and Francisco J.Perales,.: Hand Tracking
and Gesture Recognition for Human-Computer Interaction , Electronic Letters on
Computer Vision and Image Analysis 5(3):96-104, 2005.
4. Alessandro Giusti, Jawad Nagi, Luca M. Gambardella, Gianni A. Di Caro : Distributed
Consensus for Interaction between Humans and Mobile Robot Swarms (Demonstration).
5. Ihab Zaqout, Roziati Zainuddin, Sapian Baba,.: Pixel-Based Skin Color Detection
Technique, in Machine Graphics and Vision, 2005. FLEXChip Signal Processor
(MC68175/D), Motorola, 1996.
6. Qiu-yu Zhang, Mo-yi Zhang, Jian-qiang Hu,.: Hand Gesture Contour Tracking Based on
Skin Color Probability and State Estimation Model, Journal of Multimedia, Vol. 4, No. 6,
December 2009. A. Karnik, “Performance of TCP congestion control with rate feedback:
TCP/ABR and rate adaptive TCP/IP,” M. Eng. thesis, Indian Institute of Science,
Bangalore, India, Jan. 1999.
7. Lars Bretzner, Ivan Laptev, Tony Lindberg,.: Hand Gesture Recognition using Multi-Scale
Colour Features, Hierarchical Models and Particle Filtering , Proceedings of the Fifth
IEEE International Conference on Automatic Face and Gesture Recognition (2002).
Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specification,
IEEE Std. 802.11, 1997.
References
30. Contd…
• G. Lee and N. Y. Chong, “Decentralized formation control for small scale
robot teams with anonymity,” Mechatronics, vol. 19, no. 1, pp.85–
105,2009.
• G. Lee and N. Y. Chong, “A geometric approach to deploying robot
swarms,” Ann. Math. Artif. Intell., vol. 52, no. 2–4, pp. 257–280, 2009.
• E. Sahin, “Swarm robotics: From sources of inspiration to domains of
• application,” in Proc. 8th Int. Conf. Simulation of Adaptive
Behavior(LNCS),2005,vol.3342,pp.10–20.
• H. Niwa, K. Kodaka, Y. Sakamoto, M. Otake, S. Kawaguchi, K. Fujii, Y.
Kanemori, and S. Sugano, “GPS-based indoor positioning system with
multi-channel pseudolite,” in Proc. IEEE Int. Conf. Robot. Autom., 2008,
pp. 905–910. Faraj Alhwarin, Chao Wang, Danijela Risti -Durrant, Axel
Graser, Improved SIFT-Features Matching for Object Recognition, BCS
International Academic Conference- Visions of Computer Science,2008.