1. Automated Sorting Machine using Video Processing and a Robotic Arm
Luigy Bertaglia Bortolo
Advisor: Dr. Jafar Saniie
Department of Electrical and Computer Engineering
Automation is quickly becoming a key to decrease
production cost and manufacturing time in the
industrial world. To contribute to this cause, this
project proposes the use of a robotic arm and
computer vision (CV) techniques to automate the
sorting of two different classes of objects.[1]
ABSTRACT
OBJECTIVES
• Acquire a real image and define (labeling) each
object’s class.
• Setup the arm control and map the background.
• Design an algorithm to automate the action of
sorting and separating different classes of objects
using a robotic arm. • Image Segmentation : The process of partitioning a digital image into multiple segments(sets of pixels, also
known as super pixels). The goal of segmentation is to simplify and/or change the representation of an image
into something that is more meaningful and easier to analyze[4].
• Morphological Processing : Apply a structuring element to an input image, creating an output image of the
same size. The most basic morphological operations are dilation and erosion, where dilation adds pixels to
the boundaries of objects in an image, while erosion removes pixels on objects boundaries[2].
• Region Labeling : Used with a linear object discrimination to identify it class of objects, then labeling each
one[2].
STAGE 1: Implement Algorithms for Image Processing
• ArbotiX-M Robocontroller[3]
• 16MHz AVR microcontroller.
• 2 serial ports.
• 3 TTL DYNAMIXEL 3-pin style ports.
• 28 Digital I/O.
• Servo style 3-pin headers.
• Arduino compatible board.
• Open source library designed.
• Find the orientation of the bolts, using another
camera to improve the image processing and
making it more accurately.
• Separate the objects while they are moving.
FUTURE IMPLEMENTATIONS
PROJECT WORKFLOW
• Given the end point of a structure, the inverse kinematics returns what angles the joints need to be to
achieve this end point.
• This method makes the connection between the real world and the robot arm coordinates(RAC).
• The final result of the inverse kinematics is the equation of all the joint angles in function of the Cartesian
coordinates (x, y and z).
• Thus some equations were developed that will convert these angles in the RAC. These equations depend on
the direction of movement of the motor.
They can be either of the following:
; , which θ is in degrees.
• To make the robot arm move, we just have to choose the motor that will be moving and the distance it will
move using one of the equations described above.
HARDWARE DESCRIPTION STAGE 3: Interface between Computer Vision and Robotic Arm
• Project Features[3]
 (7) AX-12 Dynamixel Actuators
 PhantomX Reactor Arm
 ArbotiX Robocontroller
 12v 5amp power supply
 FTDI 5v Programming Cable
 External Web Cam
• Reactor Arm Stats[3]
Weight : 1360g(1430 w/Rotate)
Vertical Reach : 51cm
Horizontal Reach : 30cm/200G
Wrist Lift Strength : 250g
• Brazilian Scientific Mobility Program (BSMP);
• Institute of International Education (IIE);
• Coordenação de Aperfeiçoamento de Pessoal de
Nível Superior (CAPES);
• Graduate Assistants: Thomas Gonno; Guojun
Yang; Yonghui Jia.
ACKNOWLEDGEMENT
• [1] Richard Szeliski, “Computer Vision, Algorithms
and Applications” ©2010, Microsoft Research
• [2] "Image Processing Learning Resources." HIPR Top
Page. Web. 30 June 2015. Retrieved from:
homepages.inf.ed.ac.uk/rbf/HIPR2
• [3] "Quick Start Guide." Learn.TrossenRobotics.
Web.30 June 2015. Retrieved from:
learn.trossenrobotics.com/arbotix/arbotix-quick-
start.html
• [4] Linda G. Shapiro and George C. Stockman (2001):
“Computer Vision”, pp 279-325, New Jersey, Prentice-
Hall, ISBN 0-13-030796-3
REFERENCESSTAGE 2: ARM CONTROL AND BACKGROUND MAPPING
• Background Mapping
Creation of a grid that
has 40 rows and 40
columns to identify the
possible positions of the
objects, better accuracy
to the arm localization
and grip.
• In this project we successfully designed an
algorithm to differentiate and separate objects
using computer vision technics and a robotic arm.
CONCLUSION