Dynamic hand gesture recognition using cbir

International Journal of Computer Engineering and Technology (IJCET), ISSN 0976-
6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 3, May – June (2013), © IAEME
340
DYNAMIC HAND GESTURE RECOGNITION USING CBIR
Mr.Shivamurthy.R.C
Research Scholar, Department of Computer Science & Engineering
Akshaya Institute of Technology, Tumkur-572106, Karnataka, India
Dr. B.P. Mallikarjunaswamy
Professor, Department of Computer Science & Engg,
Sri Siddharatha Institute of Technology, Maralur, Tumkur: 572105, Karnataka, India
Mr.Pradeep kumar B.P.
Department of Electronics & communication Engineering
Akshaya Institute of Technology, Tumkur-572106, Karnataka, India
ABSTRACT
Image Databases and archives provide lot of research areas. Significant among them
is ; The Contentbased image retrieval (CBIR) research area for manipulating large amount of
image databases and archives. CBIR is mainly based on the way that the image is extracted.
The main focus of the proposed system is on the color and shape feature extractions for hand
tracking that is intended as a step towards palm and face tracking for a perceptual user
interface. This paper extends a default implementation to allow tracking on type of feature
spaces and arbitrary number by reviewing the k-means clustering algorithm. We weigh the
multidimensional histogram with a simple monotonically decreasing kernel profile prior to
histogram back projection in order to compute the new portability that a pixel value belongs
to the target model. We examine the effectiveness of the K-means clustering algorithm as a
general-purpose hand and face tracking approach in the case where no assumptions have been
made about the palm to tracked in this paper image retrieval
Keywords: Content–based image retrieval (CBIR), histogram back projection, k means
clustering algorithm
INTERNATIONAL JOURNAL OF COMPUTER ENGINEERING
& TECHNOLOGY (IJCET)
ISSN 0976 – 6367(Print)
ISSN 0976 – 6375(Online)
Volume 4, Issue 3, May-June (2013), pp. 340-352
© IAEME: www.iaeme.com/ijcet.asp
Journal Impact Factor (2013): 6.1302 (Calculated by GISI)
www.jifactor.com
IJCET
© I A E M E

341
1.INTRODUCTION
There has been a significant growth in the IT field for medical imaging where various
techniques and process are involved to create images of human body for clinical procedure.
This rapid growth has kept the medical science at a very high level. The developments of
large database of medical image are the results of collaborative approaches of handling
medical procedures. An intelligent, fast and accurate medical image retrieval system would
be the ultimate goal of medical imaging for it to be succeeded. With rapidly growing data in
size and semantically distinguishable as the features of various medical images are fuzzy in
nature for different organs, the retrieval system should be very adaptive.
As compared to general purpose images (GPI), the medical images are distinguished
in its characteristics. Hence, in medical image processing systems the process adopted for
searching GPI cannot be adopted.The most visually similar images to a give query image
from a database of images are importantly adopted in context-based image retrieval (CBIR).
CBIR will assist him/her in diagnosis; it does not target at replacing the physician by
predicting the disease of a particular case. The diagnostic information can be derived by the
visual characteristics of a disease. Sometimes it can also be derived out of similar images
correspond to the same disease category. The physician can use the output of CBIR system to
obtain more confidence in his/her decision or to consider other possibilities as well.
The advances in CBIR systems have led the researchers for new approaches in
information retrieval for image databases. It has already met some degree of success in
constrained problems in medical applications.Not withstanding the progress already achieved
in the few frameworks available here is still a lot of work to be done in order to develop a
commercial system able to fulfill image retrieval/diagnosis comprehending a broader image
domain.
Recently, advances in Content Based Image Retrieval prompted researchers towards
new approaches in information retrieval for image databases. In medical applications it
already met some degree of success in constrained problems. The generic framework is
shown in Figure 1.
Figure 1: Diagram for content-based image retrieval system

342
Notwithstanding the progress already achieved in the few frameworks available here
is still a lot of work to be done in order to develop a commercial system able to fulfil image
retrieval/diagnosis comprehending a broader image domain.
2.METHODS USED FOR IMPLEMENTING CBIR
2.1Shape based Method
The image feature extracted for shape based image retrieval is usually an N-
dimensional feature vector that can be regarded as a point in N-dimensional space. After the
images are stored into the database using the extracted feature vectors, the image will be
retieved to determine the similarity between the query image and the target images in the
databases. This is essentially the determination of distance between the feature vectors that
represents the images.
The desirable distance measure should reflect human perception. In image retrieval
various similarity measure have been exploited. We have used Euclidean distance for
similarity measurement in our implementation.
2.2Texture Based Method
We find larger variety of texture measures as compared to color measures. Wavelets
and Gabor filters are some of the common measures for capturing the texture of images
where Gabor filters perform better and correspond well to. The characteristics of images or
image parts with respect to changes in certain directions and scale of changes are capture by
the texture measure. This is most useful for regions or images with homogeneous texture.
2.3Using Low-Level Visual Features
Preprocessing phase and retrieval phase are the two main phases of the image
retrieval process. The description of of both phases as follows.
A feature extraction model and a classification model are the two main components of
pre-processing phase.The original image database i.e. images from the image CLEF medical
collection, with more than 66,000 medical images is the input of the preprocessing phase. An
index relating each image to its modularity and a feature database is the output of the
preprocessing phase.
3.LOW LEVEL IMAGE FEATURE
Content-based image retrieval is based principally on low level image feature. As it
has been found that users are usually more interested in specific region rather than entire
image, most current content-based image retrieval systems are region based. This means that
the image is divided in regions on whuich the other operation are
performed. To carry out this first step we need to perform a segmentation of the original
image. To specify queries we will consider several classes of features: color, texture, shape.
3.1 Image segmentation
Automatic image segmentation is a difficult task to compute. A lot of techniques have
been proposed in the past, such as curve evolution [11] and graph partitioning [12]. A lot of
known techniques works well for images that have homogeneous color regions. Let's see an
example of segmentation on one picture in figure 2. However, picture from the real world are

343
richer of color and shades. In literature there are know many segmentation algorithm, like
JSEG [10], blobworld [9] or KMCC [13].
Figure 2: Example of segmentation of a picture. (the red lines divided the regions)
3.2 Color feature
Color features are easy to obtain, often directly from the pixels intensities like color
histogram over the whole image, over a fixed sub image, or over a segmented region. It is one
of the most used features in image retrieval. The colors are described by their color space:
RGB, LAB, LUV, HSV,
RGB is the best known space color and is it commonly used for visualization. The acronym
stands for Red Green Blue. This space color can be seen as a cube where the horizontal x-axis
as red values increasing to the left, y-axis as blue increasing to the lower right and the vertical
z-axis as green increasing towards the top, as in figure 3.
Figure 3: The RGB color model mapped to a cube.
The origin, black, is hidden behind the cube. RGB is a convenient color model for
computer graphics because the human visual system works in a way that is similar, though
not quite identical, to an RGB color space.
Another famous space color is the HSV. The acronym stands for Hue Saturation
Value. Refering to the image 3 we can see the color space as a cylinder, where the angle
around the central vertical axis corresponds to hue, the distance from the axis corresponds to
saturation, and the distance along the axis corresponds to value (also called brightness).

344
Figure 4: The HSV space color.
3.3 Shape feature
There's no universal definition of what a shape is. Impressions of shape can be
conveyed by color or intensity patterns, or texture, from which a geometrical representation
can be derived, like Plato's Meno, where a Socratic dialog is made around the word “figure".
Figure is the only existing thing that is found always following color – Socrate. Shape feature
(like aspect ratio, circularity, Fourier descriptor, consecutive boundary segments, . . . ) are
very important image feature, even if they are not so commonly used in Region-Based Image
Retrieval Systems. Due to the inaccuracy of segmentation step, it is more difficult to apply
shape features instead of color or texture feature. However in literature are know some
Content-Based Image Retrieval Systems that use this feature, like [15], [13] e [14].
4. INTRODUCTION TO GESTURE
Human hand gestures have been a mode of nonverbal interaction widely used. It
ranges from simple action of using our finger to point at and using hands to move objects
around for more complex expressions for the feelings and communicating with others. The
pursuance for the Human Computer Interaction research is moved by the central dogma of
removing the complex and cumbersome interaction devices and replacing them with more
obvious and expressive means of interaction by minimizing interaction
4.1 Background Registration and foreground segmentation
When making skin color detection in real situation where it is very important process
in gesture recognition, so that divide the images into foreground and background regions
according to the dynamic characteristic of the film. The background regions such as face,
neck, areas of adjacent skin color, etc. do not change during the recognition process, In
addition to the background region, we call the hand gestures area foreground region. We
subtract the background from the images to get foreground regions. The foreground image is
used to conduct "and" operation with skin color images to get a more accurate image of hand
region.
Differential techniques are used in video image processing, that is current image
subtracts background image, the result image is called difference foreground image. Select
appropriate threshold. The difference foreground image was then be binarized to produce
foreground binary image. The basic principle of differential image processing is to carry out

345
differences calculation between the image after gray scale transformation in the detect area
with the background image in the spatial domain.
That can be expressed as
Where f(x,y,ti) and f(x,y,tj) represent the brightness values of the pixel at position of
(x,y) on the moment of ti and tj respectively. The range of the value is from 0 to 255. The
significant differences of pixel's brightness at position (x,y) is described as:
Where mk and sk(k=i,j) are the mean and variance of f(x,y,tk) within a small
neighborhood q(x,y) at position of (x,y). t is a threshold.
Segmentation is a process of partitioning an image into multiple segments based on
certain attributes. The ultimate goal of the segmentation is to convert the image into a
simplified form that is more useful as compared to the original image. The results of multiple
segmentation techniques depend upon the requirement for segmentation. Background
subtraction provides an effective means of segmenting objects moving in front of a static
background. Researchers have traditionally used combinations of morphological operations
to remove the noise inherent in the background-subtracted result.
4.2 Tracking
Tracking starts with interest of color space used for skin modeling. RGB is a
convenient color model for computer graphic because the human visual system works in a
way that is Similar to an RGB color space, when it was convenient to express color as a
combination of three based colored rays (red, green and blue). Normalized RGB skin color
model is considered to be more proper for hand skin, and can be easily obtained from the
RGB values by a simple normalization processing.
Format of YCbCr has been considered to be better in describing the properties than
RGB color space The clustering characteristic for YCbCr is better than RGB YCbCr is used
to separate out a luminance signal (Y) and two chrominance components (Cb and Cr). YCbCr
can challenge various illumination conditions by discarding the signal Y, which not only
improve the performance and also reduce the data dimension than RGB.

346
The properties of skin color can be characterized by Gaussian distribution. The single
Gaussian model is one of the simplest models to model the distribution of the certain objects
which is widely used in computer vision and pattern recognition. Gaussian distribution is
given by
Where is the mean value of the samples and the variance value. Using the
Gaussian model to model skin color is actually a process that matching each pixel of the
image to the model. If matched, it is consider the pixel as a skin pixel, else it is consider it
background.
Highlighting of the hand posture particularly in the film is done considering several steps
including skin detection are camshaft algorithm, calibration, motion detection etc.
5. BLOCK DIAGRAM
Figure 5: Proposed Hand Gesture Recognition System
Gesture recognition is important for developing an attractive alternative to prevalent
human–computer interaction modalities. The system design shown above depicts the
techniques used for designing a dynamic user interface which initializes by acquiring image

347
or video of the gestures from user. The different hand gestures considered are the sequences
of distinct hand gestures. A given hand gesture can undergo motion and discrete change.
These gestures are distinguished based on the nature of motion. The real time
recognition engine is developing which can reliably recognize these gestures despite
individual variations. The engine also has the ability to detect start and end of gesture
sequences in an automated fashion.
6. ALGORITHMS
6.1 Algorithm for calibration
In calibration part, take the snapshot of the hand region from the webcam then prompt
user has to select the hand region from the snapshot. Convert the input image from RGB
color format to LAB and HSV format for further operation on selected skin region. Then
calculate and store the mean values of A, B, H&S[7].
6.2 Skin Detection
Skin detection is very important process in detecting the hand posture so to perform with
skin detection we need to do calibration to get hand color pixels as samples then
comparing those pixels with current image to detect the hand. Hence following
processing steps can be observed in skin detection.
a. Load color samples, and update the background so as to take the median. Calculate
the mean 'a*' and 'b*' value for each area that you extracted with roipoly. These values
serve as your color markers in 'a*b*' space.
b. Trigger the image and take the difference of current image and the background.
watever the output we obtain we have to take it in 3 dimension.
c. Next process is Thersholding. This is in simple converting image into binary. Then
doing some morphological operations like Dilation defined as some kind of
expansion.Fill the circular region.
d. Convert the output to LAB format and HSV format where LAB defined as L for
luminosity, A for chrominocity layer a, B for chrominocity layer b.
e. Calculate Euclidian distance between input pixels and sample values, these values are
nothing but the inputs A,B,am,bm which are stored in calibration.
distance1 = ((a - am).^2 + (b - bm).^2 ).^0.5;
distance2 = ((H - hm).^2 +(S - sm).^2 ).^0.5;
If distance between the pixels is less than threshold value 15 for LAB
format(D1<THRESHOLD (15)) then image mask 1 is generated.
f. If distance between the pixels is less than threshold value 0.5 for HSV
format(D1<THRESHOLD (0.5)) then image mask 2 is generated.
Distance between the pixels is less if we have similar pixels and more if pixels are
different. In terms of HSV if the distance is less than 0.1 then it is matching. Get BW and
BW2. Get the mask for skin segmentation. Perform OR operation for both color space
segmentation output so that to get the skin mask [8].

348
7. RESULTS AND DISCUSSION
Figure 6: Acquired Image with subplot
Figure 7: A sample output after background registration and foreground segmentation

349
Figure 8: Selecting ROI to get skin color samples in calibration
Figure 9: Dynamic single palm Picture with Segmentation
Figure 10: two palm after skin segmentation

350
Figure 11: detection of palm and face after skin segmentation
8. INTERPRETATION OF RESULTS
Figure 6 shows the hand figure, is captured through the camera by using the Matlab
Program. Figure 8 shows the hand figure with area selected. Here, the values from area
selected are stored for the dynamic hand segmentation depending on the values (color)
calculated earlier. Figure 7 A sample output after background registration and foreground
segmentation. Figure 9,10 shows the dynamic segmentation by matching the color values.
Pictures are saved while hand in motion dynamically. Figure 11 shows there will be a
detection of face also due to the skin color segmentation.
9. CONCLUSION
This paper describes the design and implementation of a bare dynamic hand gesture
recognition system using just one color camera. The developed system can obtain a high
recognition rate of bare hand gestures. Our current research mainly focuses on the single
hand static gestures for the simplicity. However, we are far from building a general-purpose
gesture recognition system. The extensive experiments and evaluation in outdoor
environment where more uncertainties such as changing backgrounds, sunshine and shadows
may bring the hand gesture complicated. The dynamic gestures and two-handed gestures
should also be explored in the future, since they are more expressive and allow more natural
interaction. the system implementation shown above adopted by the detection of hand or
highlighting hand posture in the input image using calibration, motion detection, and resulted
with segmented hand posture is defined with results.
As in case of dynamic recognition technics video processing needs speed and higher
intensity in providing experience of virtual and real time experience.

351
REFERENCES
[1] Yoo-Joo Choi, Je-Sung Lee and We-Duke Cho, “A Robust Hand Recognition In
Varying Illumination,” Advances in Human Computer Interaction, Shane Pinder
(Ed.), 2006.
[2] N. Conci, P. Cerseato and F. G. B. De Natale, “Natural Human- Machine Interface
using an Interactive Virtual Blackboard,” In Proceeding of ICIP 2007, pp. 181-184,
2007.
[3] S. Mitra, and T. Acharya, “Gesture Recognition: A survey,” IEEE Transactions on
Systems, Man and Cybernetics (SMC) - Part C: Applications and Reviews, vol. 37(3),
pp. 211-324, 2007
[4] Xiujuan Chai, Yikai Fang and Kongqiao Wang, “Robust hand gesture analysis and
application in gallery browsing,” In Proceeding of ICME, New York, pp. 938-94,
2009.[5] Ayman Atia and Jiro Tanaka, “Interaction with Tilting Gestures in Ubiquitous
Environments,” In International Journal of UbiComp (IJU), Vol.1, No.3, 2010.
[6] José Miguel Salles Dias, Pedro Nande, Pedro Santos, Nuno Barata and André Correia,
“Image Manipulation through Gestures,” In Proceedings of AICG’04, pp. 1-8, 2004.
[7] Pradeep kumar B.P Shailesh M.L, Shankar B.B, , Kumudeesh.K.C. “Dynamic Hand
Gesture Recognition”, International Journal of Graphics And Image Processing,
published in ijgip, volume 2, issue 1, feb 2012
[8] Pradeep kumar B.P “Design and development of human computer interface (HCI)
system based on gesture recognition using svm”, International Journal of Graphics And
Image Processing, published in ijgip, volume 2, issue 2, may 2012
[9] C. Carson, S. Belongie, H. Greenspan, and J. Malik. Blobworld: im-age segmentation
using expectation-maximization and its application to image querying. Pattern Analysis
and Machine Intelligence, IEEE Transactions on, 24(8):1026{1038, 2002.
[10] Y. Deng and B. S. Manjunath. Unsupervised segmentation of color- texture regions in
images and video. Pattern Analysis and Machine Intelligence, IEEE Transactions on,
23(8):800{810, 2001.
[11] Haihua Feng, D.A. Castanon, and W.C. Karl. A curve evolution approach for image
segmentation using adaptive ows. In ICCV 2001. Proceedings. Eighth IEEE
International Conference on computer vision, volume 2, 2001.
[12] W.Y. Ma and B.S. Majunath. Edge ow: a framework of boundary detection and image
segmentation. In ICCV 2001. Proceedings. Eighth IEEE International Conference on
computer vision, 1997.
[13] V. Mezaris, I. Kompatsiaris, and M. Strintzis. An ontology approach to object-based
image retrieval, 2003.
[14] Christopher Town and David Sinclair. Content based image retrieval using semantic
visual categories. Technical report, 2001.
[15] J. Wang, J. Li, D. Chan, and G. Wiederhold. Semantics-sensitive retrieval for digital
picture libraries. D-LIB Magazine, 5(11), November 1999.
[16] Shaikh Shabnam Shafi Ahmed, Dr.Shah Aqueel Ahmed and Sayyad Farook Bashir,
“Fast Algorithm for Video Quality Enhancing using Vision-Based Hand Gesture
Recognition”, International Journal of Computer Engineering & Technology (IJCET),
Volume 3, Issue 3, 2012, pp. 501 - 509, ISSN Print: 0976 – 6367, ISSN Online: 0976 –
6375.

352
ABOUT THE AUTHOR
Mr. Shivamurthy R C received the BE degree from PDA college of
Engineering, Gulbarga University and received the M.Tech degree in
Computer Science & Engineering from Malnad College of Engineering,
Visvesvaraya Technological University, Belgaum. Currently working as
professor in the department of Computer Science at A.I.T, Tumkur,
Karnataka, and he is also a Ph.D scholar in CMJ University, India.
Dr.B.P Mallikarjunaswamy. working as a professor in the
Department of Computer Science & Engineering, Sri Siddhartha Institute
of Technology, affiliated to Sri Siddhartha University. He has more than 20
years of Experience in teaching and 5 years of R & D. He is guiding many
Ph.D scholars. He has published more than 30 technical papers in national
and International Journals and conferences. His current research interests
are in pattern Recognition and Image Processing.
Mr.Pradeep Kumar.B.P, He is presently working as a Assistant
Professor in Akshaya Institute of Technology, Tumkur and perusing his
PhD in Jain university in electronics & communication Engineering
department. He has published more than 20 technical papers in national and
International Journals and conferences. His areas of interests are signal
processing, Medical Imaging, pattern recognition, video processing,
Multimedia Communication systems.

Dynamic hand gesture recognition using cbir

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Dynamic hand gesture recognition using cbir

Similar a Dynamic hand gesture recognition using cbir (20)

Más de IAEME Publication

Más de IAEME Publication (20)

Último

Último (20)

Dynamic hand gesture recognition using cbir