Communication is always having a great impact in every domain and how it is considered the meaning of the thoughts and expressions that attract the researchers to bridge this gap for every living being. The objective of this project is to identify the symbolic expression through images so that the communication gap between a normal and hearing impaired person can be easily bridged. Github Link:https://github.com/TrilokiDA/Hand_Sign_Language

1. Sign Language Recognition based
on Hands Symbols Classification
Guided by: Dr.S.R.Balasundaram
(Professor)
Presented by: Triloki Gupta
M.Tech(Data Analytics)
205217006
Department of Computer Application1/17/2019 1

2. Content
● Introduction
● Motivation
● Problem statement
● Objective
● Literature review
● Dataset description
● Proposed work
● Result
● Conclusion and Future work
● References
1/17/2019 2
Department of Computer Application

3. Introduction
● The world is hardly live without communication, no matter
whether it is in the form of texture, voice or visual
expression.
● The communication among the deaf and dumb people is
carried by text and visual expressions.
● Gestural communication is always in the scope of
confidential and secure communication.
● Hands and facial parts are immensely influential to express
the thoughts of human in confidential communication.
1/17/2019 3
Department of Computer Application

4. Motivation
● Sign language is learned by deaf and dumb, and usually it is
not known to normal people, so it becomes a challenge for
communication between a normal and hearing impaired
person.
● Its strike to our mind to bridge the between hearing
impaired and normal people to make the communication
easier.
● Sign language recognition (SLR) system takes an input
expression from the hearing impaired person gives output to
the normal person in the form text or voice.
1/17/2019 4
Department of Computer Application

5. Problem Statement
● Understanding the exact context of symbolic
expressions of deaf and dumb people is the
challenging job in real life until unless it is properly
specified.
1/17/2019 5
Department of Computer Application

6. Objective
● Communication is always having a great impact in
every domain and how it is considered the
meaning of the thoughts and expressions that
attract the researchers to bridge this gap for every
living being.
● The objective of this project is to identify the
symbolic expression through images so that the
communication gap between a normal and hearing
impaired person can be easily bridged.
1/17/2019 6
Department of Computer Application

7. Literature Review
1/17/2019 7
Author Publication Year Problem Methodolog
y
Remark
Naresh Kumar ICCCA 2017 Hand Sign
Language
Recognition
SVM & LDA Classification
to recognition
sign language
symbols
(97.3%).
Tse-Yu Pan,
Li-Yun Lo,
Chung-Wei
Yeh, Jhe-Wei
Li, Hou-Tim
Liu, Min-
Chun Hu
IEEE 2016 Real-time Sign
Language
Recognition
SVM, PCA &
LDA
Images of the
same gesture
were captured
in different
lighting
Conditions
(94%)
Department of Computer Application

8. Cont.
Author Publication Year Problem Methodolog
y
Remark
Salem Ameen,
Sunil Vadera
University of
Salford
2017 Classify American
Sign Language
Fingerspelling from
Depth and Colour
Images
CNN This paper
explores the
applicability of
deep learning for
interpreting sign
language,
precision of 82%
and recall of
80%.
Arabic sign
language
recognition with
3D
convolutional
neural networks
IEEE 2017 3D Convolutional
Neural Network
(CNN) was used to
recognize 25
gestures from an
Arabic sign
language dictionary
CNN The system
achieved 98%
accuracy for
observed data
and 85% average
accuracy for new
data.
1/17/2019 8
Department of Computer Application

9. Dataset description
1/17/2019 9
● We analyze 4,800 images of sign images which is ISL of the English alphabet,
which have a spread of 26 class labels assigned to them. Each class label is a set of
sign images of the English alphabet.
● All the images are resized to 640 x 480 pixels, and we perform both the model
optimization and predictions on these downscaled images.
Department of Computer Application

10. Cont.
1/17/2019 10
● Below figure shows an example from every class of sign images dataset.
Department of Computer Application

11. Proposed Work
● In this work, we proposed an idea for feasible communication between hearing
impaired and normal person with the help of-
• Deep Learning
■CNN
■AlexNet
• Machine Learning
■SVM
■Random Forest
■KNN
1/17/2019 11
Department of Computer Application

12. Cont.
Work flow Diagram:
1/17/2019 12
Department of Computer Application

13. Cont.
Figure. Typical Convolutional Neural Network Architecture
1/17/2019 13
Department of Computer Application

14. Cont.
Figure. Typical AlexNet Architecture
1/17/2019 14
Department of Computer Application

15. Result
● CNN
• SGD optimizer with learning rate 0.01 and dropout 0.25
• Model Accuracy
■ Accuracy: 98.74%
• Model Loss
■ Loss: 6.53%
1/17/2019 15
Department of Computer Application

16. Cont.
● AlexNet
• SGD optimizer with learning rate 0.01, momentum 0.9, nesterov and dropout 0.25
• Model Accuracy
■ Accuracy: 99.79%
• Model Loss
■ Loss: 0.79%
1/17/2019 16
Department of Computer Application

17. Cont.
Comparison table of CNNs and AlexNet:
1/17/2019 17
Model Optimizer Dropout Learning
rate
Momentum Nestrove Accuracy
%
CNNs SGD 0.25 0.01 --- --- 98.74
CNNs SGD 0.50 0.01 --- --- 98.74
AlexNet SGD 0.25 0.01 98.42
AlexNet SGD 0.50 0.01 --- --- 95.70
AlexNet SGD 0.25 0.01 0.9 True 99.79
AlexNet SGD 0.50 0.01 0.9 True 99.79
AlexNet Adam 0.25 0.001 --- --- 99.69
AlexNet Adam 0.50 0.001 --- --- 99.58
Department of Computer Application

18. Cont.
1/17/2019 18
● Machine Learning
• SVM: 91.226%
• Random Forest: 95.719%
• KNN: 87.542%
Department of Computer Application

19. Conclusion and Future Work
● In this project, we proposed an idea for feasible
communication between hearing impaired and normal person
with the help of deep learning and machine learning approach.
● This proposed work ensures the accuracy of 91.22% using
SVM, 95.71% using Random Forest, 87.54% using KNN,
98.74% using CNN and 99.79 using AlexNet.
● There is ever the sounding challenge to develop a sign
language system in data the collection remains invariant of the
unconstraint environment. This project can be extended to the
real time data.
1/17/2019 19
Department of Computer Application

20. References
[1] Ameen, S., & Vadera, S. (2017). A convolutional neural network to
classify American Sign Language fingerspelling from depth and colour
images. Expert Systems.
[2] Naresh Kumar(2017). Sign Language Recognition for Hearing Impaired
People based on Hands Symbols Classification. International Conference on
Computing, Communication and Automation (ICCCA2017)
[3] Menna ElBadawy, A. S. Elons, Howida A. Shedeed and M. F. Tolba.
Arabic sign language recognition with 3D convolutional neural networks. 2017
Eighth International Conference on Intelligent Computing and Information
Systems (ICICIS)
[4] Pan, T. Y., Lo, L. Y., Yeh, C. W., Li, J. W., Liu, H. T., & Hu, M. C.(2016,
April). Real-time sign language recognition in complex background scene
based on a hierarchical clustering classification method. In Multimedia Big
Data (BigMM), 2016 IEEE Second International Conference on (pp. 64-67).
IEEE.
1/17/2019 20
Department of Computer Application

21. Cont.
[5] Pigou, L., Dieleman, S., Kindermans, P. J., & Schrauwen, B. (2014,
September). Sign language recognition using convolutional neural networks.
In Workshop at the European Conference on Computer Vision (pp. 572-578).
Springer International Publishing.
[6] Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of
initialization and momentum in deep learning. In: Proceedings of the 30th
International Conference on Machine Learning (ICML-13). pp. 1139{1147
(2013)
[7] Tao Liu, Wengang Zhou, and Houqiang Li. Sign Language Recognition
With Long Short-Term Memory . 2016 IEEE International Conference on
Image Processing (ICIP)
1/17/2019 21
Department of Computer Application

22. 1/17/2019 22
Department of Computer Application