Communication between normal people and people with vocal and hearing troubles

1. ADITYA COLLEGE OF ENGINEERING & TECHNOLOGY
ABSTRACT
Communication between normal people and people with vocal and hearing troubles is a
difficult task. The sign language used by these people is not understandable by the common
people, so it creates a communication barrier. People who are paralyzed also require
assistance regularly. For such people we are implementing an IOT based Smart assistance
gloves for disabled people. The gloves we design is very simple yet effective when compared
to the existing system which is switch based glove. It is designed in such a way that when
pressure is applied on the switch, the two ends come into contact and the switch will be
closed. The two terminals of the switch should come into contact when the finger is bent,
hence the switches are placed on the fingers of glove. But here is a disadvantage that the
switch instead of being open when the finger is straight, it will be in incorrect or improper
transmission of gesture data due to prolonged use. In this project we use flex sensors which is
used to detect the finger gesture and the corresponding instructions are displayed in the
android app with audio output. The proposed system is implemented by Arduino uno and
Raspberry pi, where the communication between these two modules is done by wireless serial
port module due to its secured data transmission. An alert message will be sent through the
GSM module during emergency situation.

2. ADITYA COLLEGE OF ENGINEERING & TECHNOLOGY
INTRODUCTION
Disable people struggle to communicate with other in their day-to-day life and also,
it's hard for them to express their emotions. People with disabilities generally use
sign language to communicate with each other but this sign language is difficult for
normal people to understand. Not only normal people, but people with disabilities
find it hard to learn these sign languages. It is also difficult for them to socialize and
they will not be able to voice out their opinions. This is the same case with
paralyzed people. They will not be able to move or communicate. So, the gloves we
have designed allows the person with disabilities to communicate what he/she
wants without anyone’s help.
Based on the movement which is done by the fingers, the flex sensors detect the
bend made by the finger and the output is varied in terms of resistance. Due to its
flexibility and large range of resistance, many commands can be fed into it. We
have used Arduino uno for more storage and faster response. The raspberry pi 3b
includes inbuilt Wi-Fi module, Bluetooth and USB boot which is used to connect
with the android app. The Arduino uno and the raspberry pi are transfer signals
between them with the help o f a wireless serial port module.
11/23/2022 class, section, batch number: 2

3. ADITYA COLLEGE OF ENGINEERING & TECHNOLOGY
LITERATURE SURVEY
There are 4types of existing smart glove technologies that are as follows:
• Using CMOS camera.
• Leaf Switch based Glove.
• Copper Plate based Glove.
• Flex sensor based Glove.
Using CMOS Camera :
Hand Gestures were detected using CMOS camera by the 3 steps as follows:
• Capturing the image of the gesture.
• Edge detection of that image.
• Peak detection of that image.
Disadvantage : Highly expensive ,latency and each image occupies 50KB of memory.
11/23/2022 class, section, batch number: 3

4. ADITYA COLLEGE OF ENGINEERING & TECHNOLOGY
Leaf switch based Glove :
Leaf switches are designed in such a way that when pressure is applied on the
switch, the two ends come into contact and the switch will be closed. The two
terminals of the switch should come into contact when the finger is bent, hence the
leaf switches are placed on the fingers of the glove.
Disadvantage : The switch instead of being open when the finger is straight, it will
be closed resulting in incorrect or improper transmission of gesture data due to
prolonged use.
Copper Plate based Glove :
In this technology, a copper plate is fixed on the palm as ground. In rest position,
the copper strips indicate a voltage level of logic. The voltage associated with
copper strips is drained when they come in contact with the ground plate and thus,
they indicate a voltage level of logic 0.
Disadvantage : The use of copper plate makes the glove bulky which makes it
unsuitable to use for a long time.
11/23/2022 class, section, batch number: 4

5. ADITYA COLLEGE OF ENGINEERING & TECHNOLOGY
Flex sensor based Glove :
Flex sensor is a resistive sensor changing its resistance according to the change in
bend or curvature of the finger into analog voltage. This is a haptic based
technology using flex sensors to take in physical values for processing.
All these old technology innovations have motivated us to enhance the
advancement in technology. They have their limitations of recognition rate and
time which will be overcome with our proposed glove.
• The model consists of the Transmitter and Receiver section. The transmitter
section consists of flex sensors, PIC12F683 microcontroller and a RF transmitter,
whereas the receiver section consists of PIC18F45K20 microcontroller, AC
driver, LCD display, voice module and RF receiver. The flex sensor gestures are
converted to commands and displayed in LCD. Voice recorder and Playback unit
is used to pre-record the voice to fetch the output in the speaker.
11/23/2022 class, section, batch number: 5

6. ADITYA COLLEGE OF ENGINEERING & TECHNOLOGY
• The proposed system is focused for facially paralyzed persons. It uses only one flex
sensor and predefined input is given and the output is displayed in LCD.
• Conducting line sensors (CSL) are used to detect the gesture. This line sensor is
connected to Arduino uno and the data is fed into the microcontroller in digital form.
The output of the line sensor varies on various positions. So, the corresponding
command is displayed in LCD as text.
• A flex sensor is fitted to the gloves. The instructions are fed into the Arduino AT mega.
Whenever the finger makes the gesture, the predefined instructions are displayed in
LCD.
• Five flex sensors are used. Along with the flex sensors, tactile sensors and the
measurement of the orientation of the hand is done by the accelerometer. The sensor
takes in analog inputs and the output is in the form of digital output.
• The project focuses on converting the gestures made by the deaf-dumb people into
meaningful text/speech. The predefined hand gestures made by the disabled people are
captured and stored in the database. Data is matched by the microcontroller with the
database and the output is given out in the form of audio by the speaker.
11/23/2022 class, section, batch number: 6

7. ADITYA COLLEGE OF ENGINEERING & TECHNOLOGY
METHODOLOGY
As there is no significant development for disabled people, we have designed the
smart assistance gloves for them. The proposed model is designed with the help of
flex sensors and the instructions are fed into the Arduino Uno board. The finger
gesture is captured by the flex sensor and a corresponding output is displayed in the
form of a sentence in the Android app and output is also displayed as an audio
output.
The overall process is carried out by Arduino Uno, Raspberry Pi and GSM module.
The Data transmission between the Arduino Uno and Raspberry Pi is done with the
help of a wireless serial port module. An alert message will be sent through the
GSM module during emergency situation to the emergency contact.
11/23/2022 class, section, batch number: 7

8. ADITYA COLLEGE OF ENGINEERING & TECHNOLOGY
COMPONENTS
• Flex Sensors : Flex sensors generally detect the amount of bending and
deflection. As the bending increases, the resistance also increases.
• Wireless serial port module : This module is compatible with the Arduino
uno board and it is used for two-way RF communication.
• Arduino Uno: Arduino may be a single-board microcontroller meant to form
the appliance more accessible which are interactive objects and its
surroundings. This board consists of digital pins and analog pins and it can be
interfaced with various expansion boards and circuits. This contains all the
required support needed for microcontrollers.
• Raspberry Pi : The raspberry Pi uses a next generation main processor. It has
40 I/O headers. It consists of a 1.2 gigahertz and 64 quad core arm cortex
which uses an A53 processor. This device also includes a Bluetooth module
and USB boot and also an onboard 802.11n Wi-Fi module.
• GSM: A GPRS or a GSM module is generally used as a communication
between a microcontroller and GSM network.
11/23/2022 class, section, batch number: 8

9. ADITYA COLLEGE OF ENGINEERING & TECHNOLOGY
CONCLUSION :
Thus, the proposed model has the advantage of assisting deaf and dumb as
well as the paralyzed patients by displaying the output commands in
android application with vocal output in speaker. By using wireless serial
port modules, data transmission is fast and secured. In case of emergency,
GSM will send alert messages to the respective person. Compared to the
vision-based techniques, this data-based gloves will reduce noise
disturbances and offer less complexity in algorithms. In future, we can
enhance this proposed model with a maximum number of commands.
Using AI, the data-based system can be further enhanced with speech
recognition. For home automation, by using different gestures we can
control various basic functions such as switching home appliances in an
effective manner
11/23/2022 class, section, batch number: 9