Capstone research project on current HCI technologies like BrailleTouch and thhe Eyes-Free TalkingDialer project, Interviews with the Blind professionals and Future work plan

1. “Touchscreen Typing Accessibility
for the Blind in India”
Adit Gupta
Research Project Advisor –
Dr. Nikhil Balram

2. Broad scope of the talk
Human-Computer
State of the art in
Interaction project
eyes-free typing tech
Motivation
Field Study at BPA Future work plan

3. HUMAN – COMPUTER
INTERACTION

4. MOTIVATION
Percentage distribution of Number of people (in thousands)Visually
blind worldwide Impaired per million total population
Africa
China
15%
21%
America
8%
Emirates
12%
India
21%
SEAR
(India Ex)
WPR 10% Europe
(China Ex) 7%
6%
About 8.075 million of 39.365 million in India!
Only 5% of them receive any kind of assistive technology support!
Pascolini D, Mariotti SPM. Global estimates of visual impairment: 2010. British Journal Ophthalmology Online First published December 1, 2011 as 10.1136/bjophthalmol-2011-300539.

5. A long term study by accessibility researchers in Portugal found that the blind
more time searching for contacts or typing text-
spent
messages than actually calling!
The problems faced by the blind in India in accessing
assistive technologies is much greater!
Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonsalves. 2009. NavTap: a long term study with excluded blind users. In Proceedings of the 11th international ACM SIGACCESS
conference on Computers and accessibility (Assets '09). ACM, New York,NY,USA,99-106.DOI=10.1145/1639642.1639661 http://doi.acm.org/10.1145/1639642.1639661

6. HUMAN – COMPUTER
INTERACTION

7. Current Assistive Technologies
Technology Braille Display Price $(USD)
Braille Sense Plus Yes 6000
Voice Sense No 2000
VoiceNote BT No 1900
Refreshabraille 18 Yes 1700
PAC Mate BX 400 No 1400
EasyLink & Pocketwrite No 1000
GalaTee No 400
Do these technologies solve the problem of Pictures: (top-right left-right)Braille Sense Plus;
Information accessibility for the blind VoiceNote BT; GalaTee, Refreshabraille,
Easylink respectively
completely?
Brian Frey, Caleb Southern, and Mario Romero. 2011. Brailletouch: mobile texting for the visually impaired. In Proceedings of the 6th international conference on Universal access in human-computer
interaction: context diversity - Volume Part III (UAHCI'11), Constantine Stephanidis (Ed.), Vol. Part III. Springer-Verlag, Berlin, Heidelberg, 19-25.

8. Drawbacks of these Technologies
1. Expensive!!
2. Additional item to carry
3. Batteries, and setup time
4. Procurement in India
5. Truncated features while customization
6. Reduced computational capabilities
Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonçalves. 2009. NavTap: a long term study with excluded blind users. In Proceedings of the 11th international ACM SIGACCESS conference on
Computers and accessibility (Assets '09). ACM, New York, NY, USA, 99-106. DOI=10.1145/1639642.1639661 http://doi.acm.org/10.1145/1639642.1639661

9. Research Problem
Helping the blind access general
information almost as seamlessly
as normally sighted people!

10. Our Approach
Could accessible smartphones be
a possible solution?

11. Why this Form Factor?
A simple observation- New York Times applications
Smartphone App iPad App Website
Smartphones present information in the most
condensed format, hence reduce screen-reading time!
Apart from being Ubiquitous, Affordable, Connected & Mobile

12. State of the Art in eyes-free
typing

13. Learning with Braille
The Braille code consists of a
3 by 2 binary matrix
en-codes up to 63 characters
In English Braille, a single combination encodes one
character.
Mario Romero, Brian Frey, Caleb Southern, and Gregory D. Abowd. 2011. BrailleTouch: designing a mobile eyes-free soft keyboard. In Proceedings of the 13th International Conference on Human
Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 707-709.

14. BrailleTouch
Georgia Tech Research
Drawbacks:
Numeric keypad does not have a fairly easy implementation
Needs both the hands to input text
Feedback for entire sentences along with each character not
implemented
Mario Romero, Brian Frey, Caleb Southern, and Gregory D. Abowd. 2011. BrailleTouch: designing a mobile eyes-free soft keyboard. In Proceedings of the 13th International Conference on Human
Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 707-709.

15. TypeInBraille
University of Milan
Subdivided a braille cell into 3 parts
(Derivative of the BrailleTouch project)
Drawbacks:
Needs TWO or more gestures for each letter.
Involves splitting each Braille cell into 3 parts mentally which might lead to slower typing
speeds.
Numeric keypad does not have a fairly easy implementation
João Oliveira, Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonçalves. 2011. BrailleType: unleashing braille over touch screen mobile phones. In Proceedings of the 13th IFIP TC 13 international conference
on Human-computer interaction - Volume Part I (INTERACT'11), Pedro Campos, Nuno Nunes, Nicholas Graham, Joaquim Jorge, and Philippe Palanque (Eds.), Vol. Part I. Springer-Verlag, Berlin, Heidelberg, 100-107

16. TalkingDialer
Google’s Eyes Free project
Drawbacks:
Alphabetic keypad is not intuitive at all, needs knowledge of the spacial
positions of some alphabets
Technique assumes the presence of Hardware call button.
No user interaction studies available for this project although OpenSource
OpenSource code and documentation: https://code.google.com/p/eyes-free/

17. Field Study in India - Visit to
Blind Peoples Association

18. Blind Peoples Association (BPA)
• Established in 1947
• BPA works out of 15 campuses, has 300 odd professionally
qualified staff
• It has a number of innovative programs, such as schools, a
national rehabilitation engineering institute, a computer
training centre, an electronics training centre, a bakery and
food products division for disabled women, and a number of
vocational courses.
www.bpaindia.org

19. Mr. Tarak Luhar
Principal at BPA School
“One Braille cell is mapped to THREE different languages
we teach here apart from Arithmetic numbers!”
“I want to experience color”
“My salary is Rs. 35,000 per month and I cannot afford
those „Braillo‟ computers that are available!!”
Copyright: Fynn Studio, Santa Monica, California

20. Mr. Rahim Khan
Braille Teacher, Toys and puzzle maker
“We try and maintain the teacher-student ratio at about
1:10 so that the children get more attention per class.
Also handling more children would be really tough for
blind teachers.”

21. Mr. R. P. Soni
Coordinator BPA Technology Lab
“I dislike the Braille writers because I can‟t communicate
with the normal sighted people through braille”
“Not only are the specially made devices costly but also
reduce the functionality while trying to accommodate the
needs of the blind. ”
Unplanned Demo of TalkingDialer, Quick learner!
Copyright: Fynn Studio, Santa Monica, California

22. JAWS (Job Access With Speech)
computer screen reader program (text-to-speech output or by a
Refreshable Braille display)
most popular screen reader worldwide
Usage License cost: $1095!
Drawbacks:
 On large screens, a lot of unnecessary information is audio read to
the blind user
(example- Advertisements on web pages, toolbars in
mail, options menu etc.)
 Only Audio feedback available for normal desktops
Wikipedia.org & the JAWS website. JAWS is produced by the Blind and Low Vision Group of Freedom Scientific, St. Petersburg, Florida, USA

23. Amitt Patel
Alumnus of the BPA Technology Lab
Employed at SBI, Online Banker and Stock Broker!
“I use my smartphone for send email and staying in
touch more than my laptop!”
Demo of TalkingDialer and BrailleTouch. He learnt both within a minute of
instruction (even after the bug in BrailleTouch‟s feedback)!!!
In touch with him via email!

24. Future Work Plan

25. Fundamental Scientific Questions
#Q1: What are “intuitive” gestures of text
input to the blind?
#Q2 : What feedback systems are the
Blind most comfortable with?
(Audio/Vibration Hybrid??)

26. •

27. •

28. •

29. Target Subjects
Blind people (congenitally blind or
otherwise) with a sound knowledge of
typing on mobile phone
keypads, preferably smartphones

30. FUTURE WORK
Prototyping Experiments
Implementation and Data collection
Analysis and Documentation
New Design proposition in the Indian
context

31. “Touchscreen Typing Accessibility
for the Blind in India”
Adit Gupta
Research Project Advisor –
Dr. Nikhil Balram