This presentation is for the Hello A11y conference celebrating Global Accessibility Awareness Day 2020. It introduces how artificial intelligence and machine learning is being used in assistive technology for people with disabilities.
Artificial Intelligence and Accessibility - GAAD 2020 - Hello A11y
1. AI/ML +
A11Y for
GAAD20
Artificial Intelligence and Accessibility for
Global Accessibility Awareness Day 2020
Ted Drake, Intuit Global Accessibility Leader
Hello A11y, May 21 2020
2. Today’s agenda
Machine Learning vs. Artificial
Intelligence
Recent evolution
Where do we go from here
Key Trends in AI for Accessibility
3. Machine Learning and Artificial Intelligence
Machine Learning
Computer learns from exploring data,
discovering connections, and solving a problem.
Artificial Intelligence
Computer takes initiative to do something based
on what it has learned.
Car learns how to analyze
risks in an intersection
Car takes corrective actions
to avoid an accident.
4. A.I. is not special
Anyone can work on A.I.
WikiData: structured data for learning
Build a deep learning computer for $450
Android and iOS process AI on device
Nvidia Jetson Nano - $99
AWS and Google Cloud - Powerful Options
8. QuickBooks Mileage Tracking
QuickBooks Mobile app creates mileage reports
Simply give it location permission
It tracks start, finish, mileage, and expense
deduction.
Driver or Passenger
Average users finds 37%
more expense deductions
10. Seeing AI by Microsoft
Object detection
Barcode Scanner
Recognize friends
Currency Identifier
Intelligently guides user to
rotate object to find the
barcode.
11. Safe Exit for All
Emergency exit navigation
Dynamically adjust for danger
zones
Customized for disabilities, such
as mobility or vision
13. Detecting Dyslexia and Autism
Dytective uses mobile games
to detect risk of having
dyslexia
Autism detection based on
eye movement sequences
Affordable diagnosis
Early treatments
14. Satya Panda - Dr.Bindu Sravani Nayak
Independent Developers
Open Source Machine Learning Libraries
Affordable Data Servers
Dental diagnosis and treatment project
created in collaboration with 2 dentists
Significantly reduce the costs of diagnosis
Differentiate between similar symptoms and
treatments
16. QuickBooks Capital
Unconscious Bias - Significant barrier to small
business funding
QuickBooks Capital analyzes 26 billion
transactions and data points
Loans are given based on a business’s ability to
repay
60% of customers say they didn’t
qualify for loans before QuickBooks
Capital
17. Android Live Transcribe
Speech to Text live captioning in 70
languages
Auto-correct based on context
Sound alerts
Open Source
On device neural network -
reduces network connectivity
requirement
19. Indoor Navigation via computer vision
Smith-Kettlewell Institute:
Computer vision, landmarks,
inertia, and 2-d Map
Clew: Path Tracing - Someone
guides you to a location, the
phone brings you back
20. Voice Recognition for everyone
Mozilla’s Common Voice:
Giant data set of voice recordings
from all accents, languages, and
speech ability
VoiceITT and Google’s Project
Euphonia
Designed to understand dysarthria
speech
21. Learn More
Web4All Conference:
Accessibility Research Co-located with
The Web Conference
Assets:
International ACM SIGACCESS
Conference on Computers and
Accessibility
Microsoft:
AI For Accessibility
Twitter: @Ted_Drake
Email: Ted_Drake@intuit.com
Notas del editor
Wiki Data: https://www.wikidata.org/wiki/Wikidata:Main_Page
Build a deep learning computer for $450 https://www.youtube.com/watch?reload=9&v=G6PtQis8GDo
iOS chip: https://www.wired.com/story/apple-a13-bionic-chip-iphone/
Android ML: https://developer.android.com/ml
Nvidia Jetson Nano https://developer.nvidia.com/embedded/jetson-nano-developer-kit
Artificial Intelligence can make complicated tasks simple. This is one of the most important benefits of machine learning and artificial intelligence.
Take a task that required dozens of steps and decisions and bring it down to a few. TurboTax, the tax software from Intuit, has a mission of completing a person’s tax return with a single step. This will require AI to import their financial records and determine the correct information. The user would simply verify the return and submit it.
Each simplification can makes a greater impact for customers with a disability.
Photo: Sri Kotilingeshwara Temple
Apple Watch contains two features that require no user input. The watch can detect irregular heartbeats and falls. In fact, not interacting with the screen is an interaction, for the fall monitoring it will send a request for help.
Photo: Ted “driving” an auto rickshaw in Mumbai
It’s not enough to use AI to complete a task, like object recognition or navigation. We need to make it actually work well and guide
Seeing AI is a great app: https://apps.apple.com/us/app/seeing-ai-talking-camera-for-the-blind/id999062298
But much of it’s functionality can be found with OrCam https://www.orcam.com/en/ and Lookout https://www.orcam.com/en/
But SeeingAI has a feature that makes it easier for people to successfully scan product barcodes. It uses AI to scan the object and detect the barcode before it is completely visible. It then gives audio directions to rotate the object until it can scan the code.
Accessible indoor navigation that adapts to emergency situations to ensure safe exit during emergencies. https://dl.acm.org/doi/abs/10.1145/3315002.3317569
Indoor wayfinding has remained a challenge for people with disabilities in unfamiliar environments. With some accessible indoor wayfinding systems coming to the fore recently, a major application of interest is that of emergency evacuation due to natural or man-made threats to safety. Independent emergency evacuations can be particularly challenging for persons with disabilities as there is usually a requirement to quickly gather and use alternative wayfinding information to exit the indoor space safely. This paper presents the design and evaluation of an inclusive emergency evacuation system called SafeExit4All that empowers people with disabilities (in addition to the general population) to independently find a safe exit under emergency scenarios. The SafeExit4All application drives an underlying accessible indoor wayfinding system with the necessary emergency evacuation system parameters customized to an individual’s preferences and needs for exiting safely from a premise.
Today’s ML and AI technology is making health care and education affordable
Change Dyslexia https://www.changedyslexia.org/en
Autism detection based on eye movement sequences on the web: a scanpath trend analysis approach https://dl.acm.org/doi/abs/10.1145/3371300.3383340
Satya worked with dentists to create a machine learning application that can differentiate oral diseases based on multiple factors and simplify the process of diagnosis and treatment. This allows dentists to see more patients and increase treatment efficacy.
This project was made possible by the low cost of Machine Learning technology.
Customized chairs for students at Spastics Society of Karnataka https://www.spasticssocietyofkarnataka.org/
Introducing live transcribe https://www.youtube.com/watch?v=jLCwjIaPXwA&t=16s
Open Source Live Transcribe Bringing Live Transcribe's Speech Engine to Everyone https://opensource.googleblog.com/2019/08/bringing-live-transcribes-speech-engine.html
Live transcribe gets sound alerts and transcription https://www.engadget.com/2019-05-16-android-live-transcribe-sound-alerts.html