1. DIGITAL COLLEAGUES FOR SMART AGING
| 1
Patrice D. Tremoulet, PhD
Director, Applied Informatics Group (AIG)
2. • What is a Digital Colleague (DC)?
– Enablers
– Human Augmentation history (abridged)
• 1st Use Case for DCs: Aging Workforce
– Motivation
– Scope of initial effort (& parallel projects)
– Approach & Team
• Questions / Feedback
| 2
Overview
3. Digital Colleague (DC) Vision
| 33/25/2015
A Digital Colleague augments human performance, by serving as:
1. an expert cog
2. a personal work assistant
3. a personal health coach
Like coworkers who can “read” each other, DCs discretely monitor
employees and provide customized assistance, tailored to the
current context, including the work environment, physical and
cognitive states, and ongoing task demands.
DCs alert employees to relevant new research, news, or products,
suggest tools and strategies that can enable better performance
and provide guidance to maintain and improve health.
4. DC Enablers
| 43/25/2015
1. A growing community is tackling the challenges
associated with building cognitive assistants
• Identify and share domain knowledge, customized based
upon understanding a particular employee’s interests/needs
2. Inexpensive wearable and environmental sensors
make it possible to reliably assess human physical and
cognitive states
• Human Performance Augmentation
• Cognitive supports for people with disabilities
• Personalized health support & health education
5. Human Augmentation History
1945 1960’s
2013 +
beyond
2002-2006
Vannevar Bush
As We May Think
describes “enlarged
intimate supplement
to one’s memory”
(memex)
J.C.R. Licklider
Man-Computer
Symbiosis
predicts “human
brains and
computing machines
will be coupled
together very tightly”
Doug Englebart
“Augmenting
Human Intellect”
calls for “improving the
intellectual
effectiveness of the
individual human
being” with computers
SRI ARC
established
DARPA
Augmented
Cognition
Human
Performance
Augmentation
in a wide variety
of domains
1990s
BRAIN
initiative
“decade of
the brain”
6. • Electroencephalograph (EEG)
• Electrocardiograph (EKG)
• Galvanic skin response (GSR)
• Pupilometry / Eyetracking
Augmented Cognition overview
• Goal: Maximize operator cognitive performance
in dynamic, complex operational environments
• Approach: Physiological-data based
assessment of operator cognitive state
– Detects, predicts, avoids overload to reduce
operator error and maximize effectiveness
• Benefit: Mitigate
negative effects of
cognitive overload
– Increase task
speed and
accuracy
– Improve critical
situation
understanding
Sensors
C2
System
User
SMART
7. Next-Gen Wearable Sensors
- Real-time physiological monitoring of body chemistries
Body chemistries and signature analysis through analytics will enable a wealth of
physiological and cognitive assessments not previously possible
Orexin A: alertness
Neuropeptide Y:
depression
Cortisol: stress
Dopamine & Norepinephrin: performance
•Flexible, conformal, unobtrusive form factor
•Real-time biomarker measurements
•Correlations to physical and cognitive states
that affect performance
8. 1st use case: Aging Workers
Benefits:
• Deep fund of work-relevant knowledge.
• Can help mentor younger employees.
• May be willing to work part time, saving employer costs.
• Health costs reduced when people stay cognitively active.
• High levels of engagement.
Challenges:
• Slight cognitive declines begin in the 50s
• Physical limitations may require accommodations
• User acceptance, usability of both wearable electronics and
assistive/augmentative technologies
| 83/25/2015
10. Aging Workforce in US
| 103/25/2015
• The Government Accountability Office projected in 2006 that
20% of the workforce would be aged 55 or older by 2015 [1].
• Population research indicates that over 75% of baby boomers
are planning to work past retirement age [2].
11. Aging Workers: Need for DCs
| 113/25/2015
• Employers want to retain valuable older employees, but both
lack knowledge about tools and strategies that can help
identify and accommodate for cognitive and physical changes.
• US universities in particular need to be prepared to
support an aging workforce.
0
5
10
15
20
25
30
35
40
45
2000 2005 2010 2015 2020 2025 2030 2035 2040 2045 2050
Percentage aged 60 or over
World Average United States of America Repblic of Korea
Many other
countries are also
expecting aging
workforces, e.g.
South Korea
http://data.un.org/Data.aspx?q=aged
+over+60&d=PopDiv&f=variableID%
3a33
12. DC for Smart Aging Opportunities
| 123/25/2015
• Seoul, Korea government officials are interested in
supporting Smart Aging applied research
• NSF Partnerships for Innovation: Building Innovation
Capacity program.
• Smart Aging Service System (SASS) provides
older workers with personalized, private
recommendations designed to help reduce health
risks and maintain or improve job performance.
• Enables older employees with valued skills and
experience to continue working.
• Reduces health costs by helping employees
take ownership of their healthcare.
• Provides a framework for developing
personalized health recommender systems.
13. Drexel’s multi-pronged approach
| 133/25/2015
1. Faculty develop prototype that detects and mitigates against
most common changes in cognitive function (under NSF
PFI:BIC)
2. CS PhD student builds Material Science Cognitive Assistant
3. Undergrad student(s) chart out the space of assistive
technologies that can support individuals with intellectual
disabilities
4. Faculty propose to develop new sensors, smart aging
metrics, data repository and analysis tools with IBM and Korea
14. Digital Colleagues for Smart Aging: Initial effort
Goal: Keep aging University workforce healthy and engaged
even when faced with changes in any of three cognitive
functions: memory, attention, processing speed.
Approach: Wearable and environmental sensors feed an
intelligent “Digital Colleague” that a) recommends
accommodations and strategies to enable continued
contributions, and b) provides health alerts and reminders.
| 143/25/2015
15. Initial Prototype Sensor Suite
| 153/25/2015
Commercially available products to be used in initial proof-of-concept prototype
16. Proof-of-concept prototype
| 163/25/2015
Digital Colleagues have four
major software components:
1. The robust data collection
system integrates outputs
of sensors into a data store
that supports rich queries.
2. The knowledge base
backend holds facts and
inferences.
3. The Digital Colleague Algorithms component translates information
about an employee into recommendations for assessments and/or
interventions.
4. The dialog interaction module communicates with the employee.
18. • Drexel faculty/staff:
– Yvonne Michael, Assoc. Prof, School of Public Health
– Ibiyonu Lawrence, Clinical Prof., Drexel Univ. College of Medicine
– Marcello Balduccini, Asst Prof, College of Computing & Informatics
– Gaurav Naik, Sr Research Scientist, Applied Informatics Group
• Industry Partners:
– Cognitive Compass (CEO, Madelaine Sayko)
– General Electric (Research scientists, Luis Tari & Alfredo Gabaldon)
– Independence Blue Cross (Sr. VP & CIO, Somesh Nigam)
– Evoke Neuroscience (CEO, David Hagedorn)
– Abilities Inc. subsid of Viscardi Center (CEO, Jessica Swirsky)
| 18
DC for Smart Aging Team
19. There will nevertheless be a fairly long interim during
which the main intellectual advances will be made by
men and computers working together in intimate
association… those years should be intellectually the
most creative and exciting in the history of mankind.
Licklider, J.C.R. (1960), “Man-Computer
Symbiosis” IRE Transactions on Human
Factors in Electronics, volume HFE-1,
pages 4-11.
20. | 20
Additional References:
1. Tishman, F.M., Van Looy, S., and Bruyère, M. 2012 “Employer Strategies for Responding to an Aging Workforce”,
National Technical Assistance and Research Center 2012 Report.
http://www.dol.gov/odep/pdf/NTAR_Employer_Strategies_Report.pdf
2. Saad, L. (2013). “Three out of Four US Workers Plan to Work Past Retirement Age,” Gallup / Economy Report, May
23rd, 2013. http://www.gallup.com/poll/162758/three-four-workers-plan-work-past-retirement-age.aspx
3. TIAA Cref - Aging Workforce Series, Health Fitness and the Bottom Line, July 2012 https://www.tiaa-
cref.org/public/pdf/AgingWorkforceHealthandFitness.pdf
4. Levin, Sharon G. and Paula E. Stephan, “Age and research productivity of academic scientists,” Research in Higher
Education, 1989: 30(5): 531- 549.
5. Sörensen, L.E., Pekkonen, M.M., Männikkö, K.H., Louhevaara, V.A., Smolander, J., Alén, M.J. “Associations between
work ability, health-related quality of life, physical activity and fitness among middle-aged men,” Applied Ergonomics, Nov,
2008: 39(6):786-91. http://www.ncbi.nlm.nih.gov/pubmed/18166167.
21. DC HCI Challenges – Sensor Technologies
• Unobtrusive, comfortable, but ruggedized form-factors
– Robust when dusty, wet, tapped/hit, etc.
• Largely autonomous operation. Should be able to forget
you are wearing sensors…but:
• Could it be useful to cue wearers into a potential
health problem? Under what circumstances?
Configurable?
• Replacement notifications and failure indicators –
how delivered and to whom?
• Wearer “Reset” option?
• Data download notification – to whom? Privacy
issues?
22. Additional DC HCI Challenges to explore:
• Do participants know about aging-related cognitive decline
and how it may impact work?
– How would participants want to interact with their own
health data?
– What sorts of privacy safeguards would they expect?
• How and when would they like to provide information about
themselves that sensors can’t currently capture (e.g. job
satisfaction ratings)?
• What factors should influence how information is requested
and guidance is presented (e.g. individual preferences,
current capabilities, the types and degrees of limitations,
types of recommendations), and how?
25. Logistics
• Determine pre-mission readiness
• Maintenance of safety during mission
Commander monitoring
health of squad
Accelerated Learning / HMI design
• Customize training based on cognitive
states of trainees
• Develop interfaces that support low
cognitive workload
Dynamic modifications to training
exercises to speed learning
Medical
• Early Medical Problem Detection
• Triage
• Accident Medical SA
• Recording of injury event & treatment
for use at all echelons of care
Pilot showing symptoms of Hypoxia
Monitoring applications
Developing technologies to collect and take action on
health and readiness data of individuals
In-Field Injury
Screening
Commander &
Medic SA Sys.
Learning & HMI
design
-Continuously
-With Near-Zero Footprint
