1. Contextual
Intelligence Bo Begole, Ph.D.
Samsung Research America, UX Center
bo@begole.net
Korean HCI Society Conference
31 January 2013

2. Businesses strategies for technologies
that bridge physical and digital worlds
• Capabilities, Techniques,
Limitations
• How to exploit the trends
• Key value propositions
• Case Studies in
• Contextual Intelligence
• Hyper-Personal information
filtering
• Predictive personal marketing
• Supply chain disruption
• Unanticipated device
interoperation
• Consumer decision support
FT Press, ISBN-13: 978-0-13-706443-4

3. 20 Years ago, Ubiquitous Computing
was a dream …
3
Olivetti Active
Badge
PARC Tab PARC Pad PARC “Deathstar” IR
Network Hub
“The most profound technologies are
those that disappear.
They weave themselves into the
fabric of everyday life until they are
indistinguishable from it.”
-Mark Weiser, Xerox PARC
http://www.youtube.com/watch?v=b1w9_cob_zw

5. Too Many Devices!
Too Much Information!
Too Many Friends!
Too Many Networks!
Too Much To Do!

6. Context-Aware services reduce
technology overload
Systems detec users’ locations
to help them
– Find colleagues
– Find resources
– Seamlessly connect to
nearby devices/services
– Route phone calls
– Tag information creation
by context for recall
– Diary of daily activities (1994) Bill Schillit, Norm Adams, Roy Want
Context-Aware Computing Applications
Workshop on Mobile Computing Systems and Applications,
6

7. 7
1990 92 96 2000 02 200494 98
Commercial penetration
of Ubiquitous Computing
research is
accelerating
Context-Aware
research

8. PARC
Magitti
8
Context-Aware
research
1990 1092 96 2000 02 04 06 0894 98
Google
Now
Mind-
meld
Commercial penetration
of Ubiquitous Computing
research is
accelerating
12
CALO
ends
Siri
launches

9. Context-aware is now table-stakes. New
frontier: Behavior, Activity and Goal-Aware
Context-Aware
Sensor-triggered services with
no intelligence:
Current/future location; time
Key: distance in time/space
9
IncreasingIntelligence
1990-2004
Predictive analytics from
behavior traces:
rank info by frequency/recency of
people, apps, docs, locations; …
Key: distance from “normal”
Behavior &
Activity-Aware
2005-2015
Semantic understanding
of workflows and objectives:
writing a proposal; filling sales pipeline;
shopping for gift; fostering relationships;
…
Key: distance from “objective”
Goal-Aware
2015+

10. Sternberg’s Tri-archic Theory of
Intelligence
• Analytic Intelligence
– Logic, sequences, calculations, …
• Synthetic Intelligence
– Creativity, intuition, …
• Contextual Intelligence
– Practical “street smarts”
– Understanding relationships and utility
– Achieve goals with available resources
10

11. Contextually Intelligence Assistant:
Pertinent and Actionable information
11http://youtu.be/FPL7uaKYNiU

12. Nice vision, but it’s been 20 years…
• Some classic problems are still unsolved…
– Privacy
– Missing information
• … and bigger problems have arisen
– Latencies
• Delays to build model
• Delays in detecting pattern change
• Delays to detect situation
– Accuracy: typically 85% at best
– Actionable? Tell me something I don’t know
12

13. Privacy Concerns
• Security and access control is
only half the story
• Impression Management
– Confusing interfaces,
contradicting policies
– What is this thing telling other
people about me?
• Economic Risk
– Unwanted sales pressure
– Higher insurance rates
13

14. Current digital assistants have to ask
for every detail
14
“The address was
there in my contact
card, but Siri kept
telling me, ‘I don’t
know where your
home is.’”
Why do I have to tell it everything?
Why can’t it figure things out and confirm?

15. Latencies
• Pattern models take time to build
– Frequent patterns build quickly
– Weekly, monthly, annual patterns build slowly
• Changes take time to detect
• Recognition is often slower than human
intelligence
15
GPS
NFC
EMAIL
Calendar

16. Accuracy – best are wrong 15-20%
16
• Fine for advertising or product recommendations
• What is good enough for content retrieval?

17. Optimal
Timing
Action
Context
Reminders in
actionable
context
Pertinent Information
& Actionable Context
n Communication: tell me
before someone becomes
unavailable
n Health & Fitness:
suggestions of healthy
options not already in
practice
n Technician: take preventive
parts before repair visit
n Advertisement: Deliver ads
when person is actually “in
the market”
• Health plan
• Task list
• Reminders
• Coupons

18. Technology alone cannot solve these
• Privacy – Requires understanding real user fears
• Missing information – Human conversational repair
• Latencies – Give users control of their pattern models
• Accuracies – Adjust UX depending on application need
• Actionability – Don’t be a nag. What do they already know?
What will achieve their goal? How is their goal defined?
HCI Grand Challenge:
HCI research methods can create the knowledge
to emulate human contextual intelligence
18