"Siri, did I leave the oven on?" UX for the connected home (updated for IA Summit 2013)

“Siri, did I leave the oven on?”
Mundane UX for the connected home
@clurr #connectedhome
Tuesday, 4 June 2013
Weʼve become accustomed to using technology like the web and mobiles to keep us in touch with the people and activities
that are important in our lives. Still one thing thatʼs relatively unconnected - home - big dumb box of mostly dumb things that
donʼt talk to us, or each other. But thatʼs changing.
This talk is about the challenge of making new technologies make sense to the mass market.

Wiredʼs number 1
London startup 2012
@alertmesays
You probably havenʼt heard of AlertMe. Last year we were nominated Wiredʼs number 1 startup in London.

One reason you might not have heard of us because we are mostly B2B: we create hardware and services that other
companies sell on to their end customers. Currently, you can get our products and services from British Gas, Loweʼs and
Essent. We cover areas like heating, energy monitoring, home controls and monitoring, and data analytics. Weʼll ﬁnd out a bit
more about some of this later.

Opinions are entirely my own :)
And I canʼt take credit for many of interfaces I will be able to show today.

Whatʼs a connected
home?

Embedded computing in everyday objects...
...connected up to the internet
Sensors and controllers around the home, embedded computing in everyday objects, and connecting it all up to the internet
so you can access and control it via web and phone. Lots of things you can do here...

Understand energy
use...
Energy clamp
In-home display
Smart plug
Web and mobile interfaces
For sake of time, Iʼll show some AlertMe examples to give you a ﬂavour of the kind of thing I do.
This is AlertMeʼs current energy service. Itʼs made up of sensors, displays, smartphone and web apps.
(We make apps because they are a better experience for a control system right now.)

Control your
heating...
Thermostat
Web and mobile interfaces
This is AlertMeʼs current remote heating controller. Stuff magazine gadget of the year.

Camera
Contact sensor
Key fobs
Motion sensor
...secure your home...
AlertMe home security.
Thereʼs more... lighting, switches, locks, appliances, catﬂaps...

...on a combined platform
This is the web UI for the Iris system we make for Loweʼs. I canʼt take any credit for this UI, which is a custom version for
Loweʼs and was well developed before I joined AlertMe.

This home
automation stuff
has been around for
ages though, hasnʼt
it?

Connected
home
technology has
existed since at
least as far
back as 1975...
This is X10 Powerhouse for the
Commodore 64, from 1986.
It let you schedule lights and appliances to turn on and off, control a burglar alarm and thermostat, and could be operated
remotely by telephone. Those are pretty much the things Iʼm working on right now. Except the telephoneʼs got a bit smaller
and now we have the internet.

...but you had to be rich...

...and/or a geek
For most people, the beneﬁts just didnʼt outweigh the cost

“Little bits of smartness”
Things are changing... getting cheaper, wireless... better designed...
Big smart home concept isnʼt with us yet but you can already buy a range of everyday things like bathroom scales, baby
monitors and electrical sockets with connectivity and even intelligence built in.

We have a metaphor for the
“remote control for your life”
Donʼt underestimate how powerful this is. Mobile is central control platform: whether youʼre out and about, or on the sofa.
whatʼs the point of embedding connectivity and smartness around your home only to have to sit down at a desk to use it
on your Commodore 64?

Interconnectivity is still a challenge...
but so is understanding and
delivering what the mass
market actually needs
NB: big UX
opportunity.

4 key UX challenges
There are many, here are 4 of the big, general ones for designing interoperable systems for the mass
market.

UX challenge 1:
Make it feel like home
Home is a very personal context.

AT&T Digital Life

Greenwave
Reality

Home Automation Ltd
(yes, really)
Bit unfair to pick on these guys, they are a relatively small outﬁt. But this screen sums up a lot of what I
think is wrong with home automation.

Spot the design
metaphor?

•System has users and peripheral devices
•Users have access permissions and are IN or OUT
•Their goal is to program the home for optimal
efﬁciency
Um, yeah.

ʻRomantikʼ mode
an engineering solution to a human non-problem
Modes are a common smart home feature. But they require a lot of planning and advance conﬁguration. Which isnʼt very
sexy.

Real life is too
messy to program
•People are generally a bit disorganised and bad at
predicting their future needs
•Life is full of contradictions and exceptions
•Devices are shared, and lent
•Whoʼs allowed to do what is negotiated and ﬂexible,
not completely codiﬁed
e.g. Little Jack isnʼt normally allowed to watch that much TV, but today heʼs ill so youʼre feeling sorry for him
e.g. The sheets ought to be washed but everyoneʼs busy so theyʼll do for a bit longer

“My teenagers skulk in their
bedrooms. Theyʼre not out, but
theyʼre not really in either...”

We already have a perfectly
good metaphor for the
home:
Itʼs the
home
This one happens to be my home. I donʼt
want to log into it, become a super user, or
worry that itʼs going to crash or need
debugging.
sudo open-window
Last place in the world I want to feel out of control... and we all know how people often feel out of control of computers
when they are too hard to use or do things we donʼt understand.

This is not a mass
market solution
New systems like WeMo are neat but
basically better-designed early adopter
kit
You deﬁne your own problem and conﬁgure the algorithms to execute it.
Iʼm prepared to be proven wrong, but I donʼt think this is the mass market solution

UX challenge 2:
Making data visible
has social
consequences

•There is often more than
one person in a house
•They have interpersonal
dynamics
•They may want different
things
•Relationships are
smoothed by not
necessarily knowing
everything about each
other
The home is a complex social context

Connected home technology surfaces information
about what is happening in the home
These are the unknown measures from my Withings scales: the ones it wasnʼt able to recognise as a known user.
This data is anonymous but based on time of use and estimates of mass I can infer two things from this

My cat sits on the scales

My cleaner is watching her weight
and neither of them have any idea that I know this.

•Itʼs often possible to work out
who is in, out, turning the
heating up all the time, or on
the Xbox at 4am
•When parties have different
ideas about how things should
be, that surfaces tensions
21 °C 19 °C
Thatʼs all pretty innocuous, but add in other simple data like when the burglar alarm was set, and energy monitoring, and you
can ﬁgure out...
Itʼs a healthy and necessary part of most relationships to have the right to some private space, and to ignore or pretend not
to notice some of the other personʼs behaviours. Technology makes this harder.

Tension between the person
who uses the energy monitor
and the people who use the
appliances is common
Tumble dryers are a particular source of angst.

Who came in at what time?
(Did they look drunk? Was anyone with
them??!)
How long did the cleaner
really stay?
If this information is up on
the internet, who might get
access to it?
Presence surfaces trust and privacy
issues

UX challenge 3:
The mundane should
not demand too
much attention

•A lot of what goes on in the
home is actually pretty
unremarkable and
mundane
•We develop routines to
help us stay on top of the
boring stuff without too
much conscious effort
•This allows us to save our
attention for important or
interesting things

My washing machine is
as needy as a burglar alarm
It thinks washing is most important and interesting thing in my life. It beeps when itʼs ﬁnished a load. Itʼs a bit aggressive, but I
could let that go. But it doesnʼt stop beeping until you empty it. It expects you to drop everything and come running, right now,
because the washing must come out IMMEDIATELY. This is appropriate behaviour from a burglar alarm, but not a washing
machine.

What if you had a whole home full
of attention seeking devices?...
one device, irritating, but we accept it.
a whole home full of devices with no manners.... developing some new and interesting ways to break down?

This is attention seeking
Too much work for most people.

User instructions:
1) Ignore it
We need to design things that need less attention, not more.
Wattbox - intelligent heating controller (prototype hardware shown).
Nest: works off motion, light, and settings you choose in ﬁrst week or two. If you donʼt bother to turn the heat down when you
go to bed, or your heating controller sits somewhere people donʼt pass by, it might not learn correctly. Wattbox uses electrical
activity to infer whether anyone is in, and whether they are up and about. When we're in and awake we're usually using
electricity above baseload, itʼs a good proxy for occupancy and activity.
“Donʼt make me think” harder about my heating... HEATING IS BORING!!!

UX challenge 4:
We donʼt understand
how to use half the
stuff in our homes
anyway

Most of us
understand our
heating systems
about as well as
this guy does
This is a ring tailed lemur. Itʼs a native of Madagascar, and it knows about as much about domestic heating as the average
human. I am not being ﬂippant here.

Ring-tailed lemurs in a
zoo in south west
England learned to turn
up the temperature on
their heating thermostat
when it got cold
http://www.thisissomerset.co.uk/Smart-lemurs-learn-turn-thermostat-cold-snap/story-17929368-detail/
story.html#axzz2Nod8uQoU

“When itʼs cold
you need to turn
the thermostat
up.”
This mental model is
completely wrong
This is what many humans
do too:
A thermostat is an automatic switch.
Most people treat it like a valve: turn up dial, get more.

“My thermostat is
too confusing to
use so when I
want to turn the
heating up I put it
in the fridge.”
NB: this might sound silly but
itʼs far more logical:

Sometimes people just
have illogical habits or
beliefs that challenge our
assumptions about what
to design:
“I donʼt set my
burglar alarm
when Iʼm only
going out for a few
hours.”
People are sometimes going to use it in the ways you may consider irrational. Engineers ﬁnd this bafﬂing, but you have to
deal with it.

So how do we ﬁx
this mess?
Iʼve given this talk in the past and had the response ʻyeah, what are you actually doing about this?ʼ.
I am working on design concepts for a more humanistic connected home experience.
I canʼt show you most of that, but I can talk about some of the practical design experience Iʼve gained along the way.
Some of this is applicable to more general internet of things/ubicomp type UX design.

UI/visual design
screen layout, look and feel
Platform design
conceptual architecture spanning multiple
services, devices, common design
principles
CX design
customer lifecycle, customer services,
integration with non digital touchpoints
Productisation
audience, proposition, objectives,
functionality of a specific service
Industrial design
physical hardware: capabilities and
form factor
UX/interaction design
architecture and behaviours per
service, per device
Interusability
interactions spanning multiple
devices with different capabilities
Many layers of
connected home UX
[talk through]
I tend to use term ʻservice designʼ to encompass the lot.
A lot of my work is centred on the web and mobile UIs because these are the touchpoints users will interact directly with the
most. But if you just think of it as doing web and mobile UI design, you miss a lot and risk creating a lot of problems.
Hence why Iʼm called a service design manager.
Iʼm going to talk about 3 of these, reflecting specific challenges I deal with...

“People have to
understand it
before they can
want it”
Denise Wilton, BERG
UI/visual design
Platform design
principles
CX design
Productisation
Industrial design
form factor
service, per device
Interusability
this is about making it make sense to end users.

Productisation is the extent to which
the supplier makes the user value
explicit
for some speciﬁc bits of hardware, like an energy monitor, thereʼs a close mapping between function
and value. it does one thing, hopefully well. itʼs easy for people to understand what they do.

This is not a contact sensor
this is a thing that tells you:
•when an intruder has forced your front door open
•when your child has opened her window in the middle of the night
•when someone is trying to steal your guns
This is hard to do for general purpose devices

In areas where they donʼt have expert knowledge
consumers tend to buy products, not
tools
Product Tool
A product says ʻHere is the valueʼ, and comes preconﬁgured to deliver it.
A tool provides functionality. You ﬁgure out the value, and how to get it. (Weʼre back to the computer
metaphor.)
Nothing wrong with making tools but they are less likely to go mass market

You can productise the box...
but you also need to productise the
service
• 1 SmartThings Hub
• 2 SmartSense Multi (Open/
Closed, Temperature, Vibration)
• 2 SmartSense Presence
• 1 SmartPower Outlet
• 1 SmartSense Motion Detector
• 2 SmartSense Moisture
Detectors
• 1 SmartSense Motion
Closed, Temperature, Vibration)
Closed, Temperature,
Vibration)
• 1 SmartSense Motion Detector
Smart Things are trying to do this.
Most of the contents of these kits are pretty similar:
hub, open-closed/vibration/temperature multi-sensors, presence, motion.
Productising the box is a start. Iʼm very interested to see what the service UIs are like when you
connect this all up. Is it a generic UI for all 3? Or does it have customised functionality for each
service: so for home security you get speciﬁc instructions on setting up an alarm, for home watch you
get a ﬂood alarm, temperature warnings, an earthquake alarm; and for family life you get the “oh crap
the dogʼs escaped” alarm?

UI/visual design
Platform design
principles
CX design
Productisation
Industrial design
form factor
service, per device
Interusability

Interusability:
• composition
• consistency
• continuity
Cross-Platform Service User Experience: A Field Study and an Initial
Framework. Minna Wäljas, Katarina Segerståhl, Kaisa Väänänen-Vainio-
Mattila, Harri Oinas-Kukkonen MobileHCI'10
Model I want to draw on here which I have found incredibly useful: interusability.
I know we like to kind of put usability in a box now and not use it as a catchall for broad UX, but bear with me and donʼt be put
off. I use the word “interusability” because the people who came up with it called it interusability. I like to think of it as really
talking about a type of cross-device digital service design.
Anyone who works in cross platform design should read the paper cited here, if you havenʼt already. The examples are a bit
out of date now but the principles are still highly valid.
Talks about 3 components: composition, consistency, continuity

Composition
• Figuring out which devices your service needs
• Figuring out what each device does
vs
As a designer of smart services, one of your first tasks is to think about composition: what devices will you have, and which
ones will do what.
Your decision will be influenced by whether any parts of the system need to have particular form factors/be used in certain
contexts, cost, whether any parts of the system need to work if they are offline, user expectations.
Displays and controls usually add to the cost, so itʼs often cheaper to handle user inputs and outputs on a remote mobile or
web UI.
http://www.tado.com/en/
example: tado thermostat has no UI, itʼs all on the phone. probable reasons: itʼs expensive to make a good thermostat UI,
(and no-one understands the bad ones), so just make a good phone UI, which is relatively cheap to do (and gets you round
some of the UI consistency challenges Iʼll talk about in a minute). Thereʼs a certain purist elegance to this decision but tiʼs a
brave move: if you donʼt have your phone to hand, or itʼs not working, or youʼre a guest in the house without access to the
phone UI, you canʼt adjust the heating.
AlertMe chose differently: we have a fairly standard thermostat with a conventionally bad UI but also the phone and web apps,
which offer a much better experience (the one you see here looks rather plain as itʼs our unbranded version). This means that
you, and your guests or other residents without smartphones, can still use it as a conventional thermostat. Itʼs less elegant
(you will at some point encounter a UI that has been compromised by the need to keep the thermostat price down), but itʼs
pragmatic.

Consistency
Adapting interfaces for different types of
device, but still making them feel like a family
Nest wall stat: twiddly knob on the wall that clicks. Touchscreen: up and down arrow. (Twiddly knobs are inefﬁcient and
inaccurate on touchscreens). BUT it still makes the same click :)

Continuity
Up to date data and
content across all
platforms. Fluent
cross platform
interactions.
Battery limitations
impose possible 2
minute delay!
Perhaps the biggest challenge is continuity.
If I interact with the service on one device, all other devices reﬂect that change in state. e.g. if I turn the target heating temperature up on
my wall thermostat, youʼd expect the new temperature to be immediately reﬂected on the smartphone too.
But sometimes this isnʼt technically possible.
In the case of the AlertMe heating system, there can be a delay of up to two minutes before the smartphone app is updated. This is
because the wall thermostat runs off a battery (as is normal in the UK), and sending data to the network uses a lot of power so it only
does it every two minutes. If it sent it more frequently than that, it would run the battery down very fast. We could make mains powered
controllers, but engineers donʼt like those in the UK as they are more complicated to install. So for the time being, the UX is a
compromise, albeit a small one as the main use of the smartphone app is when you are not standing in front of the wall thermostat, and
2 minutes isnʼt a long delay in turning the heating on.
The important thing is to ensure that users are as informed as possible about whatʼs going on.

Thermostat > hub >
service > phone UI
but can be separate API calls
Boiler (furnace) >
thermostat > hub >
service > phone UI
3rd party weather
service > phone UI
A complex
service can
have many
potential
points of
failure
many points of potential connectivity failure: hub offline, thermostat offline, thermostat lost connection to boiler;
also individual API calls can fail like target temperature, current temp. and sometimes some are slower to load than others
and that can be outside our control.
so there are times when you effectively have missing parts of the service, or are waiting for things to respond, and you have to
deal with this in the UI.
itʼs not like many of the apps many of us work with, where cached data may still be useful. out of date data can be a big
problem. it can lead you to believe something is on when itʼs off, or ok when itʼs not ok. it's perhaps not a disaster if it's 5 mins
out of date for a heating app, but what if it's your burglar alarm,or an emergency alarm for an old person?
rule of thumb: donʼt show old data as this can be misleading, donʼt imply that a change has been made before it is
completed,
figure out which data can be missing without rendering the service useless (like weather)
previous app loaded screen and then filled it with data. [screenshot]
i think this feels mainframey, and wanted the screen not to load until the data was there. my interaction is with the service, not
interface plus data.
but sometimes that would mean that it didnʼt load at all for ages and that would be really frustrating. decided what we could
live with (E.g. weather not updated) and what was essential to service experience
then what happens if you change one setting, e.g. turn from off to auto? more than one thing may update (e.g. mode, and .
interface needs to update to reflect status change but dontʼ want to show this change until you know itʼs been applied. but
some data not available, so end up with some blank data. it's not great (see loading on RHC homescreen when changing
thermostat setting).
hardware constraints can be limiting...nest is mains powered so can use wifi and connect more instantaneously (file under

UI/visual design
Platform design
principles
CX design
Productisation
Industrial design
form factor
service, per device
Interusability
aka the really big
IA challenge

If you’re just making a single service that supports
a limited set of devices, your platform can consist
of device control/data APIs available via web/
mobile interface

But remember, this can get complex
look at this again for a minute...
If you want to offer multiple, overlapping services, in which devices can do different things as part of different services and
users can have different sets of services, then you start to need some kind of underlying logic to tie it all together.

Empty space = more future devices?
Nest looks like it may be the beginnings of a platform.

• services: intruder alarm, lighting, garden sprinkler, heating/cooling, gun cabinet,
smoke alarm, energy usage, window blinds, Grannyʼs alarm...
• devices: motion sensors, lighting, sprinkler, thermostat, cabinet sensor, smoke
detector, energy monitor, blind controls, panic button...
• controls:on/off, up/down, less/more, timer/schedule, hot/cold, set/unset
• notiﬁcations: alarm, message, status...
• presence: whoʼs in/out, nearby/far away, available/unavailable, authorised/not
authorised
• contacts: people who live in the house, have access permissions to the physical
property or the service UI
Across a range of home services
you will have constructs like:
Within a service like this we have... [these things]
They are interrelated in potentially complex ways.

Add to: lighting controls?
security system?
both?
If you want to be smart in the ways that you offer services up...
... the more this conceptual model needs to be codiﬁed somewhere.

User tasks are heterogeneous and
overlapping
• device/device group based: turn up the TV, turn off all
the lights
• location based: set alarm downstairs, turn off outside lights,
lower blinds on west side of the house in afternoon
• time/state based: activate security lights when iʼm away
• optimisation: keep the house temperature comfortable, use
energy efﬁciently
• authorisation/presence based: lock the gun cabinet
when the adults are not at home
• person based: tell me if Granny hasnʼt got out of bed, tell me
when Jake gets home from school
There is no one hierarchy that supports all of this. You either force people to think in terms of your hierarchy... or you design
something that supports the way they think... without overloading them with options.

notiﬁcationsservices
controls
devices presence
contacts
user needs
The big IA challenge:
creating the UX logic
that bridges the two
Creating the UX logic that allows you to support all these things, and create great UXes, for services that you have already and
those you donʼt have yet, is the big IA challenge
Itʼs not stretching it too much to call it the ontology/domain model of the home
Until we make some headway here, most people wonʼt consider it worth the pain of buying wholesale into the technology.

Interoperability beyond the walled garden
makes this an even bigger challenge
Right now all these things are walled gardens... realistically, who wants their entire house to come from Samsung, AlertMe,
even Apple?
There isnʼt an open platform: there are some open network protocols for connecting devices, but Iʼm not aware of anything that
helps ﬁgure out how they work together.
But even with open services, something is going to have to happen to ensure you donʼt end up in a mess of different UIs and
metadata and control structures from different providers.
Thatʼs a really hard problem.

1 brilliant quote

“People donʼt want more
control of their homes.
“They want more control of
their lives.”
Scott Davidoff, Min Kyung Lee, Charles Yiu, John Zimmerman, Anind K. Dey: Principles of Smart Home
Control (Ubicomp 2006)

Burnt pie by Jet Lim
X10 Powerhouse from commodore.ca
Internet fridge from fuckyeahinternetfridge.tumblr.com
Messy House by Elizabeth Table4Five
Trapped by Merina
Computer by Phil Gold
Crying child by eggonstilts
Army from hdwallpapers.com
Tea cosy by Brixton Makerhood
Teeth by ktpupp
Sleeping by Stan
Frustration by dieselbug2007
Washing machine ﬁrmware error by Adam Crickett
Houses by Peter O, Clive Darr, hollandhistory.net
Usabilty lab by Leanne Waldal
Burglar by homesecurityfocus.com
Mongkok advertising by Slices of Light
Posh house by Savant Toronto
Teenage bedroom by Wendizzle
HAL smarthome by james.lipsit.com
Jack Black from bradley.chattablogs.com
Holiday home: geograph.co.uk
Older woman: soylentgreen23
Ringtailed lemur by digidave
Ringtailed lemur 2 by Tropiquaria Zoo
Thanksfor
thephotos

S Intille, The goal: Smart people not smart homes (2006)
http://web.media.mit.edu/~intille/papers-ﬁles/IntilleICOST06.pdf
Minna Wäljas, Katarina Segerståhl, Kaisa Väänänen-Vainio-Mattila, Harri Oinas-
Kukkonen: Cross-Platform Service User Experience: A Field Study and an Initial
Framework (Nordichi 2010)
http://bugi.oulu.ﬁ/~ksegerst/publications/p219-waljas.pdf
Colin Dixon, Ratul Mahajan, Sharad Agarwal, AJ Brush, Bongshin Lee, Stefan
Saroiu, and Victor Bahl, An Operating System for the Home (NSDI, USENIX, April
2012)
Pertti Huuskonen: Run to the Hills! Ubiquitous Computing Meltdown
(Advances in Ambient Intelligence, 2007)
Peter Tolmie, James Pycock, Tim Diggins. Allan Maclean, Alain Karsenty,
Unremarkable Computing (Ubiquity, 2002).
Genevieve Bell & Paul Dourish: Yesterdayʼs tomorrows: notes on ubiquitous
computingʼs dominant vision (Personal and Ubiquitous Computing, 2006)
http://www.ics.uci.edu/~jpd/ubicomp/BellDourish-YesterdaysTomorrows.pdf
Scott Davidoff, Min Kyung Lee, Charles Yiu, John Zimmerman, and Anind K. Dey:
Principles of Smart Home Control (Ubicomp 2006)
T Saizmaa, A Holistic Understanding of HCI Perspectives on Smart Home,
Networked Computing and Advanced Information Management, 2008. NCM '08
Thanksforthe
research

Thank you
@clurr
claire@clairerowland.com
Thanks to: Alex von Feldmann, Fraser Hamilton, Martin Storey, Naintara Land and Anna
Kuriakose who have contributed insights, thinking and research to this presentation

"Siri, did I leave the oven on?" UX for the connected home (updated for IA Summit 2013)

Recomendados

Recomendados

Más contenido relacionado

Destacado

Destacado (20)

Similar a "Siri, did I leave the oven on?" UX for the connected home (updated for IA Summit 2013)

Similar a "Siri, did I leave the oven on?" UX for the connected home (updated for IA Summit 2013) (20)

Más de Claire Rowland

Más de Claire Rowland (20)

Último

Último (20)

"Siri, did I leave the oven on?" UX for the connected home (updated for IA Summit 2013)