For Media Design Practices Supersense! Project Studio, Spring 2013, Art Center College of Design.

1. Supersense! Context
Massive Sensing and its Implications
Philip van Allen, Media Design Practices, Art Center College of Design

2. Supersense! Context
Massive Sensing Context - Now/ Future
Machine Sensing
Massive Sensing
Internet of Things (IoT)
Human Perception & Science

3. Supersense! Context Machine Sensing
Sensing Now
What sensors are in your phone?
Discussion: Identify all the sensors in your phone

4. Supersense! Context Machine Sensing
Sensing Now
What sensors are in your phone?
multi-touch
internal moisture/temperature
GPS
sound
accelerometer – motion in x/y/z
gyro – orientation and position
ambient light
proximity
compass
camera still/video
buttons

5. Supersense! Context Machine Sensing
What is a sensor?
Interfaces with an environment
Measures a physical property
Transduces that property into a numeric value (typically)
Has a certain range and resolution

6. Supersense! Context Machine Sensing
Sensing Now
Types
location: GPS, RFID
environmental: moisture, temp, humidity, light, sound, altitude, etc.
position/motion: acceleration, distance, pressure, orientation,
compass, etc.
energy consumption: Kill-A-Watt
vision – facial recognition, objects
language – Siri
gesture - Kinect, LeapMotion

7. Supersense! Context Machine Sensing
Sensing Future
Major corporations are actively investigating possible applications of massive sensing. Here’s one perspective from
IBM.

8. Supersense! Context

9. Supersense! Context Text
Sensing Future - IBM Videos
Overview: http://youtu.be/wXkfrBJqVcQ
Touch: http://youtu.be/Gg3tmZrwbDs
Sight: http://youtu.be/YwfJVwknvRo
Hearing: http://youtu.be/-oKfWIgDTFs
Taste: http://youtu.be/DNz23XXLa1E
Smell: http://youtu.be/RYkSvNKdyBM

10. Supersense! Context Machine Sensing
Sensing Future
Types
location inside buildings: people and things – micro location
touch: input & output
vision: scene & object recognition
smell:
trace of a person or thing that was recently in a place
medical: cancer, health status, old/young, etc.
chemicals of any sort
taste: food, water, nutrition, soil, any chemical
sound: fuller language generation/recognition
mimic sensors in nature

11. Supersense! Context Massive Sensing
What is Massive Sensing?
Billions of connected sensors
Public/Private
Low cost
It’s here already

12. Supersense! Context Massive Sensing
Massive Sensing Now
Traffic displays are an integration of data speed and location data from driver cell phones + sensors in the roads +
feeds from caltrans & news

13. Supersense! Context Massive Sensing
Massive Sensing Now
Web Analytics gather “clickstreams” from millions of people every minute as they browse the web

14. Supersense! Context Massive Sensing
Massive Sensing Now
How The "Internet Of Things" Is Turning Cities Into Living Organisms
When city services can autonomously go online and digest
information from the cloud, they can reach a level of performance
never before seen. First up, water systems that automatically know
when it will rain and react accordingly.
This is an example of massive, AUTONOMOUS BEHAVIOR as well as sensing.
What are some other examples of massive sensing?

15. Supersense! Context Massive Sensing
Massive Sensing Future
Enabling Technologies/Contexts
sensors smaller, cheaper, more energy efficient
low-power wireless mesh networks
energy harvesting techniques (e.g. Powercast Wireless Sensors)
economies of scale: sensors in phones, cars
internet infrastructure growth
nano-technology enables embedding in everything
clothes, paint, our bodies, ???
language recognition, AI, big data

16. Supersense! Context Massive Sensing
Massive Sensing Future
Enabling Technologies/Contexts
social acceptance: life-logging, cameras everywhere, etc.
corporate r&d - IBM, Intel
need for cost efficiencies
environmental monitoring/mitigation
climate change
earthquakes/tsunamis

17. Supersense! Context The Internet of Things (IoT)
Internet of Things Overview
Everything has an ID and generates data
It’s all sent/received via the cloud
Data access in & out is asynchronous
Range of models:
one-to-one
one-to-many
many-to-many
IDs = RFID, IPv6 address, custom ID
IoT used to be thought of as being based on RFIDs (see Shaping Things), now it’s active devices
Data becomes a service

18. Supersense! Context The Internet of Things (IoT)
IoT Data Services
COSM “Turn these flows of
information into action by
ThingSpeak triggering online Applications.
Sen.se Things become smart,
versatile and interact in real
time with remote things.”
--Sen.se
DEMO: Show IOBridge on web display, as well as in NETLab Toolkit

19. Supersense! Context The Internet of Things (IoT)
IoT Action Services
Exercise: Everyone pair up & create an IFTTT recipe

20. Supersense! Context The Internet of Things (IoT)
IoT Hardware
IOBridge
NinjaBlocks
Twine
Belkin WeMo
Insteon
Almond+
DEMO: IOBridge, Belkin WeMo

21. Supersense! Context The Internet of Things (IoT)
Most IoT Applications Suck
Mashups: turning your data into something useable and useful.
“We believe that looking at or processing data feeds one at a
time doesn’t help you solve real life problems. Most often you
need to combine data from several different sources in order to
understand what is really going on, take smart decisions, make
really useful, useable and non annoying applications.”
--Sen.se Blog

22. Supersense! Context Human Perception
Engaging with Human Perception
Why work with scientists?
different perspective
interesting new methodologies
creative synthesis
an interesting context to work with
need to maintain expertise/independence of design
Understand how people will perceive sensing data to
make sophisticated systems
Can use it as a metaphor for design - biomemtics,
machine human interfaces
How do people perceive the world?

23. Supersense! Context Human Perception
Perception is different than sensing
Multiple Inputs
Knowledge
Context
Actively probing env.
Apple or Potato?
Reverberation
Eat an apple slice and potato slice while holding your nose - can you tell the difference in taste?

24. Supersense! Context Human Perception
Seeing with
your skin
Sensory
Substitution
Paul Bach-y-Rita
1972
Video pixels are turned into vibrators on person’s back - note that the person is able to move the camera and learn
what things “look” like.