Talk to inspire creating surprising new user experiences (like the examples in the talk) that use powerful new technologies (like AI, AR, VR, IoT, brainwave-control, depth-sensing, eye-tracking, etc.) through mostly free SDKs and software (to which the presentation provides some links).
The presentation shows examples of experiences that are literally 'wonder'-ful, and fulfill a real need at the same time (they're not just games), like a school bus to Mars, AI robot for the elderly, and apps that teach our thoughts.
It bundles information that is freely available on the Internet, and mostly not my own work.
SlideShare doesn't run embedded videos, but they run if you downloaded this PDF.
3. 3
Different things I’ve done so far
• Background: Master In Industrial Engineering (Product Designer, 6 years)
• Evolution: Programmer (10), Project Manager (3), UX Designer (3)
• Hobbies: Interactive Sculptures, Installations, Kinetic Art
4. 4
The Red Thread
• I love innovation (including innovating myself)
• I love interaction (not just watching)
• I care about good usability (personal and intuitive)
Source: https://janemumbles.wordpress.com/2010/10/14/%E6%84%9B%E7%84%A1%E9%99%90-endless-love/
5. 5
Example Work (1/3): Skribi
I did extensive research before designing my award-winning game Skribi.
It sells well on the international market for almost 20 years, got a new
spin-off in 2015, and will be released in a new version, in 2017.
6. 6
Example Work (2/3): Reuters Slideshow
I designed and developed a mobile app for Thomson Reuters (together
with a visual designer), which won a ‘Calling All Innovators’ Grand Prize
and was pre-installed on a flagship phone at its worldwide release
(in 75 countries).
Source: http://merlin.rebrovic.net/blog/webgl-cube
7. 7
Example Work (3/3): Orange Apps
I created numerous wireframes, demos & prototypes (often based on my
extensive market research), to continuously test & present designs on real
devices (mobile, tablet & desktop), and to help specify new apps for
development & release, at Orange Design & User Experience.
9. 9
What these experiences have in common
I like (to show you) innovative experiences that behave like empathetic & smart humans
(or personal assistants), who are in an engaging conversation with us.
I like to (inspire you to) use new technologies (or old ones in new ways), to
enhance experiences, making them more:
• Personal – Content is automated, contextual & fluid
• Natural – Interaction is by speech, movement, mood, biometrics, etc.
• Immersive – You’re in-control & in the centre, merging your physical & digital worlds
• Efficient – Helping you do things faster and better
• Entertaining – Making you want more
11. 11
School Bus To Mars - Info
Lockheed Martin's Mars Experience Bus (created with the help of creative agencies
McCann & Framestore) has transparent windows that turn into opaque 4K screens,
to provide a group VR experience on Mars, in which each actual bus motion (bump
or bend) is being translated (in real-time) into the same virtual Mars world motion.
To achieve this, Framestore created 200-square-mile of interactive virtual Martial surface
(based on real street configurations), and a system to translate the bus speed, GPS &
accelerometer into the Unreal game engine, thus creating a real bus ride, inside a video
gaming environment.
Source:
• http://www.adweek.com/adfreak/lockheed-martin-rigged-school-bus-group-vr-take-kids-tour-mars-170968
12. 12
School Bus To Mars - Images
Source:
• http://www.adweek.com/adfreak/lockheed-martin-rigged-school-bus-group-vr-take-kids-tour-mars-170968
13. 13
School Bus To Mars - Video
Source:
• https://www.youtube.com/watch?v=X5JTb_7qv78
15. 15
Professional Domes For ‘Shared VR'
Igloo is a 360º projection company that creates shared VR environments. Stepping inside
their projection domes (for 5 - 500 people) is like stepping inside a giant VR headset
(fully interactive, ceiling-covering, shadow-free environments are in the pipeline).
Sources:
• Igloo - info:: http://www.igloovision.com/ & Igloo - videos: https://vimeo.com/igloo360
16. 16
Low-Cost Domes For ‘Shared VR'
Students built a low-cost cardboard projection dome, and put their instructions online
(see links below).
Sources:
• Cardboard Projection Dome - info: http://www.bealsscience.com/single-post/2016/04/04/Make-your-own-Cardboard-Geodesic-Dome-Planetarium-and-Projector
• Cardboard Projection Dome - video: https://www.youtube.com/watch?v=P9XxVt1llPA
17. 17
3D Modelling Tools & Game Engines
Low-cost tools, e.g.: Autodesk’s 3D Game Dev Software package for indie game
makers offers Maya LT (3D modelling & animation) & Stingray (3D game engine), and
ensures seamless (even live!) transitions between the modelling and interaction
environments, for about £24/m.
Free tools, e.g.: Clara and Blender (3D modelling & animation), and Unity (game engine).
Free web tools, e.g.: A-Frame (open source web framework for building WebVR
experiences).
Sources:
• Maya LT: http://www.autodesk.com/products/maya-lt/subscribe clara: https://clara.io/ Blender: https://www.blender.org/ Unity: https://unity3d.com/
• A-Frame: https://aframe.io/
18. 18
WebVR
WebVR is an experimental JavaScript API that
provides support for VR devices (like the HTC Vive,
Oculus Rift or Google Cardboard) in WebVR-Ready
browsers.
Web developers can reach a huge potential audience
of over 3 billion web users with WebVR, without the
need for native app development or game engine
experience.
Sources:
• WebVR: https://en.wikipedia.org/wiki/WebVR & https://iswebvrready.org/ & https://developer.oculus.com/webvr/
20. 20
Pay-Per-Laugh Theatre - Info
Comedy club Teatreneu installed tablets with facial-recognition technology on the back
of its chairs, to track people's enjoyment, and let them pay €0.30 per smile (until €24 in
total for a ticket). It also launched a mobile app as payment method, and invited people
to share their experience online, which many did, which brought many more visitors.
The system (by advertisement agency The Cyranos McCann) combines happy visuals
with personal (and social) experiences and transparent payment systems, and was so
successful (e.g. raising prices overall by €6) that other theatres copied it around Spain.
.
Sources:
• http://www.bbc.co.uk/news/technology-29551380
• http://www.dailymail.co.uk/sciencetech/article-2788365/the-comedy-club-laughing-costs-money-facial-recognition-tracks-smiles-charges-time-chuckle.html
24. 24
Asus’ Zenbo Robot
ASUS' Zenbo robot walks, talks and controls your home for only $599.
It’s an ‘Amazon Echo on steroids’, and can e.g.:
• Recognise natural speech (“follow me”), gestures (come “here”) and faces
• Move around, photograph, film & control home devices, on-site & remotely
• Learn & adapt to personal preferences with pro-active AI
• Let you shop & interact socially online
• Give control to others (to help you) over its screen, movement, audio & camera
Sources:
• https://zenbo.asus.com/
• http://www.engadget.com/2016/05/30/asus-zenbo-robot/
25. 25
Asus’ Zenbo Robot – What It Does
Sources:
• https://zenbo.asus.com/
• http://www.engadget.com/2016/05/30/asus-zenbo-robot/
29. 29
Microsoft Cognitive Services - Info
Microsoft Cognitive Services let you build apps with powerful algorithms, using just a
few lines of code. They work across devices & platforms like iOS, Android & Windows,
keep improving, are easy to set up, and free to get started. They let you automatically:
• Adjust a game/experience at run-time by detecting a user’s gender, age & expressions
• Describe image content (and give more related content) (e.g. for accessibility/info)
• Alter text-colours on image-backgrounds, for best readability
• Translate on the fly
• Etc.-etc.
Sources:
• https://www.microsoft.com/cognitive-services
33. 33
IFTTT (If This Then That) - Info
IFTTT is a free service that enables you to connect web apps (e.g. Facebook, Evernote,
etc.) with each other or with objects (e.g. fitness trackers, light bulbs or audio systems),
via Recipes (conditional statements), which can be turned on & off, like:
This: If it begins to rain (Ingredient “Rain” in channel "Weather")
Then…
That: change the Philips hue light colours to blue (channel “Philips hue")
There are 77+ Channels that can trigger actions when ingredients (conditions) are found.
Sources:
• https://ifttt.com/wtf
• https://www.meethue.com/en-GB
37. 37
Microsoft's AI Game - Info
Source:
• Info: http://en.wikipedia.org/wiki/Project_Milo
• Video: http://www.youtube.com/watch?v=CPIbGnBQcJY
Meeting Milo is a never-released Kinect game for the Xbox 360 video game console,
in which players interact with a 10-year-old child (Milo or Millie, selected at the start)
and a dog named Kate, playing through a story. It’s very old (2009), but still inspiring.
The game’s artificial intelligence (AI) recognises human emotion and responds to
interactions such as spoken words, gestures, or predefined actions in dynamic situations.
The player’s reality merges with Milo’s virtual game environment, for a seamless
augmented reality game experience, through a connected TV, camera & console.
38. 38
Microsoft's AI Game - Images
Source:
• Info: http://en.wikipedia.org/wiki/Project_Milo
• Video: http://www.youtube.com/watch?v=CPIbGnBQcJY
39. 39
Microsoft's AI Game - Video
Source:
• Video: http://www.youtube.com/watch?v=CPIbGnBQcJY
40. 40
Disney’s Kinectimals
Source:
• https://www.youtube.com/watch?v=Zs2x3jFTb2E
• https://en.wikipedia.org/wiki/Kinectimals
• http://marketplace.xbox.com/en-GB/Product/Kinectimals/66acd000-77fe-1000-9115-d8024d5308b3
Players stay off-screen themselves,
while they interact with a virtual animal
on-screen (after choosing 1 of many
animals to feed, teach tricks, guide on
obstacle courses, or just play with), in
this Xbox Kinect game.
41. 41
Sony’s EyePet
Source:
• Info: https://www.playstation.com/en-gb/games/eyepet-ps3/
• Info: http://www.amazon.com/EyePet-Playstation-3/dp/B002OOVJUA
• Video: http://www.youtube.com/watch?v=YZvxIjdyyII
This PlayStation game merges your reality with virtual elements on the connected TV
(via a camera & console), to provide a seamless augmented reality experience.
• Magic Cards bring EyePet to life, and trigger specific actions (e.g. to shower him).
• EyePet reacts to your movements & objects (e.g. roll him a ball and he’ll chase it).
• Customize his fur length, colour and pattern and choose from over 250 unique outfits
• Draw toys and watch as they transform into in-game 3D toys
• Record your favourite EyePet moments and creations, and then share them online
42. 42
Sony’s EyePet – Images
Source: http://www.youtube.com/watch?v=YZvxIjdyyI I
44. 44
Crayola’s Physical-Digital / 2D-3D Fun - Info
Sources:
• https://3dprint.com/35952/crayola-color-alive/
• http://www.3ders.org/articles/20150109-crayolas-color-alive-bring-kids-coloring-to-life-with-augmented-reality-3d-printing.html
• https://www.youtube.com/watch?v=d-kFECnsAPA
• https://play.google.com/store/apps/details?id=com.crayolallc.carcreator_android
• https://play.google.com/store/apps/details?id=com.DAQRI.crayola.coloralive
• Review: http://toybuzz.co.uk/crayola-colour-alive-review
• Review: http://www.getcreativejuice.com/2015/03/bring-coloring-life-crayola-color-alive.html
Crayola’s freemium apps lets you print free 2D colouring-in pages, with designs that you
scan to bring to life as virtual 3D objects (each with unique effects), which you can then
view & modify in 360 degrees; take selfies with (bringing them into your own reality);
save, share & swap with friends (the social aspect); use as interactive 3D game objects
(in many virtual realities & diverse games); and also 3D print (coming soon). You can
enhance the experience by buying special crayons for unique virtual effects (animated fire,
pixie dust, etc.), and books for more designs (meaning more profit for Crayola).
46. 46
Moleskin’s Physical-to-Digital Systems - Info
Moleskine's Livescribe Notebook and Smart Writing Set bridge analogue & digital.
They automatically transfer into digital, anything that you like to write, draw & say, so you
can playback audio (recoded by the pen); share content (1 click away); recognise, edit,
format, tag, organise & search text notes (that were hand-written); colour & change your
drawings & text (for creative use); etc.
The Smart Writing Set pen has an infrared camera that recognises the paper notepad,
so it knows which pad, page, or ‘paper button' (record, email, tag, etc.) it is on.
Sources:
• Livescribe Info: http://www.moleskine.com/gb/news/livescribe-notebook
• Livescribe Video: https://www.youtube.com/watch?v=hL5V_OfYfA4
• Smart Writing Set Info: http://www.moleskine.com/microsites/smartwritingset
• Smart Writing Set Videos: https://www.youtube.com/watch?v=e-bdmV43roQ & https://www.youtube.com/watch?v=JgbVvqhZ1v4
47. 47
Moleskin’s Physical-to-Digital Systems - Images
Sources:
• Livescribe Info: http://www.moleskine.com/gb/news/livescribe-notebook
• Livescribe Video: https://www.youtube.com/watch?v=hL5V_OfYfA4
• Smart Writing Set Info: http://www.moleskine.com/microsites/smartwritingset
• Smart Writing Set Videos: https://www.youtube.com/watch?v=e-bdmV43roQ & https://www.youtube.com/watch?v=JgbVvqhZ1v4
49. 49
Depth-Sensing Devices
Mobiles, tablets, laptops & computers are being released with built-in depth sensors,
to capture (3D-scan), customise, animate, gamify, share, sell and 3D-print all kind of
3D objects (including people & animals) and 3D environments.
They can be used for e.g. architecture, AR-gaming, and indoor navigation
(without beacons, GPS, external cameras, etc.), and can help making it ‘mainstream’
to 3D scan & print.
50. 50
Intel’s RealSense 3D camera – For Computers
Sources:
• Article: http://www.bbc.co.uk/news/technology-25632699
• Video: https://www.youtube.com/watch?v=DuUVIat40YU
• Developer Site: https://software.intel.com/realsense
• Overview: http://www.intel.com/content/www/us/en/architecture-and-technology/realsense-overview.html
• Get the Developer Camera: http://click.intel.com/creative-interactive-gesture-camera-developer-kit.html
• Article: http://www.bbc.co.uk/news/technology-25632699
Intel RealSense Technology enables natural, intuitive user interfaces (using gestures,
hand & finger tracking, speech recognition, facial analysis, and augmented reality).
It allows you to separate foreground objects from the background (e.g. to super-impose
video footage on virtual backgrounds), and to scan, modify, print & share in 3D.
New laptops, tablets & computers with Intel's RealSense camera are being released
(e.g. from Acer, Asus, Dell, Fujitsu, HP, Lenovo & NEC). I have one, and like it!
51. 51
Intel’s RealSense 3D camera - Images
Sources:
• Article: http://www.bbc.co.uk/news/technology-25632699
• Video: https://www.youtube.com/watch?v=DuUVIat40YU
• Developer Site: https://software.intel.com/realsense
• Overview: http://www.intel.com/content/www/us/en/architecture-and-technology/realsense-overview.html
• Get the Developer Camera: http://click.intel.com/creative-interactive-gesture-camera-developer-kit.html
• Article: http://www.bbc.co.uk/news/technology-25632699
52. 52
Intel’s RealSense 3D camera - Video
Sources:
• Video: https://www.youtube.com/watch?v=DuUVIat40YU
53. 53
Google Tango - For Mobiles
Sources:
• http://www.techradar.com/news/phone-and-communications/mobile-phones/project-tango-everything-you-need-to-know-1322923
• http://shop.lenovo.com/us/en/tango/
Google's Project Tango enables e.g. depth-sensing phones, like the Lenovo Phab 2 Pro
that’s already on the market.
Tango uses 3 technologies:
1. Motion-tracking: using camera, accelerometer and gyroscope data
2. Depth perception: tracking how far away surfaces are (to define e.g. table, floor, etc.)
3. Area learning: it takes 'learned' information and enhance it with e.g. points of interest or
notes in a location.
55. 55
Magic Leap - AR Headset
Sources:
• Magic Leap: http://www.wired.com/2016/04/magic-leap-vr/ & https://en.wikipedia.org/wiki/Magic_Leap
• See also Microsoft’s Hololens: https://www.microsoft.com/microsoft-hololens/en-gb
Magic Leap's head-mounted virtual retinal display super-imposes computer-generated
3D imagery over real world objects, by projecting a digital light field directly into the eye.
Footage filmed through a device demonstrates 3D spatial mapping with obstruction
technology, apparent reflections of virtual objects interacting with real objects, and
interaction through eye tracking.
The start-up's funding is estimated at $4.5 billion (from e.g. Google & Qualcomm), and
promises to become a big player in the AR & VR industry.
58. 58
Neural brain dust – Brain-Machine-Interface (BMI) - Info
University of California Berkeley researchers proposed implanting people's brains with
thousands of tiny sensors they called "neural dust“, for little circuits that gather detailed
data on brain functions. Eventually the electronic swarms may prove useful for controlling
devices via thought as brain-machine-interfaces (BMI), or for stimulating malfunctioning
brain regions to restore limb motor control for paralys.ed patients.
Source:
• http://www.insidebayarea.com/news/ci_24769683/computerizing-people-may-be-next-step-tech
• http://techland.time.com/2013/07/17/your-future-brain-machine-implant-ultrasonic-neural-dust/
59. 59
MyndPlay’s Mind-Controlled Services - Info
MyndPlay is a media player & software platform that lets you control services (40+ now, )
with your brain waves, via their head-sensors (and apps built using their free SDK).
Specific brainwaves trigger e.g. specific film segments to play automatically (seamless),
which means there is no interruption of your experience, to decide and push buttons.
Apps can prompt viewers to focus (or relax) to change the game (or can sense emotions
like anger, fear, sadness, happiness), giving them a sense of achievement & excitement,
the feeling of being part of the action (no outsider) and a different experience each time.
Sources:
• http://myndplay.com/ & https://www.youtube.com/watch?v=al7lswMe9t4
• http://www.dailystar.co.uk/tech/news/377618/New-MyndPlay-technology-lets-you-control-games-and-film-content-with-your-BRAIN
• http://home.bt.com/tech-gadgets/future-tech/control-a-movie-with-your-mind-video-11363898559184
61. 61
Emteq’s Facial Sensing Platform - Info
Emteq’s faceteq is a facial sensing platform that merges AI with facial sensing wearables
to deliver VR that responds to emotions. It’s for face-tracking without cameras.
Their wearable senses muscle activity, eye movement, heart rate, head position, etc.,
which their AI engine translates in facial expression & emotional state in almost real-time.
This allows you to understand your users: where they looked, where they went, what
they did, and how they responded, and lets you e.g. mirror their expressions onto their
avatars (which is important for social interactions in shared VR).
Source:
• http://emteq.net/ & https://www.vrfocus.com/2016/09/emteq-reveals-vr-facial-tracking-tech-that-doesnt-require-cameras/
63. 63
Sensum’s Emotional Research - Info
Sensum Insights is a (web-based and mobile-enabled) platform that allows you to
capture non-conscious data from wearable devices & biometric sensors, and combine
this with data from traditional market research methods, to understand your users better.
You can use brain waves, facial coding, heart rate, skin sweat, body posture, and other
non-conscious physiological responses for emotional research (depending on projects).
When you can appeal to people's emotions, you have a better chance to
sell your product, create great experiences, and shape brand loyalty.
Source:
• https://sensum.co/insights
• https://sensum.co/blog/2016/02/09/football-legend-ian-wright-works-with-sensum-to-rate-the-success-of-4k-tvs
• https://www.marketingweek.com/2016/10/12/why-emotional-reactions-are-a-key-driver-of-behaviour/
66. 66
Gesture & Motion Controlling Myo Armband - Info
Thalmic Labs' Myo is a wearable gesture and motion control device that lets you take
control of your phone, computer, AR / VR experience, etc., touch-free and without
external cameras or sensors.
SDKs are available for
Windows, Mac, iOS, and Android
This sounds promising and I can see
how future versions could enable
great interactions in VR.
Sources:
• https://www.myo.com/
68. 68
Authorising & Gesture-Controlling Nymi Bracelet - Info
Bionym's Nymi validates identity by cardiac rhythm (electrocardiogram or ECG), and can:
• Communicate your identity to a device (e.g. smartphone, vehicle, electronic lock, or
even a smart environment) via secure Bluetooth or NFC, removing the need for
repeated prompts for e.g. fingerprint scanning or PIN input
• Sense its distance to that device
• Recognise gestures for task-based gesture control, by its accelerometer & gyroscope
Using identity, motion control & proximity detection together opens up a new dynamic,
interactive world, revolutionising banking, gaming, phones, social media, exercise, etc.
Sources:
• Site: https://nymi.com/ & Review: http://reviews.cnet.com/computer-systems/bionym-nymi/4505-3000_7-35835279.html
71. 71
Authorising Kerv Ring - Info
Kerv is the world's first contactless payment ring, that also lets you tap-in on transport (in
London), open NFC locks, and share info with NFC-enabled phones through a single
tap (opening your online public profile page for e.g. contact, emergency & medical data).
You can switch it on or off, transfer money to it, see & do transactions, etc., when
logged into your Kerv account (a kind of e-wallet), on the secure Kerv site & mobile app.
It works with no charging or pairing, is waterproof, lightweight, hypo-allergenic, scratch-
resistant, discreet & pretty, and very affordable (about £50)
Sources:
• http://kerv.com/ & https://www.kickstarter.com/projects/philipcampbell/kerv-the-worlds-first-contactless-payment-ring
74. 74
Eye tracking as a natural & useful third input - Info
The MSI GT72S G laptop is the first to integrate Tobii's eye tracking technology, in its
hinge, with 3 infrared lights (invisible to the human eye) to see where you are & where
you're looking, to e.g. automatically bring screen-brightness back up when your attention
returns to the display, identify you & your interests, act as an extra input method, etc.
Eye tracking adds a natural and useful third input for everyday computing and gaming.
Setup is simple (within seconds): first focus on a few dots until erasing them in a small
calibration demo, then blow up space rocks in an asteroid field with your laser eyes.
Source:
• http://www.techradar.com/reviews/pc-mac/laptops-portable-pcs/laptops-and-netbooks/msi-gt72s-g-tobii-1312776/review
75. 75
Eye tracking as a natural & useful third input - Images
Source:
• http://www.techradar.com/reviews/pc-mac/laptops-portable-pcs/laptops-and-netbooks/msi-gt72s-g-tobii-1312776/review
79. Thank You
For more information about this presentation
please contact: Carin Campanario
bc.campanario@googlemail.com
More information about me:
http://ccampanario.com
https://www.linkedin.com/in/carincampanario