ITCamp 2012 - Tim Huckaby - Keynote

The Future of Software Now:
NUI - A New Genre of User
Experience
Tim Huckaby
Microsoft MVP & RD
InterKnowlogy Chairman/Founder
Actus Interactive Software CEO

@ itcampro # itcamp12 Premium conference on Microsoft technologies

ITCamp 2012 sponsors Mobile &
Development


NUI: The Natural User Interface
Multi-Touch • Common Gestures of Touch
• Physical Object Interaction
Capable • Virtual Object Interaction

Gesture • Motion Based Interface
• Tracking of full-body movement, facial expression, and
Capable voice with precision

• Multi-array Microphone
Voice Capable • Uniquely Distinguish Audio & Voices

Neural • Conscious and non-conscience interfaces

Capable • Non Invasive


Tim Huckaby
Founder / Chairman, InterKnowlogy
Founder / CEO, Actus Software
Microsoft RD & MVP


A little bit about me and why I am here…


About Tim, InterKnowlogy & Actus…
InterKnowlogy (www.InterKnowlogy.com)
• Tim Huckaby, Founder/Chairman (TimHuck@InterKnowlogy.com)
– 13+ year old services firm – Custom Application Development
– focused on building NUI solutions
– Web, mobile, and desktop
– windows 7/8, Surface, Windows phone 7, azure, Kinect,
•Largest Client: Microsoft

Actus Software (www.Actus-Software.com)
• Tim Huckaby, Founder/CEO (TimHuck@Actus-Software.com)
•Software Product Company
– Interactive Kiosk and Active Digital Signage
– Multi-Touch, Gesture, and Voice Driven Solutions


We have come a long way…

The “Green-Screen”


Image courtesy of Computer History Museum


Touch Capable Computing
Devices Are Not New

The Samsung SUR40 for
Microsoft Surface

1972: PLATO IV Touch Windows 8 - Metro
Screen Terminal

Apple iPad

Past Present Future (Now)


What is Relatively New is:
 Inexpensive Touch Capable, Non
Proprietary Hardware
 A Windows OS designed for Multi-
Touch
 A Consistent Multi-Touch API for .NET
across all Windows Hardware/Software
Platforms
 Developer Productivity in Touch
Capable Apps
 The Forecast of Multi-Touch Hardware
Capability in Every New Computing
Device in the Short Term


About the Windows 7 Computer I am
Using Today…
• HP TouchSmart TM2
• Rates in Windows as 5.0
• Price in October-2010: Starting from: $1499.99*
• Price in March-2011: Starting from: $799.99*
• Price Today (HP TouchSmart 320m ): $579.99


HTML 5 Touch Events Spec, Version 2
W3C Editor's Draft 14, November 2011

 Defines a set of low-level events that represent
one or more points of contact with a touch-
sensitive surface
 Changes of those points with respect to the
surface and any DOM elements displayed upon
it (e.g. for touch screens) or associated with it
(e.g. for drawing tablets without displays).
 Addresses pen-tablet devices, such as drawing
tablets, with consideration toward stylus
capabilities.

@ itcampro # itcamp12
Source: https://dvcs.w3.org/hg/webevents/raw-file/tip/touchevents.html conference on Microsoft technologies
Premium

HTML 5 Touch Examples

Tim Huckaby
Microsoft RD & MVP


What is also Relatively New:
Multi-Touch User Interfaces Combined with Object
recognition of Physical Objects

 The Microsoft Surface is expensive
because it has 5 cameras looking up
that do Object Recognition
 Based on a “byte tag” (not a QR
code)
 Object Recognition in Windows
 The Use Cases are limitless…


Surface Gaming - Craps

Tim Huckaby
Microsoft RD & MVP


Windows 8 - Designed for Multi-Touch
Metro - Perfect for Gesture


Windows 8:
• Metro mode / Desktop Mode
• Actus Interactive Kiosk – Metro
• Semantic Zoom

Tim Huckaby
Microsoft RD & MVP


NUI: Games as a Tool for Learning
• Learning made Fun / Entertaining
• Proven as a learning tool beyond just
Children
• Computer based Games are a significant part
of popular culture
• Educational Video Game Curriculums
– MIT, UC Irvine, Minnesota, Mich State, etc.


NASA: Mars Bound

Tim Huckaby
Microsoft RD & MVP


What is Kinect?
World's Fastest-Selling Consumer Electronics Device
… a Future Beyond Gaming

Combination of RGB camera, depth sensor and multi-array microphone
RGB camera delivers three basic color components
Depth sensors “sees” in 3-D
Microphone locates voices by sound and excludes ambient noise

The sensor provides eyes and ears - but it needs a brain
Raw data from the Kinect is just “a whole bunch of noise that someone needs to
take and turn into signal”
Our software finds the signal

Software makes all the magic possible
Skeletal Tracking
Face, Gesture Recognition
Audio Echo cancellation
Audio Beam Forming
Speech Recognition
Machine Learning

Kinect for Windows

Combination of RGB camera, depth sensor and multi-array microphone
RGB camera delivers three basic color components
Depth sensors “sees” in 3-D
Microphone locates voices by sound and excludes ambient noise
Consumer Priced: $249.99
Tracks full-body movement, facial expression, and voice with precision
Kinect sensor bar scans and monitors 1 million points on the body
Kinect for Windows SDK
Skeletal Tracking
Face, Gesture Recognition
Audio Echo cancellation
Audio Beam Forming
Speech Recognition
Machine Learning


XBOX Kinect

The Good News:
• The Kinect SDK for Windows
The Not So Good News:
• Windows Only Device; K4W does not work on your
Xbox; you cannot build apps for 18 million Xbox Kinect
owners with it.


The XBOX Kinect for Windows
Touch-less Operating Room

Tim Huckaby
Microsoft RD & MVP


Neural Interfaces


Conscious and Non-Conscious Interfaces

• High fidelity EG acquisition device
• No conductive gel, etc.
• Non-invasive
• Use Case Examples:
• Conscious: mind manipulation of objects
• Non-Conscious: Ranking music
• First target market: gaming
• $300!!!


Emotiv Software Development Kit (SDK)

 Direct API access to all Emotiv detection suite inputs and controls,
including training, profile management and gyro outputs
 Program in C++ and compatible languages, .NET (VB.NET, C#), Java
 Developer plug-in available for Unity3D
 Active developer community (exclusive Developer user forum)

 C++ Header files and Import Libraries

…which means…

 Wrap the SDK in managed C++ thereby exposing a the API to a C#
application
 That is very powerful…


We are within a decade of building the
Holodeck
“The Ultimate NUI Application”


Summary
• We are at the cusp of a revolution in software & hardware
• Hardware and software platforms will ultimately synchronize
• Using Natural User Interface (NUI) technologies improve the
User Experience
• The hardware, tools, plumbing and platforms for NUI
computing are powerful and available now.
– …and will just continue to get better, cheaper, powerful and more
available
• Within a realistic time frame NUI computing will be the norm
• The concepts of NUI change the way we design and build
software


HoloDesk - Direct 3D Interactions with a Situated See-Through Display

Tim Huckaby
Microsoft RD & MVP


Tim Huckaby
• More info on InterKnowlogy: • Contact me: Tim Huckaby
www.InterKnowlogy.com E-mail: TimHuck@InterKnowlogy.com
Silverlight.InterKnowlogy.com TimH@Actus-Software.com
www.vimeo.com/InterKnowlogy
• Phone:
• More info on Actus Software: Office: 760-444-8640
www.Actus-Software.com Mobile: 619 990 9200
• About Tim Huckaby… Blog:
http://team.interknowlogy.com/blogs/timhuckaby
– Founder/Chairman, InterKnowlogy
– Founder/Chairman, Actus Software
FaceBook:
– Microsoft® Regional Director – Southern http://www.facebook.com/Tim.Huckaby.Carlsbad
California LinkedIn:
– Microsoft® .NET Partner Advisory Council http://www.linkedin.com/in/timhuckaby
Founder / Member
– Microsoft® MVP - .NET Twitter: @TimHuckaby
– INETA Speaker – International .NET Users
Group Association
– Board Member: Association for Competitive
Technology
– Board Member: Dev Connections
– Author / Speaker


Agenda Mobile &
Development


Mobile &
Development


Q&A


ITCamp 2012 - Tim Huckaby - Keynote

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (19)

Destacado

Destacado (18)

Similar a ITCamp 2012 - Tim Huckaby - Keynote

Similar a ITCamp 2012 - Tim Huckaby - Keynote (20)

Más de ITCamp

Más de ITCamp (20)

Último

Último (20)

ITCamp 2012 - Tim Huckaby - Keynote