David Silverlight's powerpoint presentation on the Kinect for Windows SDK. Feb. 29, 2012
NUI = Natural User Interface: it's an invisible interface, the content is the interface, removing the proxy, direct manipulation, gestural interfaces
Kinect for Windows SDK:
1. Kinect explorer
2. Installing & using the Kinect sensor
3. Setting up your dev environment
4. Skeletal tracking fundamentals
5. Working with depth data
6. Audio fundamentals
7. Camera fundamentals
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AI
Building Applications with the Microsoft Kinect SDK
1. KINECT FOR WINDOWS
It’s All NUI to me:
Building Apps with the Microsoft Kinect SDK
David Silverlight
Founder and Chief Software Architect
TheSilverlightGroup.com
IsYourWebMasterDead.com
2. KINECT FOR WINDOWS
Agenda
• NUI – What?
• Demo’s – See what all the hullabaloo is about
• Getting Started/Kinect Explorer/QuickStarts
• Building Your Own Kinect apps.
• (Video, Depth, Skeleton, Audio)
• Resources
6. KINECT FOR WINDOWS
What is NUI?
• Natural User Interface
• Common term used by designers and
developers of computer interfaces to refer to a
user interface that is effectively invisible, or
becomes invisible with successive learned
interactions, to its users
7. KINECT FOR WINDOWS
What is NUI?
• An Invisible Interface.
• The Content is the Interface.
• Removing the Proxy
• Direct Manipulation
• Gestural interfaces
8. KINECT FOR WINDOWS
Seeing is believing – Kinect Development
Allows you to capture
your movements and
opens up a whole new
world to animators on
a tight budget.
12 Best Kinect Hacks Personal Data Mirror
Catch up with the
latest headlines,
weather and health
all whilst brushing
your teeth
Real Time Motion
Capture with XNA
Sample showing just
what people are
doing with the
Kinect
9. KINECT FOR WINDOWS
Fitnect Virtual Vacation
Now another reason
to never leave the
house.
Interactive Fitting
room showing how
Kinect and business
can connect.
Seeing is believing – Kinect Development
12. KINECT FOR WINDOWS
Getting Started: Building Your Own Apps
• Designed to get you started
• Assumes you have some programming experience
• Managed API - Concepts work in both VB/C#
14. KINECT FOR WINDOWS
System Requirements
• You must run applications that are built by using the Kinect for Windows
SDK in a native Windows environment. You cannot run applications in a
virtual machine, because the Microsoft Kinect drivers and this SDK must be
installed on the computer that is running the application.
• Supported Operating Systems and Architectures
• Windows 7 (x86 or x64)
• Hardware Requirements
• Computer with a dual-core, 2.66-GHz or faster processor
• Windows 7–compatible graphics card that supports Microsoft® DirectX® 9.0c
capabilities
• 2 GB of RAM
• Kinect for Xbox 360® sensor—retail edition, which includes special USB/power cabling
15. KINECT FOR WINDOWS
System Requirements
• Software Requirements
• Microsoft Visual Studio® 2010 Express or other Visual Studio 2010 edition
• Microsoft .NET Framework 4.0 (installed with Visual Studio 2010)
• For C++ SkeletalViewer samples:
• DirectX Software Development Kit, June 2010 or later version
• DirectX End-User Runtime Web Installer
• For Speech sample (x86 only):
• Microsoft Speech Platform - Server Runtime, version 10.2 (x86 edition)
• Microsoft Speech Platform - Software Development Kit, version 10.2 (x86
edition)
• Kinect for Windows Runtime Language Pack, version 0.9
(acoustic model from Microsoft Speech Platform for the Kinect for Windows
SDK Beta)
16. KINECT FOR WINDOWS
It all starts here: http://www.microsoft.com/en-us/kinectforwindows/
Developing your own Kinect aps
17. KINECT FOR WINDOWS
Kinect for Windows SDK Quickstarts
• http://channel9.msdn.com/Series/KinectQuickstart - All are
great, IMHO
• The Kinect Explorer is new from the Beta
• Kinect for Windows SDK Browser.
• Installing and Using the Kinect Sensor
• Setting up your development environment
• Skeletal Tracking fundamentals
• Camera Fundamentals
• Working with Depth Data
• Audio Fundamentals
19. KINECT FOR WINDOWS
Setting up your Development Environment
Add our references to:
- The Microsoft.Samples.Kinect.WpfViewers Project
- The Coding4Fun.Wpf.dll
Reference the assembly:
• using Microsoft.Kinect;
20. KINECT FOR WINDOWS
KINECT API BASICS
• Manage Kinect state
• Connected
• Enable Color, Depth, Skeleton
• Start Kinect
• Get Data
• Events - AllFramesReady
• Polling – OpenNextFrame
24. KINECT FOR WINDOWS
Setting the Camera Tilt Angle
• Warning: The tilt mechanism in the sensor array is not rated for
frequent use. Your code should not make calls to tilt the device more
than 15 times in any two-minute window. Changing the tilt too often
results in an error message from the function.
• To adjust the tilt or pitch, set the ElevationAngle property to a value
between –27 (Camera.ElevationMinimum) and +27
(Camera.ElevationMaximum). The code below sets the camera to the
maximum elevation:
• kinectSensorChooser.Kinect.ElevationAngle =
Convert.ToInt16(slider1.Value);
28. KINECT FOR WINDOWS
CAMERA DATA
• Events return ImageFrame
• PixelDataLength
• FrameNumber
• Timestamp
• Dimensions: Height, Width
• Use AllFramesReady event to synchronize
31. KINECT FOR WINDOWS
DEPTH DATA
• Returns the distance and player for every pixel
• Ex: 320x240 = 76,800 pixels
• Distance
• Distance in mm from Kinect ex: 2,000mm (6.56 feet)
• Player
• 1-6 players
32. KINECT FOR WINDOWS
MODE
Mode Depth & Player Center Hip Joint Other 19 Joints
Default Yes Yes Yes
Near Yes Yes No, for v1.0
34. KINECT FOR WINDOWS
FORMULAS
• Distance Formula
int depth = depthPoint >> DepthImageFrame.PlayerIndexBitmaskWidth;
• Player Formula
int player = depthPoint & DepthImageFrame.PlayerIndexBitmask;
37. KINECT FOR WINDOWS
Joints
• Maximum two players tracked at once
• Six player proposals
• Each player with set of <x, y, z> joints in meters
• Each joint has associated state
• Tracked, Not tracked, or Inferred
• Inferred - Occluded, clipped, or low confidence joints
• Not Tracked - Rare, but your code must check for this state
42. KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor
• Setting up your Development Environment
• Camera Fundamentals
• Working with Depth Data
• Skeletal Tracking Fundamentals
• Audio Fundamentals
43. KINECT FOR WINDOWS
SOUND POSITION
• Sound Source Angle – the angle and confidence level of where
audio is coming from
• Beam Angle – The angle used to record audio that you can set as a
“directional microphone”
-50 0 +50
45. KINECT FOR WINDOWS
SPEECH RECOGNITION
• Grammar – What we are listening for
• Code – GrammarBuilder, Choices
• Note: Ensure AutomaticGainControl = false
47. KINECT FOR WINDOWS
MULTI-MODAL FEEDBACK
• Engagement model
– Mic indicator for speech-enabled menus
– Keyword to engage
• Feedback and confirmation
56. KINECT FOR WINDOWS
What we have learned
• Examples of the amazing things that can be
done with the Kinect
• How amazing the Kinect Device is, equivalent to
$200,000 in capabilities
• How to create our own Kinect applications,
Video, Skeletal and Audio tracking.
• The future of Kinect