This document summarizes a proof-of-concept project to add gesture control capabilities to ParaViewWeb using a Microsoft Kinect sensor. It introduces the components used, including the Kinect sensor, Kinect driver, Kinect SDK, a C# adapter, ParaViewWeb, and JavaScript libraries. The C# adapter converts Kinect skeleton and image data to JSON that is sent to the browser and interpreted to trigger mouse events for navigation. While a proof of concept, it has benefits of remote access and hand interaction but also limitations like single hand control and performance issues. A demonstration of it working with ParaViewWeb is provided.

1. ParaViewWeb + Kinect
Towards accessible immersive
CFD results visualisation

2. Introduction
• Objective of the presentation:
• Provide a synopsis of the methodology used when
developing proof-of-concept ParaViewWeb +
Kinect.
The aim of the ParaViewWeb+Kinect project is to evaluate feasibility
and demonstrate proof-of-concept of gesture based navigation for
CFD simulation visualisation, not to provide a final product.

3. Component Chart
Kinect Sensor
Kinect Driver
Kinect SDK
Adapter (C#)
pvpython +
pv_web_visualizer.py
Geometry + CFD results files
Visualizer/index.html
index.html
+ interpreter library (JavaScript)
WebSocket
WebSocket
MouseEvent

4. Components Details
• C# Adapter:
SkeletonFrame DepthImageFrame
InteractionFrame
UserInfo
Broadcast as JSON

5. Component Details
• HTML/JavaScript
Receive JSON
Create custom HandEvents:
HandMove, HandGrip, HandRelease
Display hand
"cursors"
Convert HandEvents
into MouseEvents
(move, over, enter,
leave, down, up,
click)

6. Implementation
Pro
• Allow kinect movement with no
modification of targeted web-
page;
• Allow for kinect to be hosted
on different computer/OS than
client;
• Support hand interaction and
skeleton movement;
• Small and simple.
Cons
• No recognition of hand
gesture;
• One hand is dominant when
mapping hand movement to
mouse movement;
• Slow;
• Not 100% web-based, and
requires distribution the local
C# adapter;
• Based on kinect SDK, need
porting to openNI for non-
kinect sensors.

7. Demo
• The demo is based on ParaViewWeb, but
we trying to make it work other web-based
services
ParaViewWeb + Kinect demo:
http://youtu.be/XHXqmQPqQNs

8. The End
This project is standing on the shoulders of giants such Microsoft & Kitware...
Special thanks to
The Edge
and
Hack-the-Evening
Contact details:
julien.decharentenay@gmail.com
Twitter: @FormerlyHibou
https://juliendecharentenay.appspot.com