1. Using multi-touch and sensors
in Java
Felipe Pedroso

2. Agenda
• The future of PCs
• JavaFX and Touch Support
• Using JNI to work with sensors
2

3. Mobile Computing Lines are Blurring
Smartphone
Consumption Usages
Tablet
Ultrabook™
Convertible & Detachable
Ultrabook™
Creation/Productivity
Notebook

4. Reshaping the PC Experience
Multi-Touch
Accelerometer
Ambient
Light Sensor
Ultrabook™
Compass
GPS
Gyroscope
Near Field
Communication
Context Aware
Sensors
4

5. OK, they have all those
features…
… but how can we implement them using Java?

6. To allow users to TOUCH my Java App...
• Should I use...
• AWT?
• Swing?
• SWT?
• Actually, you can use them but...
• They aren’t made for multi-touch (OK, there’s a way to do it)
• You must optimize your UI controls to be more ‘touch friendly’
• So, how to do it in a simple and easy way?
6

7. 7

8. Actions supported
• Touch events: Down, Move and Up
• Tap (Down and Up) / Double tap
• Drag and Drop (Down, Move and Up)
• Hold
• Gestures
• Swipe: Up, Down, Left and Right
• Zoom: Pinch / Spread
• Rotate
8

9. What I need to do to manipulate components?
• Your components must extend the Node class or any of its
subclasses (StackPane, ImageView, etc)
• Set the proper EventHandler to handle the action.
9

10. What I need to do to manipulate components?
• Apply a transform to the component (Translation, Rotation and
Scale):
• Let’s dive into some code!
10

11. What about the sensors?
JNI, I choose you!

12. Available sensors on Ultrabooks
• Accelerometer
• Proper acceleration in three axis (x, y, z)
• Gyrometer
• Device orientation
• Magnetometer
• Strenght and direction of magnetic fields
• GPS (Global Positioning System)
• Location and Time information
• NFC
• Near Field Communication
• Ambient Light Sensor
• Ambient Light Level
12

13. API Windows – Sensor Fusion
13

14. API Windows – Namespaces
• Windows.Sensors.*  Common sensors
•
•
•
•
•
•
•
Accelerometer
Gyrometer
Inclinometer
OrientationSensor
SimpleOrientationSensor
Compass
LightSensor
• Windows.Devices.Geolocation
 GPS
• Geolocator
• Windows.Networking.Proximity  NFC
• ProximityDevice
14

15. Windows API – How to
• Get the default object of your sensor using the GetDefault method
• You can call the GetCurrentReading() to get the current value of the
sensors or...
• ... work with the ReadingChanged event
• Set the attribute ReportInterval (please, respect the MinimumReportInterval to avoid problems)
• Delegate a method to handle the event (something like setting a method as a listener)
• Handle the event!
• This procedure is valid only for sensors from Windows.Sensors
namespace
15

16. How can I access that?
16

17. Using JNI to access the sensors from
Windows.Sensors
1. [Java] Create a native method to register the object that will
handle the events that come from the sensor
2. Generate the header file using javah
3. [C++] Create a DLL Project in Visual Studio with the DLLs
and namespaces of the Windows 8 APIS
4. [C++] Use a variable to keep a reference to the object and
the ID (GetMethodID) of the method that will handle the
event
17

18. Using JNI to access the sensors from
Windows.Sensors
5. [C++] Initialize the sensor and delegate a C++ method to
handle the event
6. [C++] Redirect the delegate method call to the Java method
using the function CallVoidMethod with the following
parameters: the Java Object, the method ID and it’s
parameters
7. [Java] Handle the event! 
A lot of text? Please, show me the code!
18

19. What about Linux?
• Touch works fine, but there aren’t clear APIs to read
sensors. If you know how to do it, let’s work together!
19

20. Wants to know more about Intel Software?
http://software.intel.com/pt-br/
20

21. Windows Community
http://software.intel.com/en-us/windows
21

22. www.javafxcommunity.com
22

23. JFXtras.org
23

24. Take aways
• The PC experience is being reshaped
• JavaFX + Touch = AWESOME
• Using JNI to access the sensors can be LEGENDARY!

25. Any doubts?
(if yes, the answer is 42)

26. Thanks a lot!
Felipe Pedroso
facebook.com/felipe.a.pedroso
@felipeapedroso