This document summarizes a student project that integrated the Myo armband with Google Cardboard to create an immersive virtual reality experience for learning Japanese characters. The project objectives were to develop a Google Cardboard app integrated with the Myo armband to enable user control of a 3D environment using gestures. The project scope involved using 5 gestures without positional tracking to display Japanese characters (hiragana) drawn in the air. Accomplishments included creating a 3D classroom, integrating the Myo, and adding sound effects to identify correct character strokes. Future work could improve the plugin response time and add more gestures.

1. VIRTUAL INTERACTION
USING MYO AND
GOOGLE CARDBOARD
Name : ChongWee Nee
ID : 1122702155
Supervisor : Dr Poo Kuan Hoong
Co-Supervisor : IanTan
Moderator : DrWongYa Ping

2. Project Objectives
• To develop a Google Cardboard application and integrate the
Myo Armband into the 3D environment.
• To enable user to control the 3D environment using the Myo
Armband.
• To provide an immersive experience of learning Japanese
characters

3. Project Scope
• Use only 5 gestures
• Positional tracking not included
• Capture hand movement and transform the data to display air
drawn characters in the virtual world.
• The Japanese character will be limited to a few (Hiragana).
• The application will be able to determine if the Japanese
character drawn is correct or wrong.

4. INTRODUCTIONTO MYO
ARMBAND AND
GOOGLE CARDBOARD

5. Myo Armband
• Reads muscle activity in forearm
• Electromyography technique
• EMG sensor & Motion sensor (9-axis IMU)
• Detect 5 gestures
• Connect to devices via Bluetooth

6. Google Cardboard

7. BACKGROUND STUDY

8. Virtual Reality Devices
Oculus Rift HTC Vive Sony’s
Project
Morpheus
Samsung
Gear VR
Host Device Windows PC Windows PC Play Station 4 Galaxy Note
4, S6, S6 Edge
Field Of View
110° Approx. 100° Approx. 100° 96°
Latency Less than
20ms
22ms Less than
18ms
Less than
20ms
Refresh rate 90Hz 90Hz 120Hz Mere 60Hz
Positional
Tracking
Yes Yes
(Lighthouse)
Yes No

9. Gesture Recognition Devices
Device Method of Gesture Recognition
Kinect Depth Sensor
Play Station Move Camera, Orb on the controller
Wii Wii Remote, Sensor bar
Leap Motion 2 cameras, 3 infrared LED

10. Requirements
Hardware
• Android phone (Android 4.3
and up & Bluetooth LE 4.0)
• Google Cardboard
• Myo Armband
Software
• Android Studio 1.0 or higher
• Google Cardboard sdk
• MYO Android sdk
• Unity 5.0
• Blender

11. Flow Diagram
User
Myo Armband
Phone
Identify the
Pose
hand
pose
head
movement
raw
EMG
data
gestural
data
gestural
data
gestural
data
manipulated
classroom
phone movement
updated orientation
ofVR world
Plugin
Identify the
control assigned
to the gesture
Track Rotation
&Track Position

12. FYP 1 Accomplishments
• 3D classroom environment
• Integrate Myo Armband into 3D environment
• Created plugin to connect Android-Myo, Unity

13. FYP 2 Accomplishments
• Capture hand movement and convert into drawing (gyroscope)
• Included two Japanese characters for users to trace.
• Added sound effect and background music.
• Identify whether the strokes are drawn correctly (compare
points).

14. Future works
• Improve plugin (slow response)
• Add gestures

15. Screenshot

16. Screenshot

17. Screenshot

18. Screenshot

19. DEMONSTRATION

20. Q&A