2016 D-STOP Symposium ("Smart Cities") session by CTR's Christian Claudel. Get symposium details: http://ctr.utexas.edu/research/d-stop/education/annual-symposium/

1. Networks of wearables and
augmented reality for
vulnerable user protection
Christian Claudel
Assistant Professor
CAEE Department
University of Texas, Austin
D-STOP Annual Symposium 2016

2. Motivations
• Very high number of road accidents
- In the world, 1.3 million deaths/year
- 20-50 million injuries/year
• 90% of all crashes are caused by human error
• An increasing number of crashes occurs in cities,
with a high concentration of heterogeneous users
(cars, bicycles, pedestrians)
• State of the art: policy, planning, but no active safety

3. Proposed system
• Autonomous vehicles will help address the problem,
but will take time to ramp up
• Even with autonomous vehicles, we need
coordinated response by humans (ex: pedestrians,
bicyclists): human in the loop control
• Wearables (smart glasses, smart watches) can be a
solution to the problem:
- Low cost (distributed among users)
- Will penetrate market faster than autonomous vehicles
- Augmented reality capabilities for actuation

4. Vision
• Users wearing smart glasses (and other wearables)
• Positioning through higher high resolution GPS (RTK-GPS), or UAVs
equipped with cameras

5. Vision
• Learning-based framework to detect user intent and predict future
actions based on video and wearable inertial/positioning data
• Collision detection using predicted reachable sets, and resolution using
real-time path visualization (through augmented reality)

6. Current implementation

7. Challenges
• Sensing (using phone, glass, watch)
- IMUs for head and wrist tracking
- Cameras for scene detection/user intent detection
- RTK-GPSs for position estimation
• Networking
- Need for a low latency communication channel (Bluetooth, DSRC,
5G?)
• User path forecast and collision avoidance
- Motion tracking with machine learning for path forecast
- Collaborative collision avoidance with uncertain (human) actuation
– requires human factor experts
• Cybersecurity
- GPS spoofing
- Need for sensor fusion

8. Future work
• Currently: Google Glasses interfaced with RTK-GPS,
head motion tracking
• Future work:
- use ML to forecast user paths, develop collision detection
and avoidance algorithms
- use R7 glasses (with better field of view)
- Use of drones to replace RTK GPSs in the near future
(and provide data for path forecasts algorithms)
• Study human response
• Investigate for other uses: traffic intersection
management with mixed autonomous/human vehicles