Session 48 Johan Karlsson

Driver
Drowsiness and
Distraction
Detection by
Sensor Fusion
D4SF
Johan Karlsson, Autoliv
Transportforum 2011

Fordonsstrategisk Forskning och Innovation FFI – D4SF
ALR-JKAR/Jan2011/Transportforum - 1 Copyright Autoliv Inc., All Rights Reserved

Overview

Background
Goals
Drowsiness detection
(Distraction detection)
Method
Data collection
Training/optimization of classifier
Sensor fusion
Results
Reference – ground truth
Improvement by (f)using parallel detectors


Driver Drowsiness detection

Drowsy driving is a road safety problem
- drowsiness contributing in 10-30% of accidents (Anund & Patten 2010)

What can be done?
Commercial fleet traffic
Fatigue Risk Management
Work time regulation
Detection and warning
Privately owned vehicles
Detection and warning

Detection?
Detection systems offered as option from several OEMs
So far, performance is far from ideal...


Target and Goals
Sensitivity
Different indicators exist Specificity Availability

- ’Physiology’ A
Indicator measures+ blink duration etc.
- – +
- Driving performance measures - lane keeping measures
- EnvironmentBmeasures - time of day, traffic, road type
Indicator –
(previous sleep possible in commercial fleet vehicles??)
+ +

Indicator C + + –
Various indicators have different strengths and weaknesses

Improve performance by fusing data from multiple indicators+
Fusion ++ ++ +

The fusion algorithm shall show an improvement in:
- Overall performance
- Reduced number of faulty detections
- Increased number of correct detections

Data collection
On-road tests were conducted with
Data collection governmental approval (N2007/5326/TR)
and ethical approval by Regional ethics
Relevant vehicle data approval board (EPN 142-07 T34-09).

Speed, lane position, SW angle, pedals etc.
Video based gaze direction, eyelid opening, head pos
KSS value every 5 minute
EEG, EOG and EMG
Video recordings (road scenery and cabin)
In total: 43 drivers have completed 3 drives each
Procedure: Each driver drove three times during one day
(day, evening and night). Trip duration 80-90 minutes


Test Route
Road RV34
Mostly 9 m width
Driving lane width 3,75 m
Speed limit - mostly 90 km/h

Numbers on map are Yearly day traffic
volume in January 2002

We know of only a few similar studies
performed on public roads

90 minute driving,
approx 115 km distance

Rested safety driver –
dual command


Ground Truth – KSS
KSS Description in Swedish Verbal description
1 extremt pigg extremely alert
2 mycket pigg very alert
3 pigg alert
4 ganska pigg rather alert
5 varken pigg eller sömnig neither alert nor sleepy
6 första tecknen på sömnighet some signs of sleepiness
7 sömnig, ej ansträngande vara vaken sleepy, but no effort to keep alert
8 sömnig, viss ansträngning vara vaken sleepy, some effort to keep alert
9 mycket sömnig, ansträngande vara vaken, very sleepy, great effort to keep alert,
kämpar mot sömnen fighting sleep

+ Simple to collect
+ Simple to understand – immediately ready for analysis

- Training needed for participants
- Some offset for inexperienced participants?


Example indicators of driver sleepiness

Closing Opening Open
200 ms Closed

Blink duration (AS):
Amplitude
400 ms

Closed

Mean blink duration
Short Blink Long Blink

GVI (Sandberg 2008)

Lane keeping variability (Lane): G=
1 N
∑ w(zi ) | zi |k
N i=1
Variability in Steering wheel position or
zi = xi − (δ x + (1 − δ ) p)
Lane Position. e.g. using Generic
cL cR
Variability Indicator (Sandberg 2008) . w( z) = −α L ( z −β L )
+ −α R ( z −β R )
1+ e 1+ e

Time-of-day (TPM):
Expected drowsiness with regard to time
of day (circadian rythm)

* Each indicators has several parameters
that needs to be tuned for optimal
performance

Video examples

Video examples


Sensor fusion
SVM (Support Vector Machine):
Machine learning method using data from field tests to calculate “best fit” function
between indicator values and ground truth (KSS rating scale)
Indicator parameters optimized simultaneously with training of SVM
Data sets for SVM training and validation are from separate drivers.
Thus, validation is done on truly “never-before-seen” data.

Drowsy data
Goal:
Find the maximum margin hyperplane
Indicator B

Alert data

Indicator A

Evaluation Method
Assuming a binary classification,
A
alert or =
sensitivitydrowsy
A+C Ground truth
Performance is the mean value of
D Non-
specificity =
sensitivity and specificity Drowsy
Performance + D
Drowsy
B is related to the

Algorithm output
A B
sensitivity + specificity
proportion of the time where the Detect
performancis =
algorithm e correct
(hit) (false hit)

2 Non- C D
Detect (miss) (correct reject)

KSS = ground truth

Ground truth cutoff Sum A+C B+D
Binary Algo output
KSS

Time

Example of results from sensor fusion

Model Fitness Sens Spec
Blink 0.66 (0.64) 0.36 (0.32) 0.96 (0.95)
Blink + Circadian 0.80 (0.83) 0.77 (0.79) 0.83 (0.87)

Blink + Lane + Circ. 0.80 (0.78) 0.68 (0.68) 0.92 (0.88)

Blink + Steer + Circ. 0.80 (0.85) 0.76 (0.81) 0.84 (0.89)

First figure is training data performance
second figure is test data performance Decision every 1 minute
KSS >= 7 drowsy
KSS < 7 alert


Fulfillment of goals

The fusion algorithm shall show an improvement in:
- Improved performance true
- Increased number of correct detections true
- Reduced number of faulty detections (?)

Clearly improved overall performance
– Minor differences between different combinations


Summary

Controlled experiment on public roads
43 drivers so far
What is ideal performance?
Method developed with focus on mathematical performance
Most important goal is to have relevant warnings
More data is needed:
Different road types
Different conditions (weather, drive duration etc.)
Different driver types (age, cultural differences etc.)


Thank you for you attention!


Session 48 Johan Karlsson

Recomendados

Recomendados

Más contenido relacionado

Destacado

Destacado (10)

Más de Transportforum (VTI)

Más de Transportforum (VTI) (20)

Último

Último (20)

Session 48 Johan Karlsson