Kotani, K., Sun, W., Kitani, T., Shibata, N., Yasumoto, K., Ito, M.:Inter-Vehicle Communication Protocol for Cooperatively Capturing and Sharing Intersection Video, Proc. of 2nd IEEE Intelligent Vehicular Communications System Workshop (IVCS'10), (CD-ROM), Jan. 9th, 2010. DOI:10.1109/CCNC.2010.5421635 (Jan. 2010).
http://www.aist-nara.ac.jp/~sunweihua/papers-fullversion/I-10-01-02.pdf
For accident prevention at intersections, it is useful for
drivers to grasp the position of vehicles in blind spots.
This can be achieved without infrastructure if some vehicles
passing near the intersection capture and share live
video of the intersection through inter-vehicle communications.
However, such video streaming requires a congestion
control mechanism. In this paper, aiming to let a driver
grasp the situation at an intersection, we propose a method
to select vehicles that send a video in order to generate a
live bird’s-eye-view video of the intersection. In our method,
each vehicle at an intersection exchanges information with
others, such as the sub-areas of the intersection it captures,
the quality of its video, and its position and speed. Based
on the exchanged information, each vehicle autonomously
judges whether it should send its video or not. Through
simulation with a QualNet simulator, we confirm that our
method achieves a good video arrival rate and video quality
sufficient for practical use.
(Slides) A Method for Distributed Computaion of Semi-Optimal Multicast Tree i...
(Slides) Inter-Vehicle Communication Protocol for Cooperatively Capturing and Sharing Intersection Video
1. Kazuya Kotani†1, Weihua Sun†1, Tomoya Kitani†2,
Naoki Shibata†3, Keiichi Yasumoto†1, Minoru Ito†1
†1 Nara Institute of Science and Technology (NAIST), Japan
†2 Shizuoka University, Japan
†3 Shiga University, Japan
1
Inter-Vehicle Communication Protocol for
Cooperatively Capturing and
Sharing Intersection Video
(revised slides)
2. 2
Goal: Allow drivers to grasp vehicle positions
in blind spots at intersections
Overview
Proposed Method
• Vehicles shoot videos from onboard cameras
• Each vehicle decides if it should send its video wirelessly
• according to available bandwidth, etc.
• Each vehicle receives videos from multiple vehicles and
compose them into a Bird's Eye View Video (BEVV)
• In order to reduce latency, It does not use multi-hop
communication
4. 4
We propose a system that allows drivers to see
vehicle positions in blind spots in an intuitive way
Intersection
43.9%
Local Street
26.8%
Vehicles in blind spots
[1] ITS: “Ministry of Land, Infrastructure, Transport and Tourism,”
http://www.mlit.go.jp/road/ITS/.
Background
City area
74.8%
43.9% of traffic accidents happen at
◦ intersections in city area[1]
They are collisions between
◦ vehicle and vehicle
◦ vehicle and pedestrian
Most of these are caused by
6. DSSS(Driving Safety Support Systems) project [2]
◦ Gathers other vehicles’ position with inter-vehicle comm.
◦ Vehicles without communication devices are not displayed
◦ Positions are not accurate because of GPS positioning error
6
[3] Ota et al. : ``Visual Reconstruction of an Intersection by Integrating Cameras on Multiple Vehicles,’’ Proc. on
Machine Vision Applications (MVA2007), pp.335–338(2007).
[2] Honda Motor Co., Ltd.: “Honda Technology,” http://www.honda.co.jp/news/2009/4090219.html?from=rss.
Method to compose a
birds-eye-view video(BEVV)[3]
◦ Multiple videos are taken from multiple cars from different angles
◦ Videos can be composed into one BEVV
◦ The method includes no mechanism for exchanging videos
Related Work
7. 7
IEEE802.11b, 200 Kbps
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10
PacketArrivedRatio
[%]
Number of Vehicles
Problems with this method
• Vehicle density can be too high
• Wireless bandwidth is not insufficient
We need to carefully select vehicles that send videos
to save wireless bandwidth
Problem with a straightforward approach for
exchanging captured video
1. Vehicles broadcast request messages before turning right at intersection
2. All vehicles in intersection capture and send videos wirelessly
3. Vehicles receive videos and compose BEVV
The straightforward method
8. Objectvie
◦ Drivers need high quality view of the blind spots
Approach
◦ Assign priority to each vehicle for sending video
◦ The following vehicles are assigned high priorities
Vehicles that can capture the requested areas
Vehicles that can capture high quality videos
◦ Vehicles send out the captured video
according to their priorities
8
Key Idea for Selecting Vehicles
0
7
56
0
4
8
6
5
10. We assume that it is possible to compose a BEVV from
videos taken from any different angles
We conducted preliminary experiment with scaled down
model of intersection
convert
10
compose
Assumptions
Captured
on camera
Converted
video
frame
Multiple video
frames
Bird’s eye view
frame
11. An intersection is divided into rectangular cells
◦ Each cell is defined by its geographical position
◦ Each vehicle knows which cell it is located
◦ Each vehicle know cells that can be captured by its camera
Finer position can be determined by visually analyzing
captured video[4]
11
Requesting
vehicle
[4] Alberto et al. : `` Multi-Resolution Vehicle Detection using Artificial Vision,’’
Proc. on IEEE INTELLIGENT VEHICLES SYMPOSIUM (IV’04), pp.310–314(2004).
Dividing Intersection into Cells
Request cells
12. Vehicles have the following equipment
◦ On-board camera
◦ Car navigation system with GPS
◦ Wireless LAN communication device
◦ On-board computer
On-board computer can sense the state of
indicator (going straight, turning left or right)
12
Assumptions on Vehicle
13. 13
1. All vehicles periodically
exchange Hello
Message
2. Requesting vehicle
broadcasts
Request Message
3. Each vehicle which received
Request Message calculates all
vehicles’ Priority
86
7
6
0
00
70
0
7
4. Each vehicle decides if it
should send video frames
Requesting vehicle
How Proposed Method Works
14. There are two phases in the proposed method
Information Exchange Phase
◦ Vehicles exchange messages to get the information from the
surrounding vehicles
Vehicle Selection Phase
◦ Videos are sent utilizing available bandwidth as much as
possible
◦ Priority is calculated in this phase
14
Two phases in the proposed method
15. Goal : Each vehicle knows other vehicles’ information
Vehicles close to an intersection exchange the following
messages
Hello Message
◦ All vehicles inform their own status to other vehicles
position, speed, capturing cells, video quality,
number of video frames already sent
Request Message
◦ Requesting vehicle sends requested cells to other vehicles
items of Hello Message, requested cells
◦ When a vehicle receives a Request Message, it transits to Vehicle
Selection Phase
15
Information Exchange Phase
16. Goal : Making vehicles send videos utilizing available
bandwidth as much as possible
Vehicles need to know the usage of communication
bandwidth
◦ The usage of communication bandwidth is defined by the ratio
of
To decide if each vehicle should send videos
◦ Calculate the priority based on the following items
Requested cells that can be captured by the camera on vehicle
Video quality for each cell
◦ Priority is the weighted sum of these values
Preliminary experiment
We conducted preliminary experiment to determine these weights 16
Vehicle Selection Phase
Number of actually received video frames
Number of video frames informed in Hello messages
17. Deciding if each vehicle sends video
Each vehicle calculates priorities of vehicles
independently
◦ Usually, the results of calculations are almost same
For each direction, the vehicle with the highest
priority decides to send the video
◦ For each direction, usually the car closest to the
center of intersection starts sending video
◦ Then, the cars following the first car sends their
videos in turn
◦ The priority is dynamically changed according to the
positions of moving vehicle
17
Vehicle Selection Phase(contd.)
18. 1. Background
2. Related work
3. Proposed method
4. Experimental results
5. Conclusion
18
Outline
19. Goal
◦ To see if the proposed method can select vehicles that capture
requested cells in high quality
Evaluation criteria
1. The number of video frames covering each cell in requested cells
2. The number of video frames covering requested cells from each
driving direction
3. The priority of video frame received by request vehicle
19
Experimental Evaluation
20. We used QualNet[5]
◦ Created terrain data to imitate actual intersection in Kyoto, Japan
◦ Requesting vehicle goes into intersection ⇒ turns to the right
◦ The number of requesting vehicle is one
Transmitting bit rate: 200 [Kbps]
20
Settings
21. Simulation parameters
Packet parameters
21
item value
The number of vehicles 60
Requested cells 3 cells
Wireless LAN standard IEEE802.11b
Vehicle density Dense, Middle, Sparse
Carrying rate of equipments 100%, 60%, 30%
Packet name Packet size
Transmission
interval
Hello Message 300[byte] 0.5[s]
Request Message 300[byte] 0.5[s]
Video frame 1666[byte] 0.067[s]
Parameter setting of experiment
22. We compared the following methods for
selecting vehicles to send their videos
i. Proposed method
ii. All vehicles
All vehicles near intersection send video
iii. Vehicles closest to the center of intersection
The vehicle closest to the center of intersection from each
driving direction sends video
Measured evaluation criteria with these three methods
22
Comparison of Three Methods
24. In order to see vehicles in blind spot by video, video
quality of at least 10[fps] in average is necessary
RNF is the minimum number of received frames that
satisfies the above condition for each cell
◦ Simulation time and RNF value in each density setting are
below
24
Required Number of Frames (RNF)
Vehicle density Dense Middle Sparse
Simulation time 35[s] 58[s] 88[s]
RNF(each direction) 350[frame] 580[frame] 880[frame]
RNF(each cell) 1400[frame] 2400[frame] 3520[frame]
×4 (driving directions)
25. In Dense setting with 100% carrying rate
◦ Proposed method achieves RNF on all request cells
◦ All vehicles doesn’t achieves RNF on any cells
◦ Vehicles near the center achieves RNF on two cells
25
Number of video frames (Request cells) 1/3
0
500
1000
1500
2000
2500
3000
call1 cell2 cell3
NumberofReceivedFrames
Request cells
Proposed method
All vehicles
Vehicles near the center
RNF
26. In Dense Middle and Sparse setting with 100% carrying
rate
◦ Proposed method sends more frames than any other method
regardless of vehicle density
26
Number of video frames (Request cells) 2/3
0
500
1000
1500
2000
2500
3000
call1 cell2 cell3
NumberofReceivedFrames
Request cells
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
cell1 cell2 cell3
NumberofReceivedFrame
Request cells
0
1000
2000
3000
4000
5000
6000
7000
8000
cell1 cell2 cell3
NumberofReceivedFrame
Request cells
Proposed method
All vehicles
Vehicles near the
center
RNF
Dense Middle Sparse
27. In Dense setting, with 100, 60, 30% carrying ratio
◦ Proposed method provides more frames than other methods
◦ Request cells achieving RNF decreases as carrying rate decreases
27
Number of video frames (Request cells) 3/3
0
500
1000
1500
2000
2500
3000
call1 cell2 cell3
NumberofReceivedFrames
Request cells
0
500
1000
1500
2000
2500
3000
call1 cell2 cell3
NumberofReceivedFrame
Request cells
0
500
1000
1500
2000
2500
3000
call1 cell2 cell3
NumberofReceivedFrame
Request cells
Proposed
method
All vehicles
Vehicles near
the center
RNF
100% 60% 30%
28. In Dense setting with 100% carrying rate
◦ Proposed method achieves RNF from all directions
◦ Other methods achieve RNF from one or two directions
28
Number of video frames (Driving direction)
0
200
400
600
800
1000
1200
1400
1600
1800
green1 green2 red1 red2
NumberofRecievedFrame
Driving Direction
Proposed method
All vehicles
Vehicles near the center
RNF
29. In Dense setting with 100% carrying rate
◦ There are more high priority frames by proposed method than other
methods
29
0
10
20
30
40
50
60
70
80
90
100
24~30 18~24 12~18 6~12 0~6
CummulativeDensityof
Frmae[%]
Priority
Proposed method 100%
All vehicles 100%
Vehicles near the center 100%
Effective in Dense setting with 60% or more carrying rate
More efficient than other methods in all environments
Conclusions of experiment results about proposed method
Priority
Cumulative distribution function (CDF) graph
Proposed method can make BEVV which contains request cells in high qua
30. Vehicle selection method
◦ For creating a bird's eye view video by letting selected
vehicles efficiently capture and exchange video
◦ It assigns priorities to send videos to other vehicles
◦ Vehicle with higher priority sends videos within the
available communication bandwidth
Results
◦ Proposed method is especially effective in the dense
setting with 60% or more arrival rate
30
Conclusion
31. Kotani, K., Sun, W., Kitani, T., Shibata, N.,
Yasumoto, K., Ito, M.:Inter-Vehicle Communication
Protocol for Cooperatively Capturing and Sharing
Intersection Video, Proc. of 2nd IEEE Intelligent
Vehicular Communications System Workshop
(IVCS'10), (CD-ROM), Jan. 9th, 2010.
DOI:10.1109/CCNC.2010.5421635 [ PDF ]
31
Notas del editor
Thank you, chairman.
I will talk about / Inter-Vehicle Communication Protocol / for Cooperatively Capturing and Sharing Intersection Video
I will explain / the overview of this study.
Our goal is that / drivers can grasp / vehicle positions in blind spots of intersection
To achieve this goal, in proposed method / vehicles shoot videos from onboard cameras
and each vehicle decides / if it sends videos wirelessly
and each vehicle receives these videos / and compose them into a bird's eye view video
This is the outline of this presentation
Next, I will explain the research background
According to this literature (文献[1]を指しながら), most traffic accidents in Japan / happened in the intersection of the city area, / and there are many collisions of vehicle-to-vehicle and vehicle-to-pedestrian.
We think that / most of these near-intersection accidents are / mainly caused by / vehicles in blind spots
So, for traffic accident prevention, We propose a system for drivers / to see the vehicle positions in the blind spots / intuitively.
Next, I will explain the Related work
Honda Motor Company has promoted / DSSS (driving safety support systems) project / based on ITS technology.
Vehicles equipped with DSSS / can get other vehicle’s position information / using IVC (inter-vehicle communication)
However, Vehicles without communication devices are / not displayed
and positions are not accurate enough / because of GPS positioning error
Ota proposed a method / to compose bird’s eye view video(クリック)
In this method, multiple vehicles / take videos from different angles
And they composed into / one BEVV
However, this method has no mechanism / for exchanging videos among vehicles
Here, I will introduce / Straightforward approach / of exchanging captured video in Japan
First, vehicles broadcast request message / before turning right at intersection
Secondly, all vehicles in intersection / capture and send videos wirelessly
Finally, vehicles receive videos and compose BEVV
However, there are some problems in this method
Vehicle density can be high, and wireless bandwidth can be insufficient
This graph shows the simulation result / of packet arrival ratio / depending on the number of vehicles.
Here, wireless LAN standard is IEEE802.11b and the sending bit rate / of each vehicle is two hundreds[kbps].
We see that / the packet arrival ratio / rapidly decreases / as the number of vehicles increases.
So, we need to / carefully select vehicles / which send videos / for efficient use of wireless bandwidth.
Next, I will introduce the key idea of selecting vehicle / which send video
To grasp vehicles in blind spot, we think that / drivers want to get video / capturing blind spot in high quality
Proposed method / chooses vehicles to send videos / so that the request can be fulfilled with good result
So, Proposed method assigns priority to each vehicle(クリック)
In proposed method, (指しながら)these vehicles are given higher priority.
vehicles which can capture request areas, and capture high quality videos
Next, I will explain the proposed method
I will explain / some assumptions for proposed method.
First, I will explain the assumption / on composing bird’s eye view video.
We assume that / it is possible to compose bird’s eye view video / from videos taken from different angles
(図を指しながら)We conducted preliminary experiment / with scaled down model of intersection
(クリック)
We exchanged videos captured on camera by multiple nodes with wireless Adhoc communication
And We converted video in each node
and composed bird’s eye view video from multiple converted videos in real time.
Next, I will explain the assumption on intersection.
In proposed method, region in intersection is / divided into rectangular cells
and each cell is defined / by geographical position
(クリック)And, request vehicle knows / request cells which it needed
(クリック)And all vehicles know cells / which can be captured by their on-board camera
Capturing cells are determined by / vehicle positions and by .analyzing captured video
Next, I will explain the assumption on vehicle
Vehicles have / in-vehicle camera, car navigation system with GPS, wireless LAN communication device, and in-vehicle computer.
And, In-vehicle computer can sense / state of direction indicator , for example go straight, and turn right
Here, I will explain / each vehicle’s behavior in proposed method
First, All vehicles periodically exchange Hello Message.
Secondly, request vehicle broadcasts Request Message
Thirdly, Each vehicle which received Request Message / calculates other vehicle’s priority
Finally, Each vehicle decides / if it should send video frames
Next, I will propose the method / to decide vehicles which send video
For efficient use of wireless bandwidth, Proposed method chooses / the set of vehicles that sends video
And to choose the set of vehicles, Vehicles near the intersection / need other vehicles’ information
So, I will introduce Proposed method / by dividing into / Information Exchange Phase and Vehicle Selection Phase
First, I will explain Information Exchange Phase
In this phase, To know other vehicles’ information, / vehicles near the intersection exchange their information
I will show these two kinds of messages / exchanging among vehicles
In Hello Message, all vehicles inform / vehicle’s own status to other vehicles.
This message contains / vehicle position, speed, capturing cells, video quality, and the number of video frames already sent
In Request Message, request vehicle informs / request cells to other vehicles
This message contains / items of Hello Message, and request cells
When each vehicle receives Request Message, / it transits to Vehicle Selection Phase
I will introduce vehicle selection phase,
In proposed method , As many vehicles as possible / send videos / within the available communication bandwidth
So, Vehicles need to grasp / the usage of communication bandwidth
The usage of communication bandwidth / is defined by the ratio of number of actually received video frames / to / number of video frames already sent in Hello Message
And to decide / if own vehicle should send videos,
Proposed method assigns priority to send videos / to vehicles/ based on number of cells that capture request cells / and video quality
Priority is the weighted sum of these values
And we conducted preliminary experiment / to determine these weights
I will explain the method / to decide if each vehicle sends video
The judgment criteria for each vehicle are / (指しながら)these.
First, it captures videos in request cells
Secondly, it has the highest priority / in vehicles heading same driving direction.
Finally, if there is room / for communication bandwidth, it’s priority is within top ten / in all vehicles
Next, I will explain Experimental results
We conducted evaluation experiment / to evaluate whether / proposed method can select vehicles /that capture request cells in high quality / by evaluation criteria
I will explain evaluation criteria.
First is / the number of video frames / covering each cell in request cells
Second is / the number of video frames /covering request cells from each driving direction
Third is / the priority of video frame / received by request vehicle
Next, I will explain the Experiment setting
In this experiment, we used network simulator / QualNet
We created terrain data / to imitate actual intersection in Kyoto Japan
As a simulation scenario, when request vehicle goes into intersection, the scenario starts.
And when the vehicle turns to the right, the scenario ends
And, the number of request vehicle is / one
(クリック)
About Video frame packet, transmitting bit rate is / two hundreds Kilo bit per second
Next, I will explain parameter setting of experiment
In this experiment, the number of vehicle is sixty,
The number of request cells is three,
Wireless LAN standard is I triple E eight O two dot eleven b
Vehicle density is Dense, Middle, and Sparse
Carrying rate of equipments is / one hundred%, sixty%, and thirty%
packet size of Hello Message and Request Message are / three hundreds[byte], and Transmission interval is / zero point five[second]
And the parameters of video frame is /.set up this way
Next, I will explain comparison of three methods
We compared these methods / for selecting suitable vehicles / to send their videos
First, proposed method
Second, All vehicles near intersection send video
Third, vehicles / nearest to the center of intersection / from each driving direction send video
We measured evaluation criteria / with these three methods
Here, I explain the driving direction and request cells
In this picture, the green signal is (差しながら)this direction, and the red signal is (差しながら)this direction
And, this request vehicle requests (差しながら)these request cell1, 2, 3 before turning right
Here, I will explain Required Number of Frames(RNF)
RNF/ evaluates number of received frames
In this experiment, to watch the vehicles in blind spot by video / at least 10 [fps] in average is necessary
Simulation time and RNF value in each density setting are this table
For example, in Dense setting RNF of each direction is three hundreds fifty frames
And RNF of each cell is one thousand four hundreds frames
Next, I will explain / the number of video frames / in each request cell
In Dense setting, with 100 % carrying rate,
Proposed method / achieves RNF / on all request cells
All vehicles doesn’t achieves / RNF on any cells
Vehicles near the center / achieves RNF / on two cells
Next, Dense(左) Middle(真ん中) and Sparse(右) setting, with 100% carrying rate
Proposed method sends more frames / than any other method / regardless of vehicle density
Next, In Dense setting, with 100, 60, 30% carrying ratio
Proposed method provides / more frames than other methods
However, Request cells achieving RNF / decreases as carrying rate decreases
Next, I will explain / the number of video frames /containing request cells from each driving direction
In Dense setting, with 100% carrying rate
Proposed method achieves RNF from all directions
Other methods achieves RNF from one or two directions
Finally, I will explain the result of priority.
This graph is cumulative distribution function CDF graph / which shows the distribution of the priority / for every frames
In Dense setting, with 100% carrying rate,
There are more high priority frames by proposed method than other methods
So, proposed method can make BEVV which contains request cells in high quality
As Conclusions of experiment result / about proposed method,
It is effective in Dense setting / with sixty% or more carrying rate
And It is more efficient / than other methods in all environments
Finally, I will explain conclusions,
In this presentation, We proposed vehicle selection method /
for creating bird's eye view video / by letting selected vehicles / efficiently capture and exchange video
And proposed method / assigns priority to send videos to vehicles
And vehicle whose priority is higher / sends videos within the available communication bandwidth
As a result, we confirmed / proposed method is / effective in Dense setting / with 60% or more carrying rate
As future work, we will evaluate about the delay of / communication and creating bird’s eye view video
Thank you for your attention