SlideShare una empresa de Scribd logo
1 de 22
Descargar para leer sin conexión
Sensor fusion for autonomous
driving perception
Bagni Fabio
228594@studenti.unimore .it
San Francisco - 15 May 2019
1
Fabio Bagni
San Francisco - 15 May 2019
Autonomous driving perception
Precision and reactivity
Risks of accidents, traffic laws respect.
2
Fabio Bagni
San Francisco - 15 May 2019
Embedded platforms
3
Fabio Bagni
San Francisco - 15 May 2019
Goals
Real time 3D detection
● High performances
○ required 10 Hz frequency
● High precision
● Real time on embedded platforms
4
Fabio Bagni
San Francisco - 15 May 2019
UNIMORE HiPeRT prototype
5
Fabio Bagni
San Francisco - 15 May 2019
Sensors
● LiDAR
○ 3D point cloud
● Cameras
○ images
Heterogeneous outputs
6
Fabio Bagni
San Francisco - 15 May 2019
Sensor fusion
What is sensor fusion:
● Matching of different sensors
views
Why sensor fusion:
● Heterogeneous information of
environment
7
Fabio Bagni
San Francisco - 15 May 2019
Fields of view
LiDAR: 360 degrees around the vehicle
Camera: wide angle, disposed do cover 360
degrees.
8
Cameras
LiDAR
Fabio Bagni
San Francisco - 15 May 2019
Deep learning approach
Camera:
● Bird’s Eye View estimation
● Object detection
LiDAR
● Bird’s Eye View calculation
● Object detection
9
Image
Point
cloud
3D
detection
3D
detection
Fusion
Fabio Bagni
San Francisco - 15 May 2019
Deep learning approach time
Image stitching for multiple camera
fusion
10
➔ Under10 Hz
Notenoughresponsive
1. Ming Liang, Bin Yang, Shenlong Wang and Raquel Urtasun: Deep Continuous
Fusion for Multi-Sensor3D ObjectDetection.In: ECCV (2018).
Fabio Bagni
San Francisco - 15 May 2019
Static alignment sensor fusion
Alignment of LiDAR with each camera
individually.
11
Calibration
Preprocessing
Fabio Bagni
San Francisco - 15 May 2019
Features extraction
Matching is made by a calibration with a
perforated panel.
Holes are detected by all sensors.
Holes’ centers are used as features to be
matched.
12
Fabio Bagni
San Francisco - 15 May 2019
Cylindrical projection
Fields of view matching is made projecting sensors’ outputs on the surface of a
cylinder.
13
Fabio Bagni
San Francisco - 15 May 2019
Projections complexity
LiDAR cylindrical projection: Depthmap
O(n) : one operation per point
14
Camera cylindrical projection
O(n) : one operation per pixel
PARALLELIZABLE
Fabio Bagni
San Francisco - 15 May 2019
Calibration results
15
Fabio Bagni
San Francisco - 15 May 2019
Colored points
Alignment of each point with the
corresponding pixel on the image.
16
Fabio Bagni
San Francisco - 15 May 2019
Object detection
Neural network for object detection on
camera frame
Clustering of LiDAR points
Bounding boxes matching
17
Fabio Bagni
San Francisco - 15 May 2019
Object detection results
18
Fabio Bagni
San Francisco - 15 May 2019
Object detection
average time
19
LiDAR output : 10 Hz
Time limit : 100 ms
Thanks for your attention
Fabio Bagni - 228594@studenti.unimore.it
20
Fabio Bagni
San Francisco - 15 May 2019
21
Fabio Bagni
San Francisco - 15 May 2019
Homography
Homography allow to translate camera projection plane on the LiDAR projection plane.
This method make cylindrical surfaces match.
22

Más contenido relacionado

La actualidad más candente

SOFTWARE-DEFINED SUCCESS: THE NEXT GENERATION OF AUTOMOTIVE LIDAR
SOFTWARE-DEFINED SUCCESS: THE NEXT GENERATION OF AUTOMOTIVE LIDARSOFTWARE-DEFINED SUCCESS: THE NEXT GENERATION OF AUTOMOTIVE LIDAR
SOFTWARE-DEFINED SUCCESS: THE NEXT GENERATION OF AUTOMOTIVE LIDAR
iQHub
 

La actualidad más candente (20)

Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...
 
SOFTWARE-DEFINED SUCCESS: THE NEXT GENERATION OF AUTOMOTIVE LIDAR
SOFTWARE-DEFINED SUCCESS: THE NEXT GENERATION OF AUTOMOTIVE LIDARSOFTWARE-DEFINED SUCCESS: THE NEXT GENERATION OF AUTOMOTIVE LIDAR
SOFTWARE-DEFINED SUCCESS: THE NEXT GENERATION OF AUTOMOTIVE LIDAR
 
Cv_Chap 4 Segmentation
Cv_Chap 4 SegmentationCv_Chap 4 Segmentation
Cv_Chap 4 Segmentation
 
Multiple Object Tracking
Multiple Object TrackingMultiple Object Tracking
Multiple Object Tracking
 
Recent Object Detection Research & Person Detection
Recent Object Detection Research & Person DetectionRecent Object Detection Research & Person Detection
Recent Object Detection Research & Person Detection
 
ADAS.ppt.pptx
ADAS.ppt.pptxADAS.ppt.pptx
ADAS.ppt.pptx
 
Machine Learning - Object Detection and Classification
Machine Learning - Object Detection and ClassificationMachine Learning - Object Detection and Classification
Machine Learning - Object Detection and Classification
 
Stereo Matching by Deep Learning
Stereo Matching by Deep LearningStereo Matching by Deep Learning
Stereo Matching by Deep Learning
 
ICRA 2019 速報
ICRA 2019 速報ICRA 2019 速報
ICRA 2019 速報
 
IOT Based Smart Parking and Damage Detection Using RFID
IOT Based Smart Parking and Damage Detection Using RFIDIOT Based Smart Parking and Damage Detection Using RFID
IOT Based Smart Parking and Damage Detection Using RFID
 
Slam algorithms
Slam algorithmsSlam algorithms
Slam algorithms
 
Computer Vision - Artificial Intelligence
Computer Vision - Artificial IntelligenceComputer Vision - Artificial Intelligence
Computer Vision - Artificial Intelligence
 
MOBILE ROBOT NAVIGATION SYSTEM
MOBILE ROBOT NAVIGATION SYSTEMMOBILE ROBOT NAVIGATION SYSTEM
MOBILE ROBOT NAVIGATION SYSTEM
 
Parking space detect
Parking space detectParking space detect
Parking space detect
 
FastDepth: Fast Monocular Depth Estimation on Embedded Systems
FastDepth: Fast Monocular Depth Estimation on Embedded SystemsFastDepth: Fast Monocular Depth Estimation on Embedded Systems
FastDepth: Fast Monocular Depth Estimation on Embedded Systems
 
Motion detection system
Motion detection systemMotion detection system
Motion detection system
 
Introductory Level of SLAM Seminar
Introductory Level of SLAM SeminarIntroductory Level of SLAM Seminar
Introductory Level of SLAM Seminar
 
Computer Vision for autonomous driving
Computer Vision for autonomous drivingComputer Vision for autonomous driving
Computer Vision for autonomous driving
 
Visual Object Tracking: review
Visual Object Tracking: reviewVisual Object Tracking: review
Visual Object Tracking: review
 
Road Object Detection
Road Object DetectionRoad Object Detection
Road Object Detection
 

Más de NECST Lab @ Politecnico di Milano

Embedding based knowledge graph link prediction for drug repurposing
Embedding based knowledge graph link prediction for drug repurposingEmbedding based knowledge graph link prediction for drug repurposing
Embedding based knowledge graph link prediction for drug repurposing
NECST Lab @ Politecnico di Milano
 

Más de NECST Lab @ Politecnico di Milano (20)

Mesticheria Team - WiiReflex
Mesticheria Team - WiiReflexMesticheria Team - WiiReflex
Mesticheria Team - WiiReflex
 
Punto e virgola Team - Stressometro
Punto e virgola Team - StressometroPunto e virgola Team - Stressometro
Punto e virgola Team - Stressometro
 
BitIt Team - Stay.straight
BitIt Team - Stay.straight BitIt Team - Stay.straight
BitIt Team - Stay.straight
 
BabYodini Team - Talking Gloves
BabYodini Team - Talking GlovesBabYodini Team - Talking Gloves
BabYodini Team - Talking Gloves
 
printf("Nome Squadra"); Team - NeoTon
printf("Nome Squadra"); Team - NeoTonprintf("Nome Squadra"); Team - NeoTon
printf("Nome Squadra"); Team - NeoTon
 
BlackBoard Team - Motion Tracking Platform
BlackBoard Team - Motion Tracking PlatformBlackBoard Team - Motion Tracking Platform
BlackBoard Team - Motion Tracking Platform
 
#include<brain.h> Team - HomeBeatHome
#include<brain.h> Team - HomeBeatHome#include<brain.h> Team - HomeBeatHome
#include<brain.h> Team - HomeBeatHome
 
Flipflops Team - Wave U
Flipflops Team - Wave UFlipflops Team - Wave U
Flipflops Team - Wave U
 
Bug(atta) Team - Little Brother
Bug(atta) Team - Little BrotherBug(atta) Team - Little Brother
Bug(atta) Team - Little Brother
 
#NECSTCamp: come partecipare
#NECSTCamp: come partecipare#NECSTCamp: come partecipare
#NECSTCamp: come partecipare
 
NECSTCamp101@2020.10.1
NECSTCamp101@2020.10.1NECSTCamp101@2020.10.1
NECSTCamp101@2020.10.1
 
NECSTLab101 2020.2021
NECSTLab101 2020.2021NECSTLab101 2020.2021
NECSTLab101 2020.2021
 
TreeHouse, nourish your community
TreeHouse, nourish your communityTreeHouse, nourish your community
TreeHouse, nourish your community
 
TiReX: Tiled Regular eXpressionsmatching architecture
TiReX: Tiled Regular eXpressionsmatching architectureTiReX: Tiled Regular eXpressionsmatching architecture
TiReX: Tiled Regular eXpressionsmatching architecture
 
Embedding based knowledge graph link prediction for drug repurposing
Embedding based knowledge graph link prediction for drug repurposingEmbedding based knowledge graph link prediction for drug repurposing
Embedding based knowledge graph link prediction for drug repurposing
 
PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...
PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...
PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...
 
EMPhASIS - An EMbedded Public Attention Stress Identification System
 EMPhASIS - An EMbedded Public Attention Stress Identification System EMPhASIS - An EMbedded Public Attention Stress Identification System
EMPhASIS - An EMbedded Public Attention Stress Identification System
 
Luns - Automatic lungs segmentation through neural network
Luns - Automatic lungs segmentation through neural networkLuns - Automatic lungs segmentation through neural network
Luns - Automatic lungs segmentation through neural network
 
BlastFunction: How to combine Serverless and FPGAs
BlastFunction: How to combine Serverless and FPGAsBlastFunction: How to combine Serverless and FPGAs
BlastFunction: How to combine Serverless and FPGAs
 
Maeve - Fast genome analysis leveraging exact string matching
Maeve - Fast genome analysis leveraging exact string matchingMaeve - Fast genome analysis leveraging exact string matching
Maeve - Fast genome analysis leveraging exact string matching
 

Último

Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...
Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...
Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...
amitlee9823
 
➥🔝 7737669865 🔝▻ narsinghpur Call-girls in Women Seeking Men 🔝narsinghpur🔝 ...
➥🔝 7737669865 🔝▻ narsinghpur Call-girls in Women Seeking Men  🔝narsinghpur🔝  ...➥🔝 7737669865 🔝▻ narsinghpur Call-girls in Women Seeking Men  🔝narsinghpur🔝  ...
➥🔝 7737669865 🔝▻ narsinghpur Call-girls in Women Seeking Men 🔝narsinghpur🔝 ...
nirzagarg
 
Top Rated Call Girls Mumbai Central : 9920725232 We offer Beautiful and sexy ...
Top Rated Call Girls Mumbai Central : 9920725232 We offer Beautiful and sexy ...Top Rated Call Girls Mumbai Central : 9920725232 We offer Beautiful and sexy ...
Top Rated Call Girls Mumbai Central : 9920725232 We offer Beautiful and sexy ...
amitlee9823
 
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理
ezgenuh
 
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar
dollysharma2066
 
Vip Mumbai Call Girls Mumbai Call On 9920725232 With Body to body massage wit...
Vip Mumbai Call Girls Mumbai Call On 9920725232 With Body to body massage wit...Vip Mumbai Call Girls Mumbai Call On 9920725232 With Body to body massage wit...
Vip Mumbai Call Girls Mumbai Call On 9920725232 With Body to body massage wit...
amitlee9823
 
Majestic Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore Es...
Majestic Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore Es...Majestic Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore Es...
Majestic Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore Es...
amitlee9823
 
Call Girls In Kotla Mubarakpur Delhi ❤️8448577510 ⊹Best Escorts Service In 24...
Call Girls In Kotla Mubarakpur Delhi ❤️8448577510 ⊹Best Escorts Service In 24...Call Girls In Kotla Mubarakpur Delhi ❤️8448577510 ⊹Best Escorts Service In 24...
Call Girls In Kotla Mubarakpur Delhi ❤️8448577510 ⊹Best Escorts Service In 24...
lizamodels9
 
Bangalore Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore E...
Bangalore Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore E...Bangalore Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore E...
Bangalore Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore E...
amitlee9823
 
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
amitlee9823
 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN ABUDHABI,DUBAI MA...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN ABUDHABI,DUBAI MA...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN ABUDHABI,DUBAI MA...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN ABUDHABI,DUBAI MA...
Health
 
Call Girls Kadugodi Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Kadugodi Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...Call Girls Kadugodi Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Kadugodi Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
amitlee9823
 
Vip Mumbai Call Girls Navi Mumbai Call On 9920725232 With Body to body massag...
Vip Mumbai Call Girls Navi Mumbai Call On 9920725232 With Body to body massag...Vip Mumbai Call Girls Navi Mumbai Call On 9920725232 With Body to body massag...
Vip Mumbai Call Girls Navi Mumbai Call On 9920725232 With Body to body massag...
amitlee9823
 
Call Girls in Patel Nagar, Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Patel Nagar, Delhi 💯 Call Us 🔝9953056974 🔝 Escort ServiceCall Girls in Patel Nagar, Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Patel Nagar, Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 

Último (20)

Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...
Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...
Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...
 
➥🔝 7737669865 🔝▻ narsinghpur Call-girls in Women Seeking Men 🔝narsinghpur🔝 ...
➥🔝 7737669865 🔝▻ narsinghpur Call-girls in Women Seeking Men  🔝narsinghpur🔝  ...➥🔝 7737669865 🔝▻ narsinghpur Call-girls in Women Seeking Men  🔝narsinghpur🔝  ...
➥🔝 7737669865 🔝▻ narsinghpur Call-girls in Women Seeking Men 🔝narsinghpur🔝 ...
 
Top Rated Call Girls Mumbai Central : 9920725232 We offer Beautiful and sexy ...
Top Rated Call Girls Mumbai Central : 9920725232 We offer Beautiful and sexy ...Top Rated Call Girls Mumbai Central : 9920725232 We offer Beautiful and sexy ...
Top Rated Call Girls Mumbai Central : 9920725232 We offer Beautiful and sexy ...
 
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理
 
How To Fix Mercedes Benz Anti-Theft Protection Activation Issue
How To Fix Mercedes Benz Anti-Theft Protection Activation IssueHow To Fix Mercedes Benz Anti-Theft Protection Activation Issue
How To Fix Mercedes Benz Anti-Theft Protection Activation Issue
 
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar
 
Workplace-Hazards TLE EIM 10 QUARTER3 W2
Workplace-Hazards TLE EIM 10 QUARTER3 W2Workplace-Hazards TLE EIM 10 QUARTER3 W2
Workplace-Hazards TLE EIM 10 QUARTER3 W2
 
Stay Cool and Compliant: Know Your Window Tint Laws Before You Tint
Stay Cool and Compliant: Know Your Window Tint Laws Before You TintStay Cool and Compliant: Know Your Window Tint Laws Before You Tint
Stay Cool and Compliant: Know Your Window Tint Laws Before You Tint
 
Vip Mumbai Call Girls Mumbai Call On 9920725232 With Body to body massage wit...
Vip Mumbai Call Girls Mumbai Call On 9920725232 With Body to body massage wit...Vip Mumbai Call Girls Mumbai Call On 9920725232 With Body to body massage wit...
Vip Mumbai Call Girls Mumbai Call On 9920725232 With Body to body massage wit...
 
Majestic Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore Es...
Majestic Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore Es...Majestic Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore Es...
Majestic Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore Es...
 
Call Girls In Kotla Mubarakpur Delhi ❤️8448577510 ⊹Best Escorts Service In 24...
Call Girls In Kotla Mubarakpur Delhi ❤️8448577510 ⊹Best Escorts Service In 24...Call Girls In Kotla Mubarakpur Delhi ❤️8448577510 ⊹Best Escorts Service In 24...
Call Girls In Kotla Mubarakpur Delhi ❤️8448577510 ⊹Best Escorts Service In 24...
 
Bangalore Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore E...
Bangalore Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore E...Bangalore Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore E...
Bangalore Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore E...
 
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN ABUDHABI,DUBAI MA...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN ABUDHABI,DUBAI MA...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN ABUDHABI,DUBAI MA...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN ABUDHABI,DUBAI MA...
 
What Could Cause Your Subaru's Touch Screen To Stop Working
What Could Cause Your Subaru's Touch Screen To Stop WorkingWhat Could Cause Your Subaru's Touch Screen To Stop Working
What Could Cause Your Subaru's Touch Screen To Stop Working
 
Call Girls Kadugodi Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Kadugodi Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...Call Girls Kadugodi Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Kadugodi Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
 
Vip Mumbai Call Girls Navi Mumbai Call On 9920725232 With Body to body massag...
Vip Mumbai Call Girls Navi Mumbai Call On 9920725232 With Body to body massag...Vip Mumbai Call Girls Navi Mumbai Call On 9920725232 With Body to body massag...
Vip Mumbai Call Girls Navi Mumbai Call On 9920725232 With Body to body massag...
 
Why Does My Porsche Cayenne's Exhaust Sound So Loud
Why Does My Porsche Cayenne's Exhaust Sound So LoudWhy Does My Porsche Cayenne's Exhaust Sound So Loud
Why Does My Porsche Cayenne's Exhaust Sound So Loud
 
Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...
Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...
Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...
 
Call Girls in Patel Nagar, Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Patel Nagar, Delhi 💯 Call Us 🔝9953056974 🔝 Escort ServiceCall Girls in Patel Nagar, Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Patel Nagar, Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
 

Sensor fusion of LiDAR and Camera for real time object detection - talk version

  • 1. Sensor fusion for autonomous driving perception Bagni Fabio 228594@studenti.unimore .it San Francisco - 15 May 2019 1
  • 2. Fabio Bagni San Francisco - 15 May 2019 Autonomous driving perception Precision and reactivity Risks of accidents, traffic laws respect. 2
  • 3. Fabio Bagni San Francisco - 15 May 2019 Embedded platforms 3
  • 4. Fabio Bagni San Francisco - 15 May 2019 Goals Real time 3D detection ● High performances ○ required 10 Hz frequency ● High precision ● Real time on embedded platforms 4
  • 5. Fabio Bagni San Francisco - 15 May 2019 UNIMORE HiPeRT prototype 5
  • 6. Fabio Bagni San Francisco - 15 May 2019 Sensors ● LiDAR ○ 3D point cloud ● Cameras ○ images Heterogeneous outputs 6
  • 7. Fabio Bagni San Francisco - 15 May 2019 Sensor fusion What is sensor fusion: ● Matching of different sensors views Why sensor fusion: ● Heterogeneous information of environment 7
  • 8. Fabio Bagni San Francisco - 15 May 2019 Fields of view LiDAR: 360 degrees around the vehicle Camera: wide angle, disposed do cover 360 degrees. 8 Cameras LiDAR
  • 9. Fabio Bagni San Francisco - 15 May 2019 Deep learning approach Camera: ● Bird’s Eye View estimation ● Object detection LiDAR ● Bird’s Eye View calculation ● Object detection 9 Image Point cloud 3D detection 3D detection Fusion
  • 10. Fabio Bagni San Francisco - 15 May 2019 Deep learning approach time Image stitching for multiple camera fusion 10 ➔ Under10 Hz Notenoughresponsive 1. Ming Liang, Bin Yang, Shenlong Wang and Raquel Urtasun: Deep Continuous Fusion for Multi-Sensor3D ObjectDetection.In: ECCV (2018).
  • 11. Fabio Bagni San Francisco - 15 May 2019 Static alignment sensor fusion Alignment of LiDAR with each camera individually. 11 Calibration Preprocessing
  • 12. Fabio Bagni San Francisco - 15 May 2019 Features extraction Matching is made by a calibration with a perforated panel. Holes are detected by all sensors. Holes’ centers are used as features to be matched. 12
  • 13. Fabio Bagni San Francisco - 15 May 2019 Cylindrical projection Fields of view matching is made projecting sensors’ outputs on the surface of a cylinder. 13
  • 14. Fabio Bagni San Francisco - 15 May 2019 Projections complexity LiDAR cylindrical projection: Depthmap O(n) : one operation per point 14 Camera cylindrical projection O(n) : one operation per pixel PARALLELIZABLE
  • 15. Fabio Bagni San Francisco - 15 May 2019 Calibration results 15
  • 16. Fabio Bagni San Francisco - 15 May 2019 Colored points Alignment of each point with the corresponding pixel on the image. 16
  • 17. Fabio Bagni San Francisco - 15 May 2019 Object detection Neural network for object detection on camera frame Clustering of LiDAR points Bounding boxes matching 17
  • 18. Fabio Bagni San Francisco - 15 May 2019 Object detection results 18
  • 19. Fabio Bagni San Francisco - 15 May 2019 Object detection average time 19 LiDAR output : 10 Hz Time limit : 100 ms
  • 20. Thanks for your attention Fabio Bagni - 228594@studenti.unimore.it 20 Fabio Bagni San Francisco - 15 May 2019
  • 21. 21
  • 22. Fabio Bagni San Francisco - 15 May 2019 Homography Homography allow to translate camera projection plane on the LiDAR projection plane. This method make cylindrical surfaces match. 22