Imaging APPLICATIONS

1. IMAGING APPLICATIONS - AUTOMOTIVE
Technical Presentation

2. Environmental Conditions
• Brief Company Presentation
• Synthetic Environment Generation
• New Developments in Image Understanding
• Training andTesting theVision Systems
• Target Hardware
We develop custom Image Recognition systems for Aerospace and
defence applications. Using algorithms like Deep Convolutional Neural
Networks and Regional Convolutional Neural Networks.
Our algorithms for Target Recognition and Tracking are designed from
the beginning to be run on embedded systems. We target both GPU
and FPGA devices.
To Train and Validate our algorithms we developed a process to
generate photorealistic 3D environments.
Those 3D Environments are used to produce realistic video streams of
the targets in different environmental conditions (lighting, adverse
meteorological conditions, camouflage, point-of-view).
The same technology can be used to Train and Test Automotive Vision
Systems.
Meeting Agenda
We produce Highly-Realistic Virtual Environment to
Train and Test vision algorithms.
Click Here to see a demo video

3. COMPANY PRESENTATION

4. Addfor provides specialized IT services and scientific applications.
Main office: Turin
Full-Time Employees:14
Automotive Partner Companies: 2
Academic Collaboration Agreements: 5

5. About me:
CTO in Addfor
11 years in Mathworks (Senior Application Engineer 2000-2011):
Image Processing /Video Processing
Academic Programs

6. 10%
25%
30%
35%
Automotive
Aerospace
Energy
Other
REVENUES

8. SYNTHETIC ENVIRONMENT GENERATION

9. UNREAL ENGINE 4

11. Environmental Conditions
• Shadows
• Partial Occlusions (traffic, vegetation)
• Adverse meteorological conditions
• Road Signs positioning
• Different Road Sign shapes in different countries
System Conditions
• Vehicle speed
• Vibrations
• Sensor resolution and color response
• Headlights color and beam shape
• Dirty / Scratched lenses
We develop custom Image Recognition systems for Aerospace and
defence applications. Using algorithms like Deep Convolutional Neural
Networks and Regional Convolutional Neural Networks.
Our algorithms for Target Recognition and Tracking are designed from
the beginning to be run on embedded systems. We target both GPU
and FPGA devices.
To Train and Validate our algorithms we developed a process to
generate photorealistic 3D environments.
Those 3D Environments are used to produce realistic video streams of
the targets in different environmental conditions (lighting, adverse
meteorological conditions, camouflage, point-of-view).
The same technology can be used to Train and Test Automotive Vision
Systems.
3D Photorealistic Environments
for AutomotiveVision Systems
We produce Highly-Realistic Virtual Environment to
Train and Test vision algorithms.
Click Here to see a demo video

12. AUTOMATICTAGGING
Click Here to see a demo video

13. Optical system Simulation
• FOV
• Lens Flares
• Distorsions
CCD/CMOS Physical Simulation
• Sensor Resolution
• Photons Flux
• Dark Current
• Source Follower Noise
• AD Conversion
• Integral Linearity Error
• Quantization Noise
Optics and Sensors Simulation
We produce Highly-Realistic Virtual Environment to
Train and Test vision algorithms.

14. StereoVision Simulation
Adaptive Cruise control with Radar Simulation
Pedestrian Protection
Lane Departure Systems
Blind Spot Protection
High Beam Assistance
Camera positioning Simulations
Other possible testing environments:

15. ONBOARD CAMERACAMERA POSITIONING SIMULATIONS
Click Here to see a demo video

16. IMAGE UNDERSTANDING

17. Military Prototypes
• Visible and Infrared Wavelengths
• FPGA and GPUTargets
• Old approach: HOG+SVM / HOUGHTRANFORM
• New systems are based on:
• Aggregated Channel Features as Regional Proposal Method
• Finetuned AlexNet (CNN) as Main Detector
• SVM as classificator
Side Project (just for fun)
We are developing a Pedestrian Detection System that exceeds the
performances of:
JHosang, Omran, Benenson, Schiele.Taking a deeper look at pedestrians.
arXiv preprint arXiv:1501.05790, 2015
Some History about modern image detectors:
Viola&Jones Detector
This detector, proposed in 2001 by Paul Viola and Michael Jones, has been the
first object detection framework to provide competitive object detection rates
in real-time.
HOG+SVM
Introduced in 2005 by Navneet Dalal and Bill Triggs for the identification of
pedestrians in static images.
(Used for example in XYLON logiPDET)
ACF / LDCF
The Aggregated Channel Features detector is one of the most famous
detectors available at the state of the art.We use it as Regional Proposal Layer.
Alternatively we experiment with LDCF (Locally decorrelated Channel
Feature Detector).
CNN
Convolutional Neural Networks are a subclass of Deep Neural Networks.This
is the state of the Art today: we use it as Main Detector.
We develop Advanced Prototypes
of State of the Art Image Understanding Systems
The AlexNet Structure

19. Wy Deep Learning is a DisruptiveTechnology

20. The Caltech Dataset - 10h @ 30Hz video - 250,000 annotated images

21. Detector Demo
ACF
Aggregated Channel Features
AlexNet
Deep Convolutional NN
SVM
Support Vector Machine
Click Here to see a demo video

22. TARGET HARDWARE

23. Algorithm Development - GPUTarget
Target Hardware:
• NVIDIA JetsonTK1
• NVIDIA JetsonTX1
Technologies:
• CUDA
• Locally Decorrelated Channel Features (LDCF)
• Deep Convolutional Neural Networks (CNN)
• Regional Convolutional Neural Networks (RCNN)
Applications:
• Target Recognition (military application)
• TargetTracking (military application)
• Pedestrian detection
• Traffic Sign Recognition
• Vehicle Recognition andTracking
We develop custom image processing application using Deep
LearningTechnologies (DCNN and RCNN).
Those methods require big datasets to be trained, The training
datasets are provided by the customer.Alternativeli the customer
provide the technical specifications of the objects to be
recognized and we generate a synthetic dataset with 3D
modeling tools like Maya and Unreal Engine. Once the the
dataset is available the training of the systems is performed on
a GPU cluster.
The final algorithm is validated on an extensive
dataset and ported on a format suitable for
an embedded GPU processor.
When possible we prefer to use
NVIDIA target solutions like
the Jetson TK1 or the new
JetsonTX1.

24. Algorithm Development - FPGATarget
We are developing an easy-to-use Integrated Development Environment to easily and rapidly
develop and simulate a customized FPGA-based Convolutional Neural Network.
The rationale behind the idea of using an FPGA-based implementation for CNNs is mainly
related to power efficiency and cost concerns. As from the literature, the power efficiency
achieved by FPGA-based implementations of CNNs can only be enhanced with ASIC
solutions, however for low selling volumes (order of millions of units) the FPGA alternative is
more effective in terms of TCO, since NRE costs related to ASIC are stated around 2-3M.
If we consider a fixed area and power budget, CPU solutions are not able to meet the
required performance, while, on the other hand, the average utilization of GPU-based
implementation is about 40%, thus leading to wastage of power and area.
We allow a designer to define a Convolutional Neural Network (CNN) in terms of a
sequence of convolutional and fully connected layers, plus the dimensions of the input image
which will be classified by the network.
Out of this CNN model we generate
multiple targets; at the moment, one aimed at
CPUs and one aimed at FPGAs.
The first target is employed to test the overall network on a given
dataset; the latter, instead, is a streaming oriented high performance FPGA-
oriented hardware accelerator, both power efficient and with high throughput.
With respect to state of the art HLS tools we are able to mitigate the memory pressure of
CNN loads by automatically moving the computation type from iterative to data-flow.
Furthermore we are able to optimally exploit full or partial buffering of data with respect to
performance and resource requirements.
This allows, also thanks to the adoption of standard hardware interface such as AXI-Stream, to
generate a software/hardware system that can be easily integrated in a larger system.

25. IN DEVELOPMENT…

26. CNN on FPGA - User Interface:
We are working to develop a fully automatic software system to generate CNN directly in
FPGA.This system will be able to do the scaling and to allow the user to directly calculate the
tradeoff between Logic Gates and FPS.
The designer will define a Convolutional Neural Network (CNN) in terms of a sequence of
convolutional and fully connected layers, plus the dimensions of the input image which will be
classified by the network, as shown in Figure 1
Parameter selection:
Kernel height and width;
Number of feature maps both in input and in output
Hyperbolic tangent functions in the output layers
Max-pooling kernel

27. CONCLUSIONS

29. WE ARE ATTHE END
OF THE BEGINNING
(John Kelly SVP - Director of IBM Research)

30. There is a Global Effort to develop
COGNITIVE COMPUTING
IBM (IBM.N) said it will invest more
than $1 billion to establish a new
business unit for Watson
Reuters -Thu Jan 9, 2014 2:50am EST
"The biggest thing will be Artificial
Intelligence," Schmidt (Google CEO)
said at Oasis
Bloomberg - Mar 6, 2014 10:07 PM GMT+0100
China's top search engine Baidu Inc.
has hired Google Inc's former
Artificial Intelligence (AI) chief
Andrew Ng
Reuters - Fri May 16, 2014 4:58pm EDT

32. Addfor scientific applications - advantages:
Fast Development Cycle - Agile software development
Technology Assessments - Custom Algorithms + Libraries
Strong relationship with universities BUT SW Agnostics
Advanced (working) Prototypes
KnowledgeTransfer

33. Addfor s.r.l.
www.add-for.com
P.zza Solferino 7
Torino 10121 (TO) - Italy