From algorithms included in the image processing pipeline to neural networks running in vision processors, focus on the evolution of hardware in vision systems and how software disrupts this domain.
SOFTWARE IN VISION SYSTEMS
Vision systems are becoming increasingly important. Therefore, this report shows and explains the close links between embedded software and hardware in vision systems at the technology and market levels. What are the software technologies? How do they impact the hardware? Which hardware is impacted? What kinds of markets are affected? And how will they evolve?
We can consider software in vision systems as having two different levels. The first is very close to the hardware, inscribed inside standalone field programmable gate array (FPGA) or application specific integrated circuit (ASIC) chips, or integrated into more complicated architectures. This layer, not often considered, is the most important step in any image treatment after image acquisition by pixels. Image processing, realized in the image signal processor (ISP), has a quite simple function. It must transform a signal from the sensor to an understandable image for the human eye. It is composed as a pipeline of multiple blocks, where each block takes as input the output of the previous block. A lot of different algorithms are implemented to accomplish tasks such as removing artefacts, color correction and reproduction. This is done at a single-pixel or pixel-group level and does not need a lot of memory or power.
More information on that report at http://www.i-micronews.com/reports.html
16. Vision systems are becoming increasingly
important. Therefore, this report shows and
explains the close links between embedded
software and hardware in vision systems at the
technology and market levels. What are the
software technologies? How do they impact
the hardware? Which hardware is impacted?
What kinds of markets are affected? And how
will they evolve?
We can consider software in vision systems
as having two different levels. The first is
very close to the hardware, inscribed inside
standalone field programmable gate array
(FPGA) or application specific integrated
circuit (ASIC) chips, or integrated into more
complicated architectures. This layer, not
often considered, is the most important step
in any image treatment after image acquisition
by pixels. Image processing, realized in the
image signal processor (ISP), has a quite
simple function. It must transform a signal
from the sensor to an understandable image
for the human eye. It is composed as a pipeline
of multiple blocks, where each block takes as
input the output of the previous block. A lot
of different algorithms are implemented to
accomplish tasks such as removing artefacts,
color correction and reproduction. This is
done at a single-pixel or pixel-group level and
does not need a lot of memory or power.
The second software layer is completely
different, with much more diverse and
complicated functions. In this report, we
focused on embedded software and, more
precisely, inference software derived from
the latest artificial intelligence (AI) methods.
This kind of technology necessitates a lot
of memory and computing power. It uses
complete image frames as input, and its goal is
not to correct, but to analyze and understand
the world in the picture. In vision systems,
AI technology focuses on detection of eyes,
faces, traffic signs, pedestrians, lanes, objects
in front of cars and free space, and recognition
of faces, irises, behaviors and gestures based
on a mathematical technique called a neural
network. This report especially investigates
one of the most famous technologies, which
has given spectacular results in recent years:
deep learning.
FROM IMAGE PROCESSING TO DEEP LEARNING,
INTRODUCTION TO HARDWARE AND SOFTWARE
Market & Technology report - November 2017
SOFTWARE INVISION SYSTEMS
From algorithms included in the image processing pipeline to neural networks running in vision
processors, focus on the evolution of hardware in vision systems and how software disrupts this
domain.
KEY FEATURES OF THE REPORT
Get the sample of the report
on www.i-Micronews.com
• Image processing technology
description
• Description of Artificial Intelligence
(AI) technology in vision systems
• Intellectual property, hardware
and business models for the
image signal processor and vision
processor market
• AI technology and business models
in the vision system ecosystem
• Technology trends and future
outlook
(Yole Développement, November 2017)
Artificial intelligence technologies in vision systems
Computational
photography
Machine visionAutomotive
Human machine
interface
Dual
cameras Video
stabilization
HDR
Super
resolution
Digital
zoom
Deep
learning
AR/VR
SLAM
Video
analytics
3D
reconstruction
Forward
collision
warning
Lane
departure
Pedestrian
detection
Traffic sign
detection
Free space
Gesture
recognition
Behavior
recognition
Iris
recognition
Face
recognition
Face
detection
OBJECTIVES OF THE REPORT
Understand the technologies
• Image processing pipeline
• AI technologies based on neural
networks
Understand the evolution of
hardware
• Business models
• Image signal processor to vision
processors to sensor fusion
• Market share and evolution
Understand how software is
disrupting hardware and future trends
The shaded technologies are addressed in this report
17. FROM IMAGE PROCESSING TO DEEP LEARNING, INTRODUCTION TO HARDWARE AND SOFTWARE
HARDWARE MARKET AND BUSINESS MODELS
AI has completely disrupted hardware in vision
systems, and has had an impact on entire
segments, like Mobileye has in automotive, for
example. Image analysis adds a lot of value and
image sensor builders are therefore increasingly
interested in integrating a software layer to
their system in order to capture it. Today, image
sensors must go beyond taking images – they
must be able to analyze them.
However, to run these types of software, high
power computing and memory are necessary,
which led to the creation and development of
vision processors. The image signal processor
(ISP) market offers a steady compound annual
growth rate (CAGR) of 6.3%, making the total
market worth $4,400M in 2017. Meanwhile,
the vision processor market is exploding, with
a 30.7% CAGR and a market worth $653M in
2017!
Today, optimization requires software and
hardware to be developed in parallel. Depending
on the issues and specifications, companies can
invest more in hardware than software or vice
versa. However, software is easier to specify,
tune and update, and so its growth is more
important than hardware. The AI market is
therefore expected to reach $35B in 2025 with
an estimated CAGR at 50% per year from 2017-
2025.
This report carefully evaluates ISP and vision
processor market shares and their evolution
in order to correctly understand how AI
technology impacts the hardware. This market
has been divided in two different business
models: Intellectual Property (IP) companies,
which don’t have physical products, and
hardware companies, which sell the processors
physically. The leaders are pretty easy to identify
for each category. ARM and Synopsys lead the
IP segment and Omnivision, Mobileye and ON
Semiconductor lead the hardware segment.
The AI market, particularly in vision systems, is
new and still moving, with hundreds of startups
created each year. It has no clear leaders but a
lot of highly specialized companies. This report
therefore gives a high-level view of driving
forces, technology hype, and the most important
mergers and acquisitions.
The main goal of this report is to understand what
is happening with the emergence of AI. Even if it
is not a new technology, thanks to technological
factors AI has made a spectacular entry into vision
systems. It opens new perspectives in various
segments such as automotive, surveillance,
biometrics and medical. However, it also poses
ethical questions, which we have tried to answer.
AI technologies promise a bright future in
many areas, with rapid software and hardware
progress. In autonomous vehicles, AI allows cars
to understand the world around them, predict
trajectories, communicate and drive. This has led
to the development of sensor fusion boards. For
example, NVidia’s Drive PX boards provide very
high performance computing and memory, giving
the ability to compile information from many
completely different sensors. In surveillance and
security, face and iris recognition have never
been as powerful, entering the consumer world
through the iPhone X this year, and behavior
recognition is on track.
From image signal processor to sensor fusion processor
Hardware markets size evolution 2012-2021
(Yole Développement, November 2017)
(Yole Développement, November 2017)
Vision processor from Mobileye
Frame
processing
Vision processor
Amount of
data processed
Fully AI to reach
autonomy
Introduction of AI
algorithms
Is part of
Price
per
unit
$1000
$10
$1
Set of pixels
processing
ISP
Image processing
algorithms
Standalone ISP from Altek
Frame processing
+ other sensors
Sensor fusion processor
Sensor fusion processor
from NVidia
$
sion processorr
Sensor fusion processor
from NVidia
WHEN SOFTWARE DISRUPTS HARDWARE
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
2012 2013 2014 2015 2016 2017 2018 2019 2020 2021
Revenues(US$million)
Intellectual Property - Image signal processor Intellectual Property -Vision processor
Hardware -Vision processor Hardware - Image signal processor
18. MARKET TECHNOLOGY REPORT
Emergence of artificial intelligence - Technological factors
(Yole Développement, November 2017)
Find more
details about
this report here:
COMPANIES CITED IN THE REPORT (non exhaustive list)
Algolux, Amazon, Altek, Ambarella, AMD, ARM, Apple, B-Secur, Biowatch, CEVA, Canon,
ChipsMedia, Cognitec, Delta ID, Ergoneers, EyeLink, Eyesmart Technology, Eyetech,
Face++, Facebook, Fotonation, Google, IBM, Intel, Imagination, Irisys, Iritech, ISCAN, Kairos,
LC Technologies, LG, Microsoft, Mobbeel, Mobileye, Movidius, Nec, Nextchip, Nvidia, NXP,
Omnivision, ON Semiconductor, OpenCV, Parrot, Pixelworks, Princeton Identity, Qualcomm,
SensorMotoric Instruments, Smart Eye, Socionext, Sony, Sthaler, STMicroelectronics,
Sunplus, Synopsys, The Eye Tribe, Tobii, Xperi...And more.
Report objectives and methodology 5
Introduction12
Software
Embedded software
Image processing 23
Definition
Basics : Primary ISP architecture for
Bayer image sensor
Image processing to recover artefacts
Image processing for color reproduction
Image processing with pre-/post-
processing
Conclusion
Artificial intelligence 41
Definition
A modern history of AI
AI technologies
Discussion
Players and market dynamics 59
ISP and vision processor
AI in vision systems
Conclusions83
Company profile 90
TABLE OF CONTENTS (complete content on i-Micronews.com)
• 3D TSV and 2.5D Business Update - Market
and Technology Trends 2017
• 3D Imaging and Sensing 2017
• Status of the CMOS Image Sensor Industry
2017
RELATED REPORTS
Benefit from our Bundle Annual Subscription offers and access our analyses at the best available
price and with great advantages
AUTHOR
Dr. Yohann Tschudi as a Software
Market Analyst, Dr. Yohann Tschudi is a
member of the MEMS Sensors business
unit at Yole Développement (Yole). Yo-
hann is daily working with Yole’s analysts
to identify, understand and analyze the
role of the software parts within any
semiconductor products, from the ma-
chine code to the highest level of algo-
rithms. Market segments especially ana-
lyzed by Yohann include big data analysis
algorithms, deep/machine learning, ge-
netic algorithms, all coming from Artifi-
cial Intelligence (IA) technologies.
After his thesis at CERN (Geneva, Swit-
zerland) in particle physics, Yohann de-
veloped a dedicated software for fluid
mechanics and thermodynamics appli-
cations. Afterwards, he served during
2 years at the University of Miami (FL,
United-States) as a research scientist in
the radiation oncology department. He
was involved in cancer auto-detection
and characterization projects using AI
methods based on images from Magne-
tic Resonance Imaging (MRI). During his
research career, Yohann has authored
and co-authored more than 10 relevant
papers.
Yohann has a PhD in High Energy Physics
and a master degree in Physical Sciences
from Claude Bernard University (Lyon,
France).
Find all our reports on www.i-micronews.com
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
2008 2010 2012 2014 2016 2018 2020
Amount of data stored
$0,01
$1
$100
$10 000
$1 000 000
$100 000 000
$10 000 000 000
$1 000 000 000 000
1950 1960 1970 1980 1990 2000 2010 2020
IBM 7030
Stretch
Cray X-MP/48
Beowulf
cluster
KASY0
Approximate cost per GFLOPS
(109
operations per second)
(Exabytes – 1018
bytes)
Estimated
Al is very exciting for the entire area of vision systems. This report tries to show why it is
important to understand the technologies and their impacts, and how to react. AI is in vision
systems, from technology to market.
19. ORDER FORM
From image processing to deep learning,
introduction to hardware and software
SHIPPING CONTACT
First Name:
Email:
Last Name:
Phone:
PAYMENT
BY CREDIT CARD
Visa Mastercard Amex
Name of the Card Holder:
Credit Card Number:
Card Verification
Value (3 digits except AMEX: 4 digits):
Expiration date:
BY BANK TRANSFER
BANK INFO: HSBC, 1 place de la Bourse,
F-69002 Lyon, France,
Bank code: 30056, Branch code: 00170
Account No: 0170 200 1565 87,
SWIFT or BIC code: CCFRFRPP,
IBAN: FR76 3005 6001 7001 7020 0156 587
RETURN ORDER BY
• FAX: +33 (0)472 83 01 83
• MAIL: YOLE DÉVELOPPEMENT, Le Quartz,
75 Cours Emile Zola, 69100 Villeurbanne/Lyon - France
SALES CONTACTS
• North America - Steve Laferriere: +13106 008 267
laferriere@yole.fr
• Europe RoW - Lizzie Levenez: + 49 15 123 544 182
levenez@yole.fr
• Japan Rest of Asia - Takashi Onozawa: +81 3 6869 6970
onozawa@yole.fr
• Greater China - Mavis Wang: +886 979 336 809
wang@yole.fr
• Specific inquiries: +33 472 830 180 – info@yole.fr
(1)
Our Terms and Conditions of Sale are available at
www.yole.fr/Terms_and_Conditions_of_Sale.aspx
The present document is valid 24 months after its publishing date:
November 15, 2017
/
ABOUT YOLE DEVELOPPEMENT
BILL TO
Name (Mr/Ms/Dr/Pr):
Job Title:
Company:
Address:
City:
State:
Postcode/Zip:
Country*:
*VAT ID Number for EU members:
Tel:
Email:
Date:
PRODUCT ORDER - REF.YDMS17049
Please enter my order for above named report:
One user license*: Euro 5,490
Multi user license: Euro 6,490
- The report will be ready for delivery from December 8, 2017
- For price in dollars, please use the day’s exchange rate. All reports are
delivered electronically at payment reception. For French customers,
add 20% for VAT
I hereby accept Yole Développement’s Terms and Conditions of Sale(1)
Signature:
*One user license means only one person at the company can use the report.
Founded in 1998, Yole Développement has grown to become a group of companies providing marketing, technology and strategy consulting, media and
corporate finance services, reverse engineering and reverse costing services and well as IP and patent analysis. With a strong focus on emerging applications
using silicon and/or micro manufacturing, the Yole group of companies has expanded to include more than 80 collaborators worldwide covering MEMS and
image sensors, Compound Semiconductors, RF Electronics, Solid-state lighting, Displays, software, Optoelectronics, Microfluidics Medical, Advanced
Packaging, Manufacturing, Nanomaterials, Power Electronics and Batteries Energy Management.
The “More than Moore” market research, technology and strategy consulting company Yole Développement, along with its partners System Plus Consulting,
PISEO and KnowMade, support industrial companies, investors and RD organizations worldwide to help them understand markets and follow technology
trends to grow their business.
MEDIA EVENTS
• i-Micronews.com website and related @Micronews e-newsletter
• Communication webcast services
• Events: TechDays, forums,…
More information on www.i-micronews.com
CONTACTS
For more information about :
• Consulting Financial Services: Jean-Christophe Eloy (eloy@yole.fr)
• Reports: David Jourdan (jourdan@yole.fr) Yole Group of Companies
• Press Relations Corporate Communication: Sandrine Leroy (leroy@yole.fr)
CONSULTING AND ANALYSIS
• Market data research, marketing analysis
• Technology analysis
• Strategy consulting
• Reverse engineering costing
• Patent analysis
• Design and characterization of innovative optical systems
• Financial services (due diligence, MA with our partner)
More information on www.yole.fr
REPORTS
• Market technology reports
• Patent investigation and patent infringement risk analysis
• Teardowns reverse costing analysis
• Cost simulation tool
More information on www.i-micronews.com/reports