"Embedded Vision Applications Lead Way for Processors in AI: A Market Analysis of Vision Processors," a Presentation from IHS Markit

Copyright © 2019 IHS Markit
Embedded Vision Applications
Lead Way for Processors in AI
A Market Analysis of Vision Processors
Presented for 2019
Embedded Vision Summit

Agenda
What is meant by Embedded Vision Leads the Way for
AI? <Thesis>
Introduction to Market Views of AI <Context>
• Scope of AI
• AI, machine learning, neural networks,
deep learning Definitions
• Neural networks and deep learning:
training vs. inference
• AI Market by Hierarchy
• AI Market Common Functional Categories
• Processor Strategies
• Traditional Microcomponents
• Graphics and Discrete AI Processors
• System-on-Chip (SoC) Field
Programmable Gate Arrays & cSoC
Processors
2
AI applications focused on Embedded Vision <Body>
• Automotive
• ADAS, Cockpit Cameras
• Consumer
• Game Systems, Drones, Home Automation
• Data Processing
• Cloud-based Training, Social Media &
Search Engines, Data Security
• Industrial
• Machine Vision, Security, Robotics, Medical
• Wired Communications
• Broadcast & Streaming, Network
Infrastructure, Threat Mitigation
• Wireless Communications
• Smartphones, Tablets, Wearables
Summary <Conclusions>
1
2
3
4

What is meant by:
“Embedded Vision Leads the Way for
Processors in AI”

Embedded Vision Leads the Way for Processors in AI
• Computer vision research emerged in the 1950s
to ‘mimic’ human vision.
• By 2015, Microsoft, Google, and Baidu had all
developed neural networking algorithms (running
on supercomputers) capable of recognizing
images on ImageNet with greater accuracy than
the typical human (<5% error rates).
4
Vertical vision-specific analytics spread into many markets employing deep learning algorithms.
Within the next year or two, virtually every processor supplier will offer a competitive platform for AI.
AI, led today by demand for vision applications, is the next great transformative technology.

Scope of AI

AI, machine learning, neural networks, deep
learning Definitions
6
Artificial intelligence (AI) refers to the body of
science that studies how to enable machines to
perform independent problem solving, inference,
learning, knowledge representation and
decision making
Machine learning is a set of algorithms that gives
machines the ability to automatically find and
learn patterns by feeding them with data without
explicit programming
Neural Network and Deep Learning refer to
computational models that try to emulate the
structure and workings of a human brain,
including training and inference
Traditional Compute
Instructions
Raw Data
Process Output
Machine
Learning
Desired
Output
Training
Data
Process Output
Update
Operation
Artificial
Intelligence
Machine
Learning
Neural
Networks
Deep Neural
Networks

Neural networks and deep learning:
Training vs. inference
7
Training
• In this phase, the network learns and increases
its accuracy by comparing its solution to
desired output, fed by enormous datasets. The
model is built with characteristics, the weight
of those characteristics and identifying
incorrect results to re-evaluate these features.
Inference
• In this phase, the network uses the trained
model to provide output based on comparing
new data to the previously trained
characterizations.
Training: Advanced
algorithms & deep learning
Inference: Not safety
critical, and as backup
Cloud Data Centers
Training: Low-to-high
algorithms
Inference: Process
orchestration
Industrial PC,
Gateway, Server, DC,
PLC
Training: Very limited to
certain devices and apps
Inference: Broadest range
of implementations
End-Point Devices

AI Market by Hierarchy
8
Latency
Processingperformance
Energyconsumption
Development requirements
for AI solutions
Privacyandsecurity
NetworkReliability
Billions of
Nodes
Thousands of
Nodes
Millions of
Nodes
CLOUD
Data
Centers
EDGE
End-Point
Devices
EDGE
Network
Computing
At the top of the pyramid, processing performance demands are high, but application nodes are few. Resources
to provide performance, privacy and security are costly, MPUs, GPUs and discrete custom ML processors are
common. At the edge, resources are limited, low-power and reduced latency can be critical. Resource optimized
SoCs are common solutions. Converting NN to scalar for MCUs is a strategy being explored.

AI Market Common Functional Categories
9
General Cloud-based Machine Learning
Perception and Vision Processing
Natural Language Processing
Autonomous Control

Processor Strategies

Microprocessor (MPU)
• High performance good for brute force processing
• On boards optimized for discrete cutting edge components
• Complex instruction sets targeting a wide range of applications
AI Target - training and cloud-based services
Digital Signal Processor (DSP) & Microcontroller (MCU)
• DSP with Harvard Architecture targets real-time applications
• MCU targets low-power low-cost control functions
AI Target – simple scalar ML for real-time data
acquisition and control
Early Microcomponent Trends
AI Tools & SW converting ML to Scalar
• Traditional processors like MPUs
and MCUs are ubiquitous. SW
tools are being developed to map
trained neural networks to
traditional scalar processing
elements. Examples include
• STM STM32Cube.AI
• Arm Helium
• TensorFlow Lite
Machine Learning as a Service
(MLaaS) - Examples include
• AmazonML
• Microsoft AzureML
• Google Prediction API
Traditional Micro component
You may be surprised to learn that the vast majority of
the AI functions are still processed on a standard MPU!

Graphics Processing Unit & General Purpose GPU
• The GPU market is mature with significant install base.
• Over half of all GPUs used for gaming and graphic design.
• GPGPU applications are growing faster than graphics.
GPUs can provide 100x performance improvement.
Discrete Machine Learning Coprocessor
• Discrete ML processors are an emerging market.
• Most notable are the ASICs developed by cloud services providers
such as the Tensor Processor developed by Google.
AI processor design startups are attracting capital
investment at unprecedented rates.
SIMD limitations
• Single instruction multiple
data (SIMD).
• SIMD processors described
here are almost exclusively
used as coprocessors.
• An OS kernel, an application
API, & human machine
interfaces are inherently
threaded to run linearly and
would run poorer, if at all, on
these SIMD structures.
• Most software is abstracted
and the machine code is
relegated to the best
processor for the task.
Graphics Processor (GPU) and Discrete AI Coprocessor
The transition from GPU to GPGPU and its boost to MPUs
alone has been a strong enabler for AI market growth.

SoCs
• Propelled by early demand for smartphones
• Accommodates space, power & heat dissipation, cost and other
constraints through integrating components and coprocessors
Come in many application-specific designs including
optimization for vision and artificial intelligence.
FPGAs
• Reputation as prototype solution, but market is much larger.
• Lack of VHDL or Verilog engineers can still be a limitation.
Configurability enables wide parallel processing
strategies well suited for AI.
Configurable SoCs
• Xilinx and Intel (Altera) have
developed configurable SoCs
following the heterogeneous
design trends of SoCs.
• cSoC use applications
processor cores identical to
SoCs, but integrate
configurable logic similar to
an FPGA.
• This strategy has adapted to
AI demand. Xilinx Adaptive
Compute Acceleration
Platform (ACAP) is an
example with integrated ML
acceleration.
System-on-Chip (SoC), Field Programmable Gate Array
(FPGA) & Configurable SoC
As Moore’s Law, based solely on miniaturization becomes
unsustainable, the processor market turns to integration.

AI Applications focused on
Embedded Vision

Automotive Electronics
Automotive Trends
• Automobile sales—<100m in
2018.
• Vehicle Electronic Control
Units—vehicle electronics cost
rising from $1200 in 2015 to
$1700 by 2023.
• Domain Controllers—
decentralized electronic
control units yielding to
centralized domain controllers
• Barriers to Entry—AI and
vision expertise lowering
barriers for new suppliers.
Cockpit
ADAS Training ITS Cloud
ITS & V2I
ADAS
Advanced Driver Assistance Systems (ADAS) & Cockpit Market
*Data Processing Market
*Wireless
Communications
Market

Automotive Electronics
Trends directly impacting Processors for AI
• ADAS dominates In-vehicle AI market short term.
• The primary ADAS AI platforms are SoCs with
graphics and other AI subsystems targeting
embedded vision.
• DCs will reduce ECUs, but AI performance will
grow revenue as SoC platforms & ADAS systems
become more sophisticated.
• Intelligence may still be distributed with near
camera imaging and DC control in the long term.
• Platforms from suppliers with significant software
& tools development are winning solutions.
• GPUs are currently exclusively in some Nvidia
platforms.
0
500
1,000
1,500
2018 2023
MPU MCU
DSP GPU
SoC PLD
Discrete ML Processor
Revenue$(M)
Annual Revenue per Processor Class
Advanced Driver Assistance Systems (ADAS) and Cockpit Market
AI Processor Market for Automotive
Source: IHS Markit

Consumer Electronics
Consumer Trends
• Highly competitive — Cost &
time-to-market are barriers for
growth of AI in Consumer
• Voice Analytics — Besides the
“wake word”, almost all voice
analytics for HA is processed
in the cloud. There is demand
for increased local analytics
for privacy and security.
• Region — The Asian markets
are growing significantly more
rapidly in Consumer
AI markets.
Game Consoles
Game & Home
Automation Hosts
Gateways &
HA Hubs
Cobots &
Drones
Game Consoles , Home Automation, and Consumer Robots & Drones
*Data Processing
Market
*Wired
Communications
Market

Consumer Electronics
• Most advanced consumer AI applications are
cloud-based; this is wholly true of training.
• The resources required for vision analytics and
natural voice speaking are costlier than are
typical of the consumer mass market.
• MCUs, DSPs and entry level SoCs are common
for consumer because of the constraints (cost,
area, power, time-to-market, etc.).
• For home automation, there is demand for
advanced gateways to improve the HMI, privacy
and security. The target SoCs include improved
graphics and even NN acceleration integrated.
• Low-cost AI specific discretes targeting edge
nodes are expected to emerge in consumer
applications.
AI Processor Market for Consumer
0
500
1,000
1,500
2,000
2,500
2018 2023
MPU MCU
DSP GPU
SoC PLD
Revenue$(M)
Game Consoles , Home Automation, and Consumer Robots & Drones
Source: IHS Markit

Data Processing
Data Processing Trends
• Training continues to be the
largest AI function for data
processing. This is likely to be
the case for a very long time.
• Despite limited CAPEX for data
center & IT, the training of AI is
driving growth in the overall
server market. More dramatic
than server demand, is the
increase in BoM. HPC and
servers with coprocessors for
running AI are increasing
penetration rapidly.
Social Media
& Search
Cloud Training
Financial ID
PC Facial ID
Cloud-based Training, Social Media & Search Engines, Data Security

Data Processing
• Despite marketing espousing the advantages of
specialized processors for AI, the vast majority of
processor revenue in 2018 was still on MPUs.
• The end of Moore’s Law is being felt first in the
cutting edge processor markets such as MPUs.
• It is only in the last 5-10 years that the total cost
of ownership in data centers including systems
and power for cooling has driven more interest in
performance/watt metrics favoring coprocessors.
• GPGPU and its synergy between embedded vision
applications and multimedia processing has
provided the leverage for significant recent
growth of GPUs in data processing.
• VLIW, TPUs and discrete ML processors will gain
rapidly, especially outside graphics.
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
2018 2023
MPU MCU
DSP GPU
SoC PLD
Revenue$(M)
AI Processor Market for Data Processing
Cloud-based Training, Social Media & Search Engines, Data Security
Notes: Primary Drivers are Training, Social Media & Search Engines, Data Security
Source: IHS Markit
Source: IHS Markit

Industrial Electronics
Industrial Trends
• Industrial markets are
complex, diverse and slow to
adopt to technology changes.
Time-to-market can take years
and equipment expected to
last decades.
• Some applications may require
working in harsh conditions
such as infrastructure, military
& aerospace, process
automation, environmental
sciences.Robots &
Drones
Security
Medical
Machine Vision
Machine Vision, Security, Robots and Drones, Medical

Industrial Electronics
• Industrial & Commercial applications are diverse
and the mix of processors for AI is just as diverse.
However, they still follow some patterns:
• MPUs & GPUs are used where cutting edge
performance, cost, power, and size are not
barriers.
• MCUs and DSPs are common, but too resource
constrained for most neural networking. Small
scale sensor hubs controlled by scalar-based
ML are an emerging market .
• SoCs will tend to be the typical edge node
solutions & FGPAs in edge networks
• Solutions based on unique constraints in
specific applications will be critical.
0
500
1,000
1,500
2,000
2018 2023
MPU MCU
DSP GPU
SoC PLD
Revenue$(M)
Machine Vision, Security, Robots and Drones, Medical
AI Processor Market for Industrial
Source: IHS Markit

Wired Communications
Wired Communication Trends
• Data center networks &
network security represent
coprocessor growth markets
for wired communications.
• Solutions for AI are at the
edge, on-premises for security
and privacy, and on the
service provider side for
network integrity & quality of
service. Intelligent gateways
are targets for incumbents and
emerging providers.
Threat
Mitigation
Broadcast
Services
Network
Infrastructure
Broadcast & Streaming, Network Infrastructure, Threat Mitigation

Wired Communications
• Outside of computers, network infrastructure and
security systems consume more MPUs than any
other market.
• Because, networking has had little association
with HMI, there is little demand for GPUs outside
of broadband streaming and other multimedia
services, Recently, the use of AI for network QoS
is driving additional demand for GPUs.
• FPGAs, due to their highly parallel logic, are
common solutions for packet-forwarding and AI
optimization for networking.
• SoCs are common, however, tend to be optimized
for packet forwarding. NextGen SoCs and discrete
coprocessors include a good deal more AI-
specific acceleration such as integrated TPUs.
0
500
1,000
2018 2023
MPU MCU
DSP GPU
SoC PLD
Revenue$(M)
Broadcast & Streaming, Network Infrastructure, Threat Mitigation
AI Processor Market for Wired Communications
Source: IHS Markit

Wireless Communications
Wireless Communication Trends
• The wireless market has
matured, replacement rates
and new users have slowed
dramatically.
• The wearables market
represents a transformative
technology to open new
market opportunities.
• 5G represents a way of
growing the connected device
market size, but AI represents
the greater opportunity for
transforming what is possible.
Smartphones
& Tablets
Wireless
Infrastructure
Wearables
Smartphones & Tablets, Wearables & Wireless Infrastructure

Wireless Communications
• While the concept of SoC has been around since
the MCU, it was the smartphone market that
really drove the development of SoCs for
applications processors into a $50 billion +
market.
• MPUs, GPUs and FPGAs are too resource
demanding for portable devices, but for wireless
infrastructure, the performance and parallel
processing is a valuable asset and the
constraints are much less limiting
• MCUs and DSPs are common in the wireless
market, but their effectiveness as an AI processor
is limited.
• Discrete ML processor development is expected
to target wireless applications heavily, but not
dislodge SoCs.
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
16,000
2018 2023
MPU MCU
DSP GPU
SoC PLD
Revenue$(M)
Smartphones & Tablets, Wearables & Wireless Infrastructure
AI Processor Market for Wireless Communications
Source: IHS Markit

Conclusion

Conclusion
Processors Market for AI Summary
• Processors used for AI applications are expected
to well over triple by 2023 topping $35 billion.
• In 2018, the largest processor revenue for AI was
from MPUs used to train AI applications, and that
will continue to grow. optimized servers with
GPUs, FPGAs, discrete ML accelerators and other
coprocessors will grow faster than the GP market.
• In 2023, the largest portion of processors
running inferencing will be SoCs in smartphones,
but with significant interest in SoCs for ADAS.
• Development in AI in some markets, such as
ADAS, is transforming the industry.
• Not all machine learning is neural networking.
There are strategies for translating the NN code
to run a scalar equivalent – MPUs & MCUs. These
scalars can still be used for ML.
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
2018 2023
MPU MCU
DSP GPU
SoC PLD
Revenue$(M)
Key Takeaways
Processor Market For AI Summary
Source: IHS Markit

Copyright © 2019 IHS Markit 29
Questions?
IHS Markit Customer Care
CustomerCare@ihsmarkit.com
Americas: +1 800 IHS CARE (+1 800 447 2273)
Europe, Middle East, and Africa: +44 (0) 1344 328 300
Asia and the Pacific Rim: +604 291 3600
Disclaimer
The information contained in this presentation is confidential. Any unauthorized use, disclosure, reproduction, or dissemination, in full or in part, in any media or by any means, without the prior written permission of IHS Markit Ltd.
or any of its affiliates ("IHS Markit") is strictly prohibited. IHS Markit owns all IHS Markit logos and trade names contained in this presentation that are subject to license. Opinions, statements, estimates, and projections in this
presentation (including other media) are solely those of the individual author(s) at the time of writing and do not necessarily reflect the opinions of IHS Markit. Neither IHS Markit nor the author(s) has any obligation to update this
presentation in the event that any content, opinion, statement, estimate, or projection (collectively, "information") changes or subsequently becomes inaccurate. IHS Markit makes no warranty, expressed or implied, as to the
accuracy, completeness, or timeliness of any information in this presentation, and shall not in any way be liable to any recipient for any inaccuracies or omissions. Without limiting the foregoing, IHS Markit shall have no liability
whatsoever to any recipient, whether in contract, in tort (including negligence), under warranty, under statute or otherwise, in respect of any loss or damage suffered by any recipient as a result of or in connection with any
information provided, or any course of action determined, by it or any third party, whether or not based on any information provided. The inclusion of a link to an external website by IHS Markit should not be understood to be an
endorsement of that website or the site's owners (or their products/services). IHS Markit is not responsible for either the content or output of external websites. Copyright © 2018, IHS MarkitTM. All rights reserved and all intellectual
property rights are retained by IHS Markit.

"Embedded Vision Applications Lead Way for Processors in AI: A Market Analysis of Vision Processors," a Presentation from IHS Markit

Recomendados

Recomendados

Más contenido relacionado

Más de Edge AI and Vision Alliance

Más de Edge AI and Vision Alliance (20)

Último

Último (20)

"Embedded Vision Applications Lead Way for Processors in AI: A Market Analysis of Vision Processors," a Presentation from IHS Markit