For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/texas-instruments/embedded-vision-training/videos/pages/may-2019-embedded-vision-summit For more information about embedded vision, please visit: http://www.embedded-vision.com Manisha Agrawal, Software Applications Engineer at Texas Instruments, presents the "Machine Learning at the Edge in Smart Factories Using TI Sitara Processors" tutorial at the May 2019 Embedded Vision Summit. Whether it’s called “Industry 4.0,” “industrial internet of things” (IIOT) or “smart factories,” a fundamental shift is underway in manufacturing: factories are becoming smarter. This is enabled by networks of connected devices forming systems that collect, monitor, exchange and analyze data. Machine learning, including deep neural network algorithms such as convolution neural networks (CNNs), are enabling smart robots and machines to autonomously complete tasks with precision, accuracy and speed. The Texas Instruments (TI) Sitara line of processors is helping to enable vision-based deep learning inference at the edge in factory automation products. TI’s AM57x class processors with specialized neural network accelerators and integrated industrial peripherals provide the processing and connectivity needed to enable smart factory vision applications to reduce production costs, improve quality and create safer work environments. This presentation covers TI’s deep learning solution on AM57x processors for smart factories.

1. © 2019 Texas Instruments Inc
Machine Learning at the Edge in
Smart Factories Using TI Sitara
Processors
Manisha Agrawal
Texas Instruments Incorporated
May 2019

2. © 2019 Texas Instruments Inc
Agenda
• Smart factory overview
• Sitara processors for smart factories
• Sitara machine learning overview
• Getting started with Sitara machine learning
• Third-party ecosystem
2

3. © 2019 Texas Instruments Inc
Smart factory overview

4. © 2019 Texas Instruments Inc
Industry 4.0 / IIOT / Smart Factory
4
Smart factory overview
Industry 4.0, the Industrial Internet of
Things (IIoT) or Smart Factory are all terms
describing a network of devices connected
via communications technologies to form
systems that monitor, collect, exchange
and analyze data.
Based on four design principles –
Interconnection: Machines, devices, sensors, and people to connect and communicate with each other via the
internet of things (IoT) or the Internet of People (IoP).
Information transparency: the systems create a virtual copy of the physical world through sensor data in order to
contextualize information
Technical assistance: Ability of the systems to support humans in making decisions and solving
problems and the ability to assist humans with tasks that are too difficult or unsafe for humans.
Decentralized decisions: The ability of cyber-physical systems to make simple decisions on their own and
become as autonomous as possible

5. © 2019 Texas Instruments Inc
Industry 4.0 / IIOT / Smart Factory
Key technology considerations for smart factories:
• Industrial communication protocols
• Real time control of cyber physical system
• Security and functional safety requirements
• Machine learning
• Enhanced reliability
• Device longevity
5
Smart factory overview
Industry 4.0, the Industrial Internet of
Things (IIoT) or Smart Factory are all terms
describing a network of devices connected
via communications technologies to form
systems that monitor, collect, exchange
and analyze data.

6. © 2019 Texas Instruments Inc
Sitara processors for smart
factories

7. © 2019 Texas Instruments Inc
Sitara processors enabling smart factories
7
• Industrial robots for automated sorting
• Vision computer & optical Inspection
• Automated Guided Vehicles (AGVs)
• Smart HMI (teaching operators)
• Optimize equipment settings
• Predictive Maintenance (PdM)
• Classify and recognize specific
sounds/audio patterns
• Track/identify/count people and objects
• Extracting useful information gathered
from the data aggregator
Sitara processors for smart factories

8. © 2019 Texas Instruments Inc
Sitara processors: Key differentiation for smart factories
• Support for multiple industrial communication
protocols on one platform
• Real-time control and processing with
incorporated PRU-ICSS and C66x DSP
• Accelerated machine learning and multimedia
capabilities with integrated hardware accelerators
• Advanced integration for enhanced functionality
• Enables machine learning inference at the edge
for many vision applications
• Predictive Maintenance at the edge at the field,
controller or operator level devices
• CSI-2 and parallel interface for camera
connection
• Support for PCIe, USB3, SATA, etc. connectivity
standards 8
Sitara processors for smart factories
Highly integrated processors supporting industrial peripherals and connectivity needed for
smart factories products
Highly integrated processors for smart
factory applications

9. © 2019 Texas Instruments Inc
Sitara Processors: Key differentiation for smart factories
• Enhanced Reliability
• Extended temperature & high voltage I/O
• Low Failure in Time (FIT)
• High POH (Power On Hours)
• Enabling lower system power
• Low Latency
• Scalable family of devices
• Software scalability across the entire portfolio
‒ Reuse software across platforms with TI’s processor SDK
• TI.com support
• Technical documentation, including reference designs, on machine learning with Sitara
processors
• Technical support by subject matter experts on e2e.ti.com
9
Sitara processors for smart factories

10. © 2019 Texas Instruments Inc
Sitara Arm processor overview
10
Sitara processors for smart factories Layout Compatible
AM43x
•1x Cortex-A9, 1 GHz
•Secure Boot, Tamper, QSPI
•Camera, 2x PRU-ICSS, GBE, 3D
•17x17mm
AM335x
•1x Cortex-A8, 1 GHz
•1x PRU-ICSS, 3D, GbE Switch
•DDR3, CAN
•13x13, 15x15mm
AM572x
• 2x Cortex- A15, 1.5 GHz
• 2x C66x DSP, 750 MHz
• 4x Machine Learning Accelerators
• 2x 3D, 4x M4, 2x PRU-ICSS
• 23x23mm HD
AM571x
• 1x Cortex-A15, 1.5 GHz
• 1x C66x DSP, 750 MHz
• 3D, 4x M4, 2x PRU-ICSS
• 23x23mm HD
AM574x
• 2x Cortex-A15, 1.5 GHz
• 2x C66x DSP, 750 MHz
• 2x Machine Learning Accelerators
• 2x 3D, 4x M4, 2x PRU-ICSS
• 23x23mm, ECC HD
AM570x
•1x Cortex-A15, 1.0 GHz
•1x C66x DSP, 750 MHz
•3D, 4x M4, 2x PRU-ICSS
•17x17mm HD
Performance&Integration
AM654x
• 4x A53, up to 1.1 GHz
• 2x R5F MCU/Safety Processor
• 32-bit DDR3/4, PCIe, USB3, 3D
• 3x PRU-ICSS (Gb) , ECC
• 23x23mm
• Highly integrated solution
• Brings Deep Learning to the edge
• Dedicated machine learning accelerators
(Embedded Vision Engine) on AM5729 and
AM5749 device for high performance and low
power
• AM5729 is the highest performing device
• Up to 120 GOPS

11. © 2019 Texas Instruments Inc
Sitara machine learning overview

12. © 2019 Texas Instruments Inc
Sitara machine learning
12
Sitara machine learning
Sitara machine learning solutions bring machine learning to the edge by enabling deep learning
inference on all Sitara devices.
Sitara machine learning
solutions consists of:
• TI Deep Learning (TIDL)
software framework
• Available on
AM57x device only,
runs on C66x DSP
cores and/or on
Embedded Vision
Engine (EVE)
subsystems.
• ArmNN: Runs on all
Sitara devices.

13. © 2019 Texas Instruments Inc
TI Deep Learning (TIDL) software framework on AM57x
13
Sitara machine learning
• TIDL software framework consists of:
– Converter tool to convert TensorFlow, Caffe and
ONNX formats to TIDL network format
– Quantize weight with 4-12 bit accuracy
– Dynamic quantization of activation layer outputs
with 8-bit accuracy
– Accumulation is done with 40-bit accuracy
– C++ based TIDL API on Arm/Linux
– TIDL Library to offload processing on C66x DSP
cores and EVE subsystem
• Ease of use
– Easily integrate TIDL APIs into other frameworks
such as OpenCV
– Provides a common host abstraction for user
applications across multiple compute engines
(EVEs and C66x DSPs)
TIDL software framework enables machine learning inference on Sitara AM57x processors by
leveraging integrated EVE and/or C66x DSP hardware accelerators to run CNN models specifically.
Format Conversion
(desktop / Arm Linux)
Inference – embedded
deployment
(Sitara AM57x)
TIDL-InferenceTIDL-translator
TIDL Import
TIDL API
OpenCL
TIDL Library

14. © 2019 Texas Instruments Inc
TI Deep Learning (TIDL) software framework
• Provides several TI network models:
• JacintoNet11 for image classification
• JDetNet for classification and detection
• JSeg21 for pixel level semantic segmentation
• TI network models leverages the following for embedded performance optimization:
• Complexity reductions
• Sparse convolutions
• fixed point dynamic quantization
• These tools result in minor decrease in accuracy but result in major computational and
bandwidth reductions
14
Sitara machine learning
Classification
+ Localization
Object detection +
classification
Semantic
Segmentation

15. © 2019 Texas Instruments Inc
Performance of validated network models on AM57x
15
Sitara machine learning
Network
Model
Type of
Model
ROI
size
Performance in ROIs/Sec
AM5729
(2x Arm A15 @1.5GHz, 4x
EVE @650MHz,
2x C66x DSP @750MHz)
AM5749
(2x Arm A15 @1.5GHz, 2x
EVE @650MHz,
2x C66x DSP @750MHz)
AM572x/AM574x
(2x Arm A15@1.5GHz,
2x C66x DSP @750MHz)
MobileNet_v1 Image
Classification
224x224 7.3 5.4 2.6 1.6
SqueezeNet_v1 Image
Classification
227x227 19.7 9.6 4.6 5.2
InceptionNet_v1 Image
Classification
224x224 6.3 3.2 1.4 1.6
JacinctoNet11 Embedded
Image
Classification
224x224 64.6 33.4 16.6 NA
JSeg21 Embedded
Semantic
Segmentation
1024x512 11.8 3.4 1.8 NA
JDetNet Embedded
Object detection
768x320 22.6 11.2 NA NA
*A15 is available for a real-time control loop or
communication protocol or run additional NN models
Utilizing 4xEVE+1xDSP* Utilizing 2xEVE+1xDSP* Utilizing DSP* Utilizing ARM**
• Power efficient embedded solution at the edge, < 5W
**Arm benchmarking done using Arm NN.

16. © 2019 Texas Instruments Inc
Getting started with
Sitara machine learning

17. © 2019 Texas Instruments Inc
beagleboard.zorg/ai
Getting started with Sitara machine learning
Features: 4 EVEs with built-in AM5729
processor in addition to the following
development support:
⚫ Zero-download training workshops
⚫ Tool for everyday automation
⚫ Definitive AM5279 reference design
17
Available from 3Q, 2019
BeagleBone AI (AM5729 SoC)TMDSIDK574 AM574x Industrial Development
Kit (IDK) Board
Features: 2 EVEs, 4 Ethernet ports with concurrent
operation including 2 from PRU-ICSS DDR3 with ECC
Profibus connection, EtherCAT and RS485 Headers On-
board eMMC Mini PCIe, USB3, and HDMI connectors
http://www.ti.com/tool/SITARA-MACHINE-LEARNING

18. © 2019 Texas Instruments Inc
Third-party ecosystem

19. © 2019 Texas Instruments Inc
Third-party ecosystem
• Cross Compass: Cross Compass has a strong
reputation in the manufacturing industry, provide
AI consulting services, and develops and provides
algorithms and neural networks.
• Path Partner: Provides services for development,
optimization and integration of AI solutions on TI
platforms
• D3 Engineering: Provides engineering services,
software development and platforms on AI
solutions with TI technology.
• Ittiam Systems: Provides services for high-
performance computer vision, machine learning
and analytics
19
www.cross-compass.com
www.d3engineering.com
www.pathpartnertech.com
www.ittiam.com

20. © 2019 Texas Instruments Inc
Resources
Sitara Arm processor overview
http://www.ti.com/processors/sitara
-arm/overview.html
Sitara machine learning
http://www.ti.com/tool/SITARA-
MACHINE-LEARNING
Software development Kit
http://software-dl.ti.com/processor-sdk-
linux/esd/AM57X/latest/index_FDS.html
TI Design on TI Deep Learning
http://www.ti.com/tool/TIDEP-01004
TI Deep Learning document
http://software-dl.ti.com/processor-
sdk-
linux/esd/docs/latest/linux/Foundati
onal_Components_TIDL.html
Embedded Vision Summit
“Sitara Machine learning for smart factories”
(Wednesday, May 22 from 11:20 – 11:50 am)
20

21. © 2019 Texas Instruments Inc
Backup

22. © 2019 Texas Instruments Inc
Sitara processors enable machine learning inference at
the edge
22
Data transmission
cost
Network latency
Security
Reliability
System power
Network Bandwidth
Network connectivity
Data uploaded to
cloud
Prediction result
returned
Machine learning inferenceSensor data
Action
The Cloud
Inference on the cloud
Sitara machine learning addresses
following concerns from cloud processing
ActionSensor data Machine learning inference
Local hardware
Inference at the edge

23. © 2019 Texas Instruments Inc
TIDL software framework reference design
TIDEP-01004 Design: available now
• Highly integrated solution bringing deep learning
inference to the edge.
• Guide through dense and sparse model training,
importing and deploying the model for inference run
on AM57x SoC
• API to run multiple (same or different) networks in
parallel on different EVE subsystem and DSP cores.
• Example trained networks ready to run
Benefits
• TIDEP-01004
• Design Guide
• Design Files: Schematics, BOM, Gerbers,
Software, etc.
• Device Datasheets:
‒ AM5749 IDK
‒ AM5749 SoC
• This TI Design demonstrates TI Deep Learning (TIDL)
inference running on an AM5749 IDK.
• Shows running TIDL inference both on the EVE subsystems
and on the C66x DSP cores
• Includes benchmarks of several popular networks on AM57x
• Walk-through of deep learning development flow including
training for sparse network, import and inference deployment
Features
• Machine Vision: Vision Computer, Code Readers
• Automated Machinery: Automated Sorting Equipment,
Optical Inspection
• Logistics Robots: Logistics Robots CPU board
• EPOS: ATMs, Currency Counter
• Many others…
Applications
Tools & Resources