Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

"MediaTek’s Approach for Edge Intelligence," a Presentation from MediaTek

26 visualizaciones

Publicado el

For the full video of this presentation, please visit:

For more information about embedded vision, please visit:

Bing Yu, Senior Technical Manager and Architect at MediaTek, presents the "MediaTek’s Approach for Edge Intelligence" tutorial at the May 2019 Embedded Vision Summit.

MediaTek has incorporated an AI processing unit (APU) alongside the traditional CPU and GPU in its SoC designs for the next wave of smart client devices (smartphones, cameras, appliances, cars, etc.). Edge applications can harness the CPU, GPU and APU together to achieve significantly higher performance with excellent efficiency.

In this talk, Yu presents MediaTek’s AI-enabled SoCs for smart client devices. He examines the features of the AI accelerator, which is the core building block of the APU. He also describes the accompanying toolkit, called NeuroPilot, which enables app developers to conveniently implement inference models using industry-standard frameworks.

Publicado en: Tecnología
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

"MediaTek’s Approach for Edge Intelligence," a Presentation from MediaTek

  1. 1. © 2019 MediaTek MediaTek’s Approach for Edge Intelligence Bing Yu MediaTek May 22, 2019
  2. 2. © 2019 MediaTek Summary • MediaTek’s Product Position • MediaTek’s Approach (P90) • MediaTek AI Accelerator’s Features and Architecture • NeuroPilot & Platform-aware MLKits
  3. 3. © 2019 MediaTek MediaTek’s Product Position
  4. 4. © 2019 MediaTek Leading Market Positioning with cross-platform synergies Source (ranking by 2018 market share): Strategy Analytics, Gartner, IDC, IC Insight, iSuppli and MediaTek company data. *Combined with MStar. Digital TV* Feature Phone Android Tablet Optical Drive & BD Player Voice Assistant DeviceSmartphone Connectivity Network #1 #1#1 #1#1 #2 #1 Growth HarvestMobile Computing
  5. 5. © 2019 MediaTek Devices around us are getting smarter everyday. Smartphone Digital TV Automotive AP / Router VAD IoT Performance / Power Balance Heterogeneous Computing Various Processing Units in SoC Booming Intelligent Devices
  6. 6. © 2019 MediaTek Customer Product Software & System Connectivity Modem SoC Design Compute RF & Analog Multimedia Technology / IP Core Technologies Enable Intelligent Devices with Leading Customers
  7. 7. © 2019 MediaTek DNN Inference is Moving to the Edge • The inherent limitations of the cloud are difficult to improve • Rapid improvement in DNN efficiency is enabling more edge AI applications • Cloud-Edge collaborative model will provide the best user experience ⎼Most training will remain in the cloud Thousands Billions CLOUD Data Centers EDGE Devices Computing Resource Memory Capacity Thermal Budget Network Latency Availability Privacy Energy Efficiency Cloud Edge
  8. 8. © 2019 MediaTek MediaTek’s Approach
  9. 9. © 2019 MediaTek MediaTek Helio P90 SoC and System 12 GB/s1.1 TMACs 5 W Modem ISPAPU Wi-Fi Bluetooth GPUCPU Sensors Codec GPS 45 °C No fan 12 nm APU (VPU + AIA)
  10. 10. © 2019 MediaTek ETH Zürich AI Benchmark 3.0 (latest release@Mar 27) AI Score MediaTek Helio P90 19496 2nd Place 18924 QUANT Score QUANT Accuracy FP16 Score FP16 Accuracy MediaTek Helio P90 6212 98 9910 95 2nd Place 3695 55 7361 37
  11. 11. © 2019 MediaTek Network Quantization and Pruning • Network Quantization • DNNs are generally represented in floating point 32-bit (FP32) format • Human can do image classification without FP32 precision • Quantizing FP32 to INT8 can reduce the complexity with negligible accuracy loss • Recent research shows promising results with lower bit precision, such as 4/2/1 bit • Network Pruning • DNN architectures have many redundant weights to help the model to converge faster during the training process. • Unimportant weights can be removed to increase weight matrix sparsity • Importance assessment of weights is the key Accuracy Energy DRAM Pruning + Qantization Int8 Within 1% ~93% reduction 80~90% reduction
  12. 12. © 2019 MediaTek AIA Features and Architecture
  13. 13. © 2019 MediaTek AIA Key Features ▪ Bandwidth reduction techniques - TCM for data-exchange - Sparsity compression ▪ High Performance Engine AIA: 806 GMAC/s AIAx2: 1.6 TMAC/s @788MHz ▪ Flexible quantization scheme - Asymmetric or symmetric quant. - No extra performance overhead ▪ Power Efficient >1 TMACs/W (2x better than VPU) @12FFC ▪ Bandwidth-Aware Design ▪ Dual AXI Port for high BW ▪ High Throughput Load/Store ▪ Simultaneous execution of OPs (CONV/ACT/POOL) ▪ Support INT8/INT16/FP16
  14. 14. © 2019 MediaTek AIA Architecture • DNN performance and efficiency driven architecture design ⎼Acceleration HW for operations used intensively (CONV, Pooling, ReLu, etc.) ⎼Specialized scheduler to maximize PE utilization rate Pooling … … ElementwiseConvolution Activation Convolution Buffer PE PE PE PE APU Data Flow Controller and Scheduler
  15. 15. © 2019 MediaTek PE PE PE PE PE PE PE PE Convolution Engine • 16 GCUs, 32 CUs per GCU, each CU has dual-MAC units • INT8: 16x32x2 = 1024 MAC/cycle • INT16:16x32= 512 MAC/cycle • FP16:16x16 = 256 MAC/cycle • Dual-MAC unit provides: two 8-bit MAC or one 16-bit MAC • FP16 has a separate design not shown in the diagram. 16 GCUs PE PE PE PE PE PE PE PE
  16. 16. © 2019 MediaTek (Sequential) (Simultaneous execution of conv., Relu and pooling) Convolution ReLu Pooling ReLu Pooling Convolution Simultaneous Execution • Engines working in a pipelined fashion Different engines work in parallel to deliver high throughput.
  17. 17. © 2019 MediaTek NeuroPilot & Platform-aware MLKits
  18. 18. © 2019 MediaTek NeuroPilot Platform-aware MLKits Super-Resolution Depth Estimation Segmentation MediaTek Platform Network Reduction Network Architecture Search Network Deep Fusion (Tiling + Fusion) BW Req.: 2.0GB/s HW Util.: 80% FPS: 100 FPS Power: < 40mW MediaTek Platform-aware MLKits Platform-friendly NN StructureUser-defined NN Structure Conv0 Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… Conv… FC Application Developers Network Quantization
  19. 19. © 2019 MediaTek NeuroPilot for Developer ANN Runtime ANN API ANN HAL Interpreter .tflite format Tensowflow Model CPU NN HAL impl. GPU NN HAL impl. VPU NN HAL impl. Caffe / ONNX Model MTK Ext. API 1. Bind Op with HW 2.Profiler 3.Debugger (Log) TOCO Offline Tool Quantization NeuroPilot specified On Device CPU GPU VPU Developers AIA NN HAL impl. AIA Supports Tensorflow as well as Caffe and ONNX Highly integrated with Android Neural Network MediaTek additions 1. Binding Op with HW 2. Profiler 3. Debugger
  20. 20. © 2019 MediaTek 20 MediaTek NeuroPilot Toolkit- utility and debug tool NN Utility Debugger Profiling NeuroPilot Toolkit • Model Convertor (TensorFlow/Caffe/ONNX) • Quantization • Power API • Performance • Memory • System Crash • Mobilelog
  21. 21. © 2019 MediaTek Conclusion 1. Provides flexible HW (CPU, GPC, VPU and AIA) for the ever changing AI algorithm. 2. Improves the performance of AI applications by reducing the memory bandwidth and increasing the compute efficiency with specialized AI accelerator. 3. Toolchain is highly integrated with Android NN, and adding performance monitor to give feedback to the algorithm developer for the optimization of AI algorithm runs on MediaTek’s platform. 4. Adding AI capability to enhance the user experience and create new use cases which mean more value for our customers. 5. Rapid advancement in edge AI technology will drive more AI applications to the edge
  22. 22. © 2019 MediaTek Our mission is to be a change catalyst, empowering our partners with smart technology solutions that will inspire them to connect with “next billion” people. By building technologies that help connect individuals to the world around them, we are enabling people to expand their horizons and more easily achieve their goals. We believe anyone can achieve something amazing. And we believe they can do it every single day. We call this idea Everyday Genius and everything we do is dedicated to making it possible. Enhance & Enrich Everyone’s Life Everyday Genius 22
  23. 23. Copyright © MediaTek Inc. All rights reserved. 23
  24. 24. © 2019 MediaTek Resource and Info 24 Product Description MediaTek helio P90 phones/mediatek-helio-p90 Benchmark ETH Zurich Benchmark 3.0 http://ai- Embedded Vision Summit MediaTek’s Approach for Edge Intelligent 10:45 AM – 11:15 AM on May 22, 2019
  25. 25. © 2019 MediaTek Thank you