Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

HKG18-302 - Huawei HiAI : Unlock The Future

1.185 visualizaciones

Publicado el

Session ID: HKG18-302
Session Name: HKG18-302 - Huawei HiAI : Unlock The Future
Speaker: Zhou Hua
Track: LEG AI;Ecosystem Day


★ Session Summary ★
Huawei HiAI is one of the world's first mobile AI computing platforms with a dedicated Neural-network Processing Unit (NPU). We provide three levels of open platform which are

HiAI foundation, HiAI engine and HiAI service, based on these platforms, developers can focus on developing new AI applications instead of focusing on performance tuning .

In this talk :

- Will interpret the trend of AI computing and explain why a dedicated NPU is required for the growing mobile computing market.

- What HiAI Mobile computing platform is , will introduce HiAI foundation, HiAI engine and HiAI service ,

(In this part Will introduce what are the objectives and goals of HiAI Engine and What is it expect to achieve)
---------------------------------------------------
★ Resources ★
Event Page: http://connect.linaro.org/resource/hkg18/hkg18-302/
Presentation: http://connect.linaro.org.s3.amazonaws.com/hkg18/presentations/hkg18-302.pdf
Video: http://connect.linaro.org.s3.amazonaws.com/hkg18/videos/hkg18-302.mp4
---------------------------------------------------
★ Event Details ★
Linaro Connect Hong Kong 2018 (HKG18)
19-23 March 2018
Regal Airport Hotel Hong Kong

---------------------------------------------------
Keyword: LEG AI;Ecosystem Day
'http://www.linaro.org'
'http://connect.linaro.org'
---------------------------------------------------
Follow us on Social Media
https://www.facebook.com/LinaroOrg
https://www.youtube.com/user/linaroorg?sub_confirmation=1
https://www.linkedin.com/company/1026961

Publicado en: Tecnología

HKG18-302 - Huawei HiAI : Unlock The Future

  1. 1. Huawei HiAI Unlock the Future Eric Zhou Kirin Chipset Solution Planning Director zhouchen@huawei.com
  2. 2. AI is coming round ● Get AI for your apps ● 25 Apps, 9 with AI The 25 Best Apps of the Year So Far (2017.6)
  3. 3. Why On-device AI? Real-time Privacy
  4. 4. AI Knowledge in an instant burst
  5. 5. AI Knowledge processing in real time is key point Mobile AI Processing Knowledge
  6. 6. A real APP needs More Resource CPU GPU • Task scheduling • Load balance • Memory allocation • UI rendering • Graphics processing • AI Computing ISP/DSP • Camera 3A • Image processing NPU
  7. 7. Work load for a AI APP CPU little core CPU big core GPU NPU
  8. 8. Two Key Questions for AI APPs ●What Performance we need? ●Who will make the APPs?
  9. 9. Real-time Processing is the Requirement 5fps 15fps >25fps
  10. 10. To Match Real-time AI Processing Ruequirement ● The NPU supports a dedicated set of AI instructions for neural network model operations that allow more efficient parallel execution of more neural network operators within minimal clock cycles. Convolution Deconvolutio n Pooling ReLU Normalize BatchNorm FullConnectio n Sigmoid CPU = Scalar • General-purpose computing • Logic control GPU = Vector • Image processing • Large-scale parallel computing NPU = Tensor • Knowledge model processing • Dedicated AI Instruction- Set • Large-scale parallel computing
  11. 11. High Performance Density 5.5Billion Transistors Die Size NN Perf. NPU 3% 100% CPU 6% 5% GPU 18% 60%
  12. 12. What developers care about ? ● Development Cost ○ The porting cost : operators , tools , documents and FAE ● Benefits ○ users , downloads , marketing ● Intellectual property protection ○ Model (algorithm) encryption
  13. 13. HiAI Mobile Computing Platform ● Open AI ecosystem ● 3rd parties APPS empowered by HiAI mobile computing platform APP APP CPU Caffe/Caffe2 … HiAI APIs HiAI Model and Resouse Management/Runtime GPU DSP NPU TensorFlow/TensorFlow Lite
  14. 14. What HiAI provides? ● Provides frequently used artificial intelligence function APIs which can efficiently run on mobile devices. ● Supports popular frameworks. ● Provides processor-independent acceleration APIs to allow the Kirin hardware to accelerate model and operator computing. ● Schedules heterogeneous resources flexibly to enable developers to accelerate neural network model and operator computing. Tool chain Complete documentsAbundant APIs Easy-to-use source code
  15. 15. HiAI SDK V100 Framework Operators Tools Documents and Support 99 Caffe, TensorFlow HiAI SDK V150 42 Caffe, TensorFlow TensorFlow Lite Huawei HiAI SDK Android NN Command line User manual Source code FAE FAQ / Open class New more sample code and document More FAE supporting Android Studio plug-in Compile tool Sparsifying tool Code automatically generated Log tools
  16. 16. What kinds of APP that HiAI can unlockShort videos and live streaming Facial recognition, gesture recognition, portrait segmentation, posture recognition, and video stylization Social platform Photo classification, image recognition, and super- resolution imaging AR Depth estimation, light estimation, environmental understanding, and simultaneous localization and mapping (SLAM) Photo taking and retouching Facial beautification and image enhancement Shopping Shopping through image recognition Translation and words processing Instant camera translation, optical character recognition (OCR), word splitting, named entity recognition (NER), textual emotion recognition, and intelligent textual response
  17. 17. Portrait Segmentation HiAICPU
  18. 18. Shopping through image recognition on device
  19. 19. ● The HiAI mobile computing platform supports sparse model acceleration. The NPU can skip the multiply-add algorithms with a coefficient of zero, which can greatly improve the computing efficiency and reduce the bandwidth while maintaining computing precision. ● The HiAI mobile computing platform supports 8-bit and 1-bit quantization, effectively reducing the computing bandwidth and storage consumption and improving energy efficiency. HiAI Mobile Computing Platform Technologies ToolsTools Energy Efficiency Ratio During CNN Model Execution
  20. 20. HiAI Mobile Computing Platform Execution● As shown in the figure below, by using conversion tools, a trained neural network model is converted into an offline model that can be efficiently executed on the HiAI mobile computing platform, and output as the binary file, the offline model. The main purpose of converting the standard neural network model (such as Caffe) into an offline model is to optimize the network configuration. After conversion, an optimized target file, namely the offline model, is generated. The offline model is serialized and stored on the disk. As a result, when the inference is performed, the optimized target file is used, which is faster.  Offline Mode Generation
  21. 21. HiAI Mobile Computing Platform Execution ● As shown in the figure below, during the offline model computing, the offline model is loaded from the file, and data such as images input by users is copied to HiAI's NPU for computing. User data needs to be transferred from the DDR SDRAM to the NPU and then returned to the DDR SDRAM for each inference during computing. Offline Mode Computing NPU Offline Model Data Layer Copy Data Load
  22. 22. How to integrate your knowledge model Operator compatibility assessment Model conversion Model loading Model execution Model unloading 1 2 3 4 5
  23. 23. HiAI Mobile Computing Platform Advantages● HiAI: enables offline models to be compiled without occupying the runtime, which improves runtime efficiency. ● HiAI: supports multiple frameworks such as Caffe and TensorFlow/Lite, and will support Caffe2, ONNX in the future. ● HiAI: supports multiple operators and rapid iteration, and supports operator customization in later versions. ● HiAI: supports Kirin 970 and later chip platforms and rapidly increasing devices, and is widely used. ● HiAI is also integrated into HiKey970 which is the third generation of Hikey series in 96boards
  24. 24. More and More
  25. 25. New HiKey970 AI Development Board is coming ● Huawei HiAI SDK ● Up to 25X Performance 50X Power Efficiency ● Dedicated Neural-network Processing Unit (NPU) ● Heterogeneous Resource Management More stronger Compute abilities More Hardware interfacesMainly OSPopular AI stacks
  26. 26. HiKey970 Spec Software Hardware OS Choices Stacks • Huawei HiAI • Android NN • OpenGL ES • OpenCL • Ubuntu • Debian • Android Master • CPU: 4 x A73 2.36GHz + 4 x A53 1.8GHz • GPU: Mali G72 MP12 • NPU: 256MAC/cycle • 6GB 1866MHz, 4 Channel LPDDR4x • 4 lanes CSI + 2 lanes CSI • CAN V2.0B up to 1Mbps • Video Dec up to H.265 3840x2160@60fps • UEFI + ARM Trusted Firmware • Kernel 4.9 • CAN driver • CSI driver • WiFi enable • Video Codec enable
  27. 27. Shining Star Plan Talent cultivation Developme nt support Innovation support Marketing Campus project Training Certification Innovation lab Development environment Testing tools Funded Cloud-Client resource Contest awards User growth Co- Marketing promotion
  28. 28. How to Get HiAI SDK and HiKey970? • http://developer.huawei.com/consumer/en/home • https://www.96boards.org/ • http://www.hihope.org • Amazon channel for worldwide market, two stores in Amazon: “Hoperun” and “HiHope”,  Secure96 is on sell now • Taobao channel for China market https:// shop596522926.taobao.com/index.htm?spm=2 013.1.w5002-17633002258.2.32b43ca7TxdF52 • http://www.lenovator.com/ Available in APR. 2018 $299Huawei Developer Website
  29. 29. Thank You #HKG18 HKG18 keynotes and videos on: connect.linaro.org For further information: www.linaro.org

×