Se ha denunciado esta presentación.
Se está descargando tu SlideShare. ×

AIoT: Intelligence on Microcontroller

Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Cargando en…3
×

Eche un vistazo a continuación

1 de 68 Anuncio

AIoT: Intelligence on Microcontroller

Descargar para leer sin conexión

A late upload. This slide was presented on Aug 31, 2019, when I delivered a talk for AIoT seminar in University of Lambung Mangkurat, Banjarbaru. It's part of Republic of IoT 2019 event.

A late upload. This slide was presented on Aug 31, 2019, when I delivered a talk for AIoT seminar in University of Lambung Mangkurat, Banjarbaru. It's part of Republic of IoT 2019 event.

Anuncio
Anuncio

Más Contenido Relacionado

Presentaciones para usted (20)

Similares a AIoT: Intelligence on Microcontroller (20)

Anuncio

Más de Andri Yadi (16)

Más reciente (20)

Anuncio

AIoT: Intelligence on Microcontroller

  1. 1. AIoT: Intelligence on Microcontroller Intro & Live Demo ULM | Banjarbaru - Aug 31, 2019 Andri Yadi CEO, DycodeX
  2. 2. Andri Yadi Co-founder & CEO of DycodeX Deputy Chairman, Indonesia IoT Association a (at) dycodex.com | http://andriyadi.com A Physicist, Developer, Maker, Community Guy, Entrepreneur About Me MicrosoM Most Valuable Professional (MVP) of Azure for 12 years Code for food & passion for 20 years Break & make electronic stuTs for 22 years Trying to change the world through entrepreneurship, 15 years now
  3. 3. Internet of Things
  4. 4. IoT Common (Simplified) Architecture Gateways / Base Station Rule + Alert, Internet CloudEdge Ingestion & Storage Things Visualization User Apps Devices & Firmware Mgt. Analytics (lots of them) Radio
  5. 5. What if we put AI into IoT?
  6. 6. What’s AI for
  7. 7. Ingestion / API Intelligence EngineInternet AI-powered, Cloud-backed Application Edge Cloud Simplified Architecture Inference result Raw data
  8. 8. AI + IoT IoT is about automated data collecting, storing, visualisation, and reacting, in massive scale Combining AI + IoT enables a lot of use cases & business models! “Data Supplier” AI is about making sense of plethora of data, generating insights & recommendation, and predicting the future outcome “Data Miner”
  9. 9. Hyperlocal Weather Prediction Solar sensor Air temperature, humidity, barometric pressure Rain gauge Wind speedWind direction “Cheap” weather station on each blocks (city, farm, …) Air quality Weather Prediction Use Cases: Farming Online “Ojek” Fare and Distribution Logistics Flooding Damage predictionDeep Learning Model Examples: IBM Deep Thunder SmaRphone data
  10. 10. Ingestion / API Azure Cognitive Service: Face API Internet Example: Gender & Emotion Recognition Edge Cloud Microcontroller + Camera Sensor Inference result (JSON) Image
  11. 11. Ingestion / API Azure Cognitive Service: Face API Internet Example: Gender & Emotion Recognition Edge Cloud Microcontroller + Camera Sensor Inference result (JSON) Image
  12. 12. Ingestion / API Azure Cognitive Service: Face API Internet Example: Gender & Emotion Recognition Edge Cloud Male, Neutral Microcontroller + Camera Sensor Inference result (JSON) Image
  13. 13. Demo: Gender & Emotion Recognition in the Cloud hPps://www.youtube.com/watch?v=vvQfYqrUwFg
  14. 14. …and those are cloud-powered Artificial Intelligence Source: https://www.riskgroupllc.com/wp-content/uploads/Cloud-Powered-AI.jpg
  15. 15. AI on The Cloud Pros: You can always have more, just scale “Unlimited” computing power: for ML learning & inference “Unlimited” storage: for AI model and dataset Easier to maintain Easier to update (soMware, AI model/algorithm), to secure, etc Cons: High latency: as data need to transferred to cloud Must be online: to exchange data with cloud. Connectivity can be challenges for cecain use cases Potential privacy issue: your face goes to the cloud, who know what will happen
  16. 16. AI on The Cloud - Cons High latency Must be online Potential privacy issue What If… We move intelligence closer or at the Edge
  17. 17. Ingestion / API Intelligence EngineInternet Edge Cloud Simplified Architecture of AI-powered Application Inference result Raw data Back to…
  18. 18. Ingestion & API Intelligence Engine Internet Edge Cloud Intelligence Current Trend PaR of “intelligence” move to the Edge
  19. 19. But, why AI at the Edge?
  20. 20. AI at The Edge Pros: High pemormance: less latency as no data need to go to cloud Works onine: no internet connectivity needed to exchange data with cloud. Beher privacy: data can stays on-device if necessary Eocient power consumption: no need to send large data to cloud Cons: ML model needs to be optimised: optimisation during or post training. Post training optimisation involves model conversion which not always be trivial Challenging deployment & maintenance: new ML model means OTA update is needed for many devices
  21. 21. So, how can we achieve AI at the Edge?
  22. 22. ML-accelerated processor a class of microprocessor designed as hardware acceleration for AI applications, e.g. for neural networks, machine vision and machine learning Example: GPU FPGA ASIC Development Board / Accessories a standalone computing system with ML- accelerated processor (main or co-) as a main or additional development unit Example: Coral DevBoard or USB Accelerator Intel Neural Compute Stick Sipeed Maix Sooware Tools Compilers ML Model Convecer SDK OS Example: TensorFlow & TensorFlow Lite OpenVino nncase SuppoRs Documentation Model Zoo Examples Datasheets Hardware Sooware What do we need?
  23. 23. Object Detection at the Edge
  24. 24. Object Detection at the Edge What does really happen there?
  25. 25. But I’m using A Raspberry Pi (+ Intel NCS 2)
  26. 26. ML-accelerated processor Optimized deep learning solutions across multiple Intel® hardware plarorms Example: CPU (Xeon) FPGA (Arria) ASIC (Movidius) Development Board / Accessories Example: Intel Neural Compute Stick (Movidius) UP Square Board + AI Vision X Dev Kit IEI TANK AIoT Dev Kit Sooware Tools Model Optimizer Inference Engine Model Downloader SuppoRs Documentation Reference Implementation Model Zoo Examples Datasheets Hardware Sooware Intel OpenVino Open Visual Inferencing and Neural Network Optimization hhps://docs.openvinotoolkit.org
  27. 27. UP Square Board + AI Vision X Dev Kit IEI TANK AIoT Dev Kit Dev Kit for Edge Intelligence using Intel Hardware
  28. 28. UP Square Board + AI Vision X Dev Kit Nvidia Jetson Nano Google Coral Dev Board Raspberry Pi 3 + Intel NCS 2 We can use SBC + NNA
  29. 29. 5-10Wahs But…
  30. 30. For Low Power Use Case Use Microcontroller (MCU) Sipeed Maix K210-powered dual core 64-bit RISC-V 400 MHz With/Without WiFi SparkFun Edge  32-bit ARM Cortex-M4F 48 MHz; 1MB Flash; 384KB SRAM BLE 5 Radio SensorTile  STM32L476JG 32-bit ARM Cortex- M4 80 MHz; 1MB Flash; 128KB SRAM Accel, Gyro, Compass, Pressure, Microphone sensor BLE 4.2 Radio
  31. 31. Let me talk a bit of Sipeed Maix Disclaimer: I have no asliation with Sipeed (yet)
  32. 32. Sipeed Maix Go - K210-based AI Board hPps://www.seeedstudio.com/Sipeed-MAix-GO-Suit-for-RISC-V-AI-IoT-p-2874.html
  33. 33. 0.3-0.5Wahs Better yet… Idle to full power inference 0.03 waP for sleeping mode
  34. 34. Image Classification at the Edge
  35. 35. Choosing Network Arch for Image Classiwcation
  36. 36. KPU DVP K210 Processor Camera Sensor MobileNet Model CPU Classiwcation Result Object MaixPy (MicroPython) But, 17MB is still too big! Image Classification using Maix
  37. 37. Train me, master…
  38. 38. ML Model Pipeline for K210 Compress (toco) Optimize (nncase) Train (on your awesome machine or Cloud) .h5 .pb .Ilite .kmodel Copy to MCU via SD Card, or Burn to its Flash Inference (KPU library) Inference Result
  39. 39. ML Model Pipeline for AI at The Edge - in General Optimize (compress, remove, replace) Train (on your awesome machine or Cloud) .h5 .pb .cayemodel ONNX Intermediate Representation File Transfer IR gle to Edge device Inference (Edge-optimized Inference library) Inference Result
  40. 40. Image Classification with your own dataset What about…
  41. 41. AIoT-powered Smart Trash Management Platform We build… by DycodeX
  42. 42. Sorting Plastic and non-plastic trash using AI-Powered machine vision Compacting Optimizing trash storage Reward Calculate reward point based on trash type Notify & Monitoring Let the waste management official knows upon full capacity or other conditions What it can do
  43. 43. AITrash: Classiwcation Demo
  44. 44. Train Trash Classification Model Compress (toco) Optimize (nncase) Train (on your awesome machine or Cloud) .h5 .pb .Ilite .kmodel Copy to MCU via SD Card, or Burn to its Flash Inference (KPU library) Inference Result Trash DataSet (TrashNet) Trash Layers (Based on MobileNet v1)
  45. 45. KPU DVP K210 Processor Camera Sensor A Model CPU Plastic BoPle Object MaixPy Glass BoPle Other Trash Classiwcations Result Trash Classification using Maix + MaixPy
  46. 46. AI at the Edge for ESPectro32? Now, what about…
  47. 47. NB-IoT + GPS/GNSS Kit LoRa/LoRaWAN + GNSS Kit Sensors & Actuators Kit (20+ Peripherals) 3G + GNSS + Sensors ESPectro32 Optimized for learning STEM, coding & electronics, also for fast prototyping. An IoT development Board: Low-power, dual-core 240 MHz Microcontroller Built-in WiFi & Bluetooth connectivity 7x7 LED matrix for user intemace Sensors: temperature digital sensor, phototransistor, accelerometer microSD card slot Extensible with Backpacks: NB-IoT/Cellular, LoRa, Sensor Kits, Motor Driver, and more ESPectro32 All products’ electronic board is designed & assembled in-house by 100% Indonesian Backpacks
  48. 48. Pipeline Train (on your machine or Cloud) .nnp wle (Intermediate Representation File) Inference (nnabla C runtime) Inference Result Design (Python, C++) Generate Code (nnabla_cli)
  49. 49. MNIST LeNet
  50. 50. using ESPectro32 and NNABLA MNIST
  51. 51. GET /index.html POST /upload/wle Inference/predict hand-wriPen number Display predicted number on LED Matrix Demo Flow
  52. 52. AI at the Edge for non-image data? Now, what about…
  53. 53. Let’s do activity recognition
  54. 54. KPU I2C K210 Processor HAR Model (.kmodel) CPU SiPing MaixPy Standing Walking Classiwcations Result Activity Recognition using Maix + MaixPy IMU Sensor Running Retrain Compress & Optimize .h5 .pb HAR DataSet HAR Layers
  55. 55. using Keras Train Activity Recognition Model
  56. 56. Activity Recognition using M5StickV + MaixPy
  57. 57. Artificial Intelligence of Things
  58. 58. Intel OpenVino: Docs: hhps://docs.openvinotoolkit.org/ Model Zoo: hhps://github.com/opencv/open_model_zoo Implementations: hhps://soMware.intel.com/en-us/smac-video MicroPython for K210: Sipeed Maix Boards: hhps://www.seeedstudio.com/tag/MAIX.html MaixPy: hhp://github.com/sipeed/MaixPy MicroPython for Kendryte K210: hhps://github.com/loboris/MicroPython_K210_LoBo DycodeX: SMARTernak: hhps://smacernak.com ESPectro32: hhps://shop.makestro.com/product/espectro32-v2/ Other products & solutions: hhps://dycodex.com Azure: Azure Machine Learning: hhps://azure.microsoM.com/en-in/services/machine-learning- service/ Contact me: andri@dycodex.com hhp://github.com/andriyadi hhps://www.slideshare.net/andri_yadi/ Call to Action
  59. 59. Contact us to get these
  60. 60. Contact hPps://facebook.com/asioti/ hPp://bit.ly/asioti-tgram  hPp://bit.ly/MemberASIOTI
  61. 61. makestroid makestroid makestroid makestro.com An Indonesia Platform for Maker: to “democratize” knowledge, hardware kit, and software to help makers to start making in hardware, to drive into Internet of Things Start Making at Learning Hardware Marketplace Software & Cloud Community RIoT is Makestro’s program for nurturing IoT makers, officially supported by Indonesia’s Ministry of ICT
  62. 62. Want to put “AI” in “BrAIns”?
  63. 63. hi@dycodex.com | https://dycodex.com AIoT & Maker Movement enabler Bandung, Indonesia Keep in touch

×