Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

機器學習應用於蔬果辨識

機器學習應用於蔬果辨識,佈署於樹莓派並使用Intel Movidius NCS提升運算效能。

2018 11 28 人工智慧學校 台中分校

  • Sé el primero en comentar

機器學習應用於蔬果辨識

  1. 1. 機器學習 應用於蔬果偵測 Kobe Yu
  2. 2. 投影片授權聲明 投影片內容採 CC BY-SA-NC 3.0TW(姓名標示-相同方式分享-非商業性 3.0 台灣)。 完整授權內容 : https://creativecommons.org/licenses/by-nc-sa/3.0/tw/
  3. 3. About Me ● 清大資工博士班學生 ○ 從系統軟體的角度提升電腦視覺與類神經網路的運算效能。 ● 8+軟體開發經驗 ○ 電腦視覺/多媒體串流/iOS App開發/攝影機模組驅動與調校。 ● 開源專案 ○ Farmharvest bot (農場採收機器人) FB: FarmBot Taiwan User Group-FBTUG ○ Agrino (農業感測) http://bit.ly/2MDaAdy
  4. 4. Outline ● 農場採果機器人(FarmHarvestbot)專案簡介 ● ML開發流程以蔬果辨識為例 ● Demo ● Q&A
  5. 5. FarmBot Taiwan User Group-FBTUG ● 專案緣起 ○ 希望藉由開源農業機器的推廣與貢獻,給友善農業一點幫助。 ● 社群成員 ○ 發起人: 哈爸 ○ 成員:~2400 (各領域專家或有興趣的大眾 ) ○ 與採果專案有關的幾位成員 ■ 志弘大哥 硬體與CNC專家 ■ CH大 ML(YOLO) geek ■ Jack AR/VR、電腦視覺專家 ■ Eric 埔里小農(一點都不小的小農) ■ Olin Dataset協調人 ■ …. 默默協助的伙伴們
  6. 6. FarmHarvestbot ● 農場採果機器人 ○ 開源專案 ■ 軟體/硬體/機構/資料集 ○ Why ■ 農業缺工問題 ■ 採收時效性 ■ 知識與經驗共享 ○ How ■ Core/Support群組 ■ 開放討論 FB: FarmBot Taiwan User Group-FBTUG Doc: https://paper.dropbox.com/doc/FBTUG-FarmHarvestBot-w2FKkhc4ZTlj6knhOK43p
  7. 7. Machine Learning + Computer Vision ● 目標 ○ 蔬果辨識與定位 ■ 種類 ● 小果番茄/小黃瓜/彩椒 ■ ML ● 辨識蔬果 ■ CV ● 影像前處理:擷取影像+Resize ● 座標位置(x,y,z) ○ 蔬果中心位置 ○ 切割點位置 ○ 座標轉換 ■ 夾具移動向量
  8. 8. Machine Learning + Computer Vision ● 建立自己的YOLO辨識模型 – 以柑橘辨識為例 - CH大大 source https://projectplus.cc/Projects/yolo-201810/
  9. 9. Machine Learning Pipeline Phase 1. Problem Understanding Phase 3. Model Processing Phase 2. Data Processing Phase 4. Deploy
  10. 10. Problem Understanding ● 目標 ○ 專案目標:蔬果採收機器人 ■ NN+Vision:辨識可採收的蔬果位置 (x,y,z) + 座標轉換(camera <-> arm) ■ 即時處理 ● 系統架構 ○ 軟體 Raspbian + ROS ○ 硬體 RPi ○ 機構 (Arm/Camera/autonomous vehicles) ● 作物種類 ○ 小果番茄/彩椒/小黃瓜 ○ 採收標準/採收方法 ○ 夾具 ● 場域 ○ 溫室為主,一個溫室基本上只種一種作物 Phase 1 Phase 2 Phase 3 Phase 4
  11. 11. Problem Understanding ● 目標物種 ○ 小果番茄 - 玉女番茄 ● 場域與栽種方式 ○ 溫室 / 掛網式 (走道寬度/目標物件高度) ● USB Camera ○ 解析度 / F0V(影響目標物件在影像中的成像大小 ) ● 判斷標準(採收標準/採收方式) ● 運行平台 ○ RPi+Intel Movidius NCS source: https://paper.dropbox.com/doc/FBTUG--ASl58MS7PkuY4SIYoHLxJXHoAg-QdMWCaBBoyudOTNuvhd4c Phase 1 Phase 2 Phase 3 Phase 4
  12. 12. Data Processing ● 資料特性 ● 蒐集照片 ● 資料標記 ● 資料轉換 Phase 1 Phase 2 Phase 3 Phase 4
  13. 13. Data Processing ● 資料特性 ○ 目標物種 ■ 小果番茄 - 玉女番茄 ○ 標記類別 ■ 0_pepper_flower 花朵 ■ 1_pepper_young 尚未成熟果實 ■ 2_pepper_matured 成熟果實可摘採 ■ 3_pepper_covered 成熟果實需撥葉摘採 ○ 判斷標準與注意事項(from domain expert) Phase 1 Phase 2 Phase 3 Phase 4 source https://paper.dropbox.com/doc/FBTUG--ASl58MS7PkuY4SIYoHLxJXHoAg-QdMWCaBBoyudOTNuvhd4c
  14. 14. Data Processing ● 蒐集照片 ○ 活動紀錄 ■ 基本資訊(人事時地物) ■ 作物資料 ■ 相機資料 ■ 拍攝方式 source 活動紀錄 https://paper.dropbox.com/doc/FBTUG-FarmHarvestBot--ASkl43eqEeSkI3g~dcTtYHVWAg-XHZpHOCR4Tw6amCxMY0E0 資料蒐集紀錄表https://paper.dropbox.com/doc/JsCHkjEGzSIAvT65J0fc4 Phase 1 Phase 2 Phase 3 Phase 4
  15. 15. Data Processing ● 資料標記 ○ 工具 ■ ImageLabel (offline) ■ Datatrunk (online) ○ 輸出格式 ■ PASCAL VOC Format(XML) source https://drive.google.com/file/d/1WfdacRUEJ1HgIlQsimbMVLVe2GlKsKJS/view https://dataturks.com/projects/kobeyu/FarmHarvestBot-tomato Phase 1 Phase 2 Phase 3 Phase 4
  16. 16. Pascal VOC format. <annotation> <folder>GeneratedData_Train</folder> <filename>000001.png</filename> <path>/my/path/GeneratedData_Train/000001.png</path> <source> <database>Unknown</database> </source> <size> <width>224</width> <height>224</height> <depth>3</depth> </size> <segmented>0</segmented> <object> <name>21</name> <pose>Frontal</pose> <truncated>0</truncated> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>82</xmin> <xmax>172</xmax> <ymin>88</ymin> <ymax>146</ymax> </bndbox> </object> </annotation> Phase 1 Phase 2 Phase 3 Phase 4
  17. 17. Data Processing ● 資料轉換 ○ VOC to YOLO ○ VOC to tfrecord ■ create_pascal_tf_record.py source https://github.com/tensorflow/models/blob/master/research/object_detection/dataset_tools/create_pascal_tf_record. py Phase 1 Phase 2 Phase 3 Phase 4
  18. 18. Model Processing ● 定義問題 ● 選擇網路架構 ● 訓練 ● 驗證 Phase 1 Phase 2 Phase 3 Phase 4
  19. 19. Classification/Detection/Segmentation source https://medium.com/comet-app/review-of-deep-learning-algorithms-for-object-detection-c1f3d437b852
  20. 20. Object Detection ● R-CNN ○ Fast R-CNN ○ Faster R-CNN ● YOLO ○ YOLO ● SSD ○ SSD+Mobilenet Phase 1 Phase 2 Phase 3 Phase 4
  21. 21. Object Detection Dataset ● PASCAL VOC ● MSCOCO ● ImageNet Det Phase 1 Phase 2 Phase 3 Phase 4
  22. 22. Model Processing ● 選擇網路 ○ 準確度 v.s 執行速度 ● 回顧Phase1 ○ 運行平台 ■ RPi + Intel Movidius NCS ○ 實測結果 ■ SSD-Mobilenet(5.5 FPS) ■ Tiny-yolo v2(0.7 FPS) reference https://github.com/movidius/ncappzoo https://www.youtube.com/watch?v=llBhBSgoWPs Phase 1 Phase 2 Phase 3 Phase 4
  23. 23. Model Processing ● 效能指標 ○ mAP ○ IoU Phase 1 Phase 2 Phase 3 Phase 4 source https://github.com/Cartucho/mAP
  24. 24. Mobilenet ● Separate convolutions ○ Depthwise + pointwise convolution ● Multiplier ○ α:Width Multiplier (0.25、0.5、0.75、1) ○ ρ:Resolution Multiplier (128、160、192、224) source: https://yinguobing.com/bottlenecks-block-in-mobilenetv2/ https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.md https://arxiv.org/abs/1704.04861 Phase 1 Phase 2 Phase 3 Phase 4
  25. 25. Mobilenet 輸入資料: W_in * H_in * Nch,Kernel Map: k * k * Nk,輸出資料: W_out * H_out * Nk 一般卷積 計算量為 W_in * H_in * Nch* k * k * Nk Depthwise separable convolution計算量為 (Depthwise convolution計算量+Pointwise convolution計算量) Depthwise convolution計算量為 W_in * H_in * Nch* k * k Pointwise convolution計算量為 Nch*Nk * W_in * H_in source https://medium.com/@chih.sheng.huang821/%E6%B7%B1%E5%BA%A6%E5%AD%B8%E7%BF%92-mobilenet -depthwise-separable-convolution-f1ed016b3467
  26. 26. Mobilenet Million MACs Million Parameters Top-1 Accuracy Top-5 Accuracy MobileNet_v1_1.0_224 569 4.24 70.9 89.9 MobileNet_v1_1.0_192 418 4.24 70.0 89.2 MobileNet_v1_1.0_160 291 4.24 68.0 87.7 MobileNet_v1_1.0_128 186 4.24 65.2 85.8 MobileNet_v1_0.75_224 317 2.59 68.4 88.2 MobileNet_v1_0.50_224 150 1.34 63.3 84.9 MobileNet_v1_0.25_224 41 0.47 49.8 74.2 MobileNet_v1_1.0_224_quant 569 4.24 70.1 88.9 MobileNet_v1_0.25_128_quant 14 0.47 39.5 64.4 Phase 1 Phase 2 Phase 3 Phase 4
  27. 27. MobilenetV2 ● Classification ○ MobilenetV2 ● Detection ○ MobileNetV2 + SSDLite ● Segmentation ○ MobileNetV2 + DeepLabV3 source https://ai.googleblog.com/2018/04/mobilenetv2-next-generation-of-on.html
  28. 28. MobilenetV1 v.s MobilenetV2
  29. 29. Model Processing ● Model ○ SSD+Mobilenetv1 ○ SSDLite+Mobilenetv2 ○ yolo ● Dataset (Image + PASCAL VOC) ○ Farmharvestbot ■ 小果番茄 ■ 彩椒 ○ MS COCO Phase 1 Phase 2 Phase 3 Phase 4
  30. 30. Model Training ● Model ○ SSD-Mobilenetv1 ● Dataset ○ MS COCO ● Training framework ○ Tensorflow + Tensorflow/Models/research/object_detection ● Script python object_detection/model_main.py --pipeline_config_path=./object_detection/ssd_mobilenetv1/model/ssd_mobilenet_v1_pets.config --model_dir=./object_detection/ssd_mobilenetv1/model/ --num_train_steps=200000 --num_eval_steps=8000 --alsologtostderr source https://github.com/kobe0308/models.git (branch develop) Phase 1 Phase 2 Phase 3 Phase 4
  31. 31. Model Processing ● Model ○ SSD+Mobilenetv1 ● Dataset ○ MS COCO ● Training parameters ○ batch size:24 ○ steps: 200K ○ classes:20 Phase 1 Phase 2 Phase 3 Phase 4
  32. 32. Model Processing ● Model ○ SSD+Mobilenetv1 ● Dataset ○ Farmharvestbot-tomato ● Training parameters ○ batch size:24 ○ steps: 200K ○ classes:4 Phase 1 Phase 2 Phase 3 Phase 4
  33. 33. Model Processing ● Model ○ SSD+Mobilenetv1 ● Dataset ○ Farmharvestbot-pepper ● Training parameters ○ batch size:24 ○ steps: 200K ○ classes:4 Phase 1 Phase 2 Phase 3 Phase 4
  34. 34. Deploy ● Edge compute (compare with client/server) ● Platform(RPi+Neural Compute Stick) ■ SBC(RPi 3 Model B) ● 1.4GHz Quad-Core ARM Cortex-A53 (64Bit) ■ NCS ● Intel Movidus NCS ○ 12 Shave Core(VLIW Arch) ○ 1 TOPs ● Intel Movidus NCS2 ○ 16 Shave Core(VLIW Arch) ○ Dedicated Neural Compute Engine ○ 4 TOPs ● Orange Pi AI Stick 2801 ○ Matrix Processing Engine ○ 9.3 TOps source https://qconsf.com/sf2017/system/files/presentation-slides/qconf_presentation.pdf Phase 1 Phase 2 Phase 3 Phase 4
  35. 35. NN+Vision Pipeline ● Recap problem understanding ○ Realtime! ○ Capture image ■ resolution @30fps ○ Acceleactor ■ NN inference - NCS ■ Image Processing - CPU(Multi-thread + SIMD/ARM compute library) ○ Programming language ■ Python or C++ Image Preprocess Capture Image NN Inference Image Postprocess source https://www.anandtech.com/show/7591/answered-by-the-experts-arms-cortex-a53-lead-architect-peter-greenhalgh Phase 1 Phase 2 Phase 3 Phase 4
  36. 36. Movidius Toolkit NCSDK ● Compiler (from trained nn to inference engine) ● Checker ● Profiler (profile performance in each nn layer) ● NCSAPI(C++) source https://qconsf.com/sf2017/system/files/presentation-slides/qconf_presentation.pdf Phase 1 Phase 2 Phase 3 Phase 4
  37. 37. Movidius Toolkit ● Command ○ mvNCCompile -s <number of shaves> <name of the model>_frozen.pb -in=input -on=<name of the output node> ● Support training framework ○ Tensorflow ○ Caffe Phase 1 Phase 2 Phase 3 Phase 4 source https://movidius.github.io/ncsdk/tf_modelzoo.html
  38. 38. Movidius Architecture ● SHAVE core ○ 8 parallel function units ■ predicated execution unit (PEU) ■ branch and repeat unit (BRU) ■ two 64-bit load-store units (LSU0/LSU1) ■ 32-bit integer arithmetic unit (IAU) ■ 32-bit scalar arithmetic unit (SAU) ■ 128-bit vector arithmetic unit(VAU) ■ 128-bit compare move unit (CMU) source https://en.wikichip.org/wiki/movidius/microarchitectures/shave_v2.0 Phase 1 Phase 2 Phase 3 Phase 4
  39. 39. Single Issue RISC v.s VLIW source https://slideplayer.com/slide/4896590/ Phase 1 Phase 2 Phase 3 Phase 4
  40. 40. Demo ● Platform ○ RPi3+Intel Movidius NCS ● Model ○ SSD+Mobilenet ● FPS ~5 Phase 1 Phase 2 Phase 3 Phase 4
  41. 41. Robot Arm and Camera Calibration source https://slideplayer.com/slide/7491989/
  42. 42. TODO... ● 坑坑相連到天邊... ● Join us!
  43. 43. Farmharvestbot FB: FarmBot Taiwan User Group-FBTUG Doc: https://paper.dropbox.com/doc/FBTUG-FarmHarvestBot--ASlFVrNQa1XWOCWxktW_~JkgAg-w2FKkhc4ZTlj6knhOK43p 專案成員: https://paper.dropbox.com/doc/FBTUG-FarmHarvestBot--ASkpEA1~UFD9CQDm4hhPaWKMAg-9q9yQmaWjhkgh1EdB4EBc
  44. 44. Q & A

    Sé el primero en comentar

    Inicia sesión para ver los comentarios

  • kunLZCHENG

    Nov. 28, 2018
  • ouowen

    Aug. 11, 2019

機器學習應用於蔬果辨識,佈署於樹莓派並使用Intel Movidius NCS提升運算效能。 2018 11 28 人工智慧學校 台中分校

Vistas

Total de vistas

968

En Slideshare

0

De embebidos

0

Número de embebidos

1

Acciones

Descargas

29

Compartidos

0

Comentarios

0

Me gusta

2

×