Más contenido relacionado
La actualidad más candente (20)
Similar a 機器學習應用於蔬果辨識 (20)
機器學習應用於蔬果辨識
- 3. About Me
● 清大資工博士班學生
○ 從系統軟體的角度提升電腦視覺與類神經網路的運算效能。
● 8+軟體開發經驗
○ 電腦視覺/多媒體串流/iOS App開發/攝影機模組驅動與調校。
● 開源專案
○ Farmharvest bot (農場採收機器人) FB: FarmBot Taiwan User Group-FBTUG
○ Agrino (農業感測) http://bit.ly/2MDaAdy
- 5. FarmBot Taiwan User Group-FBTUG
● 專案緣起
○ 希望藉由開源農業機器的推廣與貢獻,給友善農業一點幫助。
● 社群成員
○ 發起人: 哈爸
○ 成員:~2400 (各領域專家或有興趣的大眾 )
○ 與採果專案有關的幾位成員
■ 志弘大哥 硬體與CNC專家
■ CH大 ML(YOLO) geek
■ Jack AR/VR、電腦視覺專家
■ Eric 埔里小農(一點都不小的小農)
■ Olin Dataset協調人
■ …. 默默協助的伙伴們
- 6. FarmHarvestbot
● 農場採果機器人
○ 開源專案
■ 軟體/硬體/機構/資料集
○ Why
■ 農業缺工問題
■ 採收時效性
■ 知識與經驗共享
○ How
■ Core/Support群組
■ 開放討論
FB: FarmBot Taiwan User Group-FBTUG
Doc: https://paper.dropbox.com/doc/FBTUG-FarmHarvestBot-w2FKkhc4ZTlj6knhOK43p
- 7. Machine Learning + Computer Vision
● 目標
○ 蔬果辨識與定位
■ 種類
● 小果番茄/小黃瓜/彩椒
■ ML
● 辨識蔬果
■ CV
● 影像前處理:擷取影像+Resize
● 座標位置(x,y,z)
○ 蔬果中心位置
○ 切割點位置
○ 座標轉換
■ 夾具移動向量
- 8. Machine Learning + Computer Vision
● 建立自己的YOLO辨識模型 – 以柑橘辨識為例 - CH大大
source https://projectplus.cc/Projects/yolo-201810/
- 10. Problem Understanding
● 目標
○ 專案目標:蔬果採收機器人
■ NN+Vision:辨識可採收的蔬果位置 (x,y,z) + 座標轉換(camera <-> arm)
■ 即時處理
● 系統架構
○ 軟體 Raspbian + ROS
○ 硬體 RPi
○ 機構 (Arm/Camera/autonomous vehicles)
● 作物種類
○ 小果番茄/彩椒/小黃瓜
○ 採收標準/採收方法
○ 夾具
● 場域
○ 溫室為主,一個溫室基本上只種一種作物
Phase 1 Phase 2 Phase 3 Phase 4
- 11. Problem Understanding
● 目標物種
○ 小果番茄 - 玉女番茄
● 場域與栽種方式
○ 溫室 / 掛網式 (走道寬度/目標物件高度)
● USB Camera
○ 解析度 / F0V(影響目標物件在影像中的成像大小 )
● 判斷標準(採收標準/採收方式)
● 運行平台
○ RPi+Intel Movidius NCS
source: https://paper.dropbox.com/doc/FBTUG--ASl58MS7PkuY4SIYoHLxJXHoAg-QdMWCaBBoyudOTNuvhd4c
Phase 1 Phase 2 Phase 3 Phase 4
- 13. Data Processing
● 資料特性
○ 目標物種
■ 小果番茄 - 玉女番茄
○ 標記類別
■ 0_pepper_flower 花朵
■ 1_pepper_young 尚未成熟果實
■ 2_pepper_matured 成熟果實可摘採
■ 3_pepper_covered 成熟果實需撥葉摘採
○ 判斷標準與注意事項(from domain expert)
Phase 1 Phase 2 Phase 3 Phase 4
source https://paper.dropbox.com/doc/FBTUG--ASl58MS7PkuY4SIYoHLxJXHoAg-QdMWCaBBoyudOTNuvhd4c
- 14. Data Processing
● 蒐集照片
○ 活動紀錄
■ 基本資訊(人事時地物)
■ 作物資料
■ 相機資料
■ 拍攝方式
source
活動紀錄 https://paper.dropbox.com/doc/FBTUG-FarmHarvestBot--ASkl43eqEeSkI3g~dcTtYHVWAg-XHZpHOCR4Tw6amCxMY0E0
資料蒐集紀錄表https://paper.dropbox.com/doc/JsCHkjEGzSIAvT65J0fc4
Phase 1 Phase 2 Phase 3 Phase 4
- 15. Data Processing
● 資料標記
○ 工具
■ ImageLabel (offline)
■ Datatrunk (online)
○ 輸出格式
■ PASCAL VOC Format(XML)
source
https://drive.google.com/file/d/1WfdacRUEJ1HgIlQsimbMVLVe2GlKsKJS/view
https://dataturks.com/projects/kobeyu/FarmHarvestBot-tomato
Phase 1 Phase 2 Phase 3 Phase 4
- 17. Data Processing
● 資料轉換
○ VOC to YOLO
○ VOC to tfrecord
■ create_pascal_tf_record.py
source
https://github.com/tensorflow/models/blob/master/research/object_detection/dataset_tools/create_pascal_tf_record.
py
Phase 1 Phase 2 Phase 3 Phase 4
- 22. Model Processing
● 選擇網路
○ 準確度 v.s 執行速度
● 回顧Phase1
○ 運行平台
■ RPi + Intel Movidius NCS
○ 實測結果
■ SSD-Mobilenet(5.5 FPS)
■ Tiny-yolo v2(0.7 FPS)
reference
https://github.com/movidius/ncappzoo
https://www.youtube.com/watch?v=llBhBSgoWPs
Phase 1 Phase 2 Phase 3 Phase 4
- 24. Mobilenet
● Separate convolutions
○ Depthwise + pointwise convolution
● Multiplier
○ α:Width Multiplier (0.25、0.5、0.75、1)
○ ρ:Resolution Multiplier (128、160、192、224)
source:
https://yinguobing.com/bottlenecks-block-in-mobilenetv2/
https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.md
https://arxiv.org/abs/1704.04861
Phase 1 Phase 2 Phase 3 Phase 4
- 25. Mobilenet
輸入資料: W_in * H_in * Nch,Kernel Map: k * k * Nk,輸出資料: W_out * H_out * Nk
一般卷積 計算量為 W_in * H_in * Nch* k * k * Nk
Depthwise separable convolution計算量為 (Depthwise convolution計算量+Pointwise convolution計算量)
Depthwise convolution計算量為 W_in * H_in * Nch* k * k
Pointwise convolution計算量為 Nch*Nk * W_in * H_in
source
https://medium.com/@chih.sheng.huang821/%E6%B7%B1%E5%BA%A6%E5%AD%B8%E7%BF%92-mobilenet
-depthwise-separable-convolution-f1ed016b3467
- 26. Mobilenet
Million MACs Million Parameters Top-1 Accuracy Top-5 Accuracy
MobileNet_v1_1.0_224 569 4.24 70.9 89.9
MobileNet_v1_1.0_192 418 4.24 70.0 89.2
MobileNet_v1_1.0_160 291 4.24 68.0 87.7
MobileNet_v1_1.0_128 186 4.24 65.2 85.8
MobileNet_v1_0.75_224 317 2.59 68.4 88.2
MobileNet_v1_0.50_224 150 1.34 63.3 84.9
MobileNet_v1_0.25_224 41 0.47 49.8 74.2
MobileNet_v1_1.0_224_quant 569 4.24 70.1 88.9
MobileNet_v1_0.25_128_quant 14 0.47 39.5 64.4
Phase 1 Phase 2 Phase 3 Phase 4
- 29. Model Processing
● Model
○ SSD+Mobilenetv1
○ SSDLite+Mobilenetv2
○ yolo
● Dataset (Image + PASCAL VOC)
○ Farmharvestbot
■ 小果番茄
■ 彩椒
○ MS COCO
Phase 1 Phase 2 Phase 3 Phase 4
- 30. Model Training
● Model
○ SSD-Mobilenetv1
● Dataset
○ MS COCO
● Training framework
○ Tensorflow + Tensorflow/Models/research/object_detection
● Script
python object_detection/model_main.py
--pipeline_config_path=./object_detection/ssd_mobilenetv1/model/ssd_mobilenet_v1_pets.config
--model_dir=./object_detection/ssd_mobilenetv1/model/
--num_train_steps=200000
--num_eval_steps=8000
--alsologtostderr
source https://github.com/kobe0308/models.git (branch develop)
Phase 1 Phase 2 Phase 3 Phase 4
- 31. Model Processing
● Model
○ SSD+Mobilenetv1
● Dataset
○ MS COCO
● Training parameters
○ batch size:24
○ steps: 200K
○ classes:20
Phase 1 Phase 2 Phase 3 Phase 4
- 32. Model Processing
● Model
○ SSD+Mobilenetv1
● Dataset
○ Farmharvestbot-tomato
● Training parameters
○ batch size:24
○ steps: 200K
○ classes:4
Phase 1 Phase 2 Phase 3 Phase 4
- 33. Model Processing
● Model
○ SSD+Mobilenetv1
● Dataset
○ Farmharvestbot-pepper
● Training parameters
○ batch size:24
○ steps: 200K
○ classes:4
Phase 1 Phase 2 Phase 3 Phase 4
- 34. Deploy
● Edge compute (compare with client/server)
● Platform(RPi+Neural Compute Stick)
■ SBC(RPi 3 Model B)
● 1.4GHz Quad-Core ARM Cortex-A53 (64Bit)
■ NCS
● Intel Movidus NCS
○ 12 Shave Core(VLIW Arch)
○ 1 TOPs
● Intel Movidus NCS2
○ 16 Shave Core(VLIW Arch)
○ Dedicated Neural Compute Engine
○ 4 TOPs
● Orange Pi AI Stick 2801
○ Matrix Processing Engine
○ 9.3 TOps
source https://qconsf.com/sf2017/system/files/presentation-slides/qconf_presentation.pdf
Phase 1 Phase 2 Phase 3 Phase 4
- 35. NN+Vision Pipeline
● Recap problem understanding
○ Realtime!
○ Capture image
■ resolution @30fps
○ Acceleactor
■ NN inference - NCS
■ Image Processing - CPU(Multi-thread + SIMD/ARM compute library)
○ Programming language
■ Python or C++
Image
Preprocess
Capture
Image
NN
Inference
Image
Postprocess
source https://www.anandtech.com/show/7591/answered-by-the-experts-arms-cortex-a53-lead-architect-peter-greenhalgh
Phase 1 Phase 2 Phase 3 Phase 4
- 36. Movidius Toolkit
NCSDK
● Compiler (from trained nn to inference engine)
● Checker
● Profiler (profile performance in each nn layer)
● NCSAPI(C++)
source https://qconsf.com/sf2017/system/files/presentation-slides/qconf_presentation.pdf
Phase 1 Phase 2 Phase 3 Phase 4
- 37. Movidius Toolkit
● Command
○ mvNCCompile -s <number of shaves> <name of the model>_frozen.pb -in=input
-on=<name of the output node>
● Support training framework
○ Tensorflow
○ Caffe
Phase 1 Phase 2 Phase 3 Phase 4
source https://movidius.github.io/ncsdk/tf_modelzoo.html
- 38. Movidius Architecture
● SHAVE core
○ 8 parallel function units
■ predicated execution unit (PEU)
■ branch and repeat unit (BRU)
■ two 64-bit load-store units (LSU0/LSU1)
■ 32-bit integer arithmetic unit (IAU)
■ 32-bit scalar arithmetic unit (SAU)
■ 128-bit vector arithmetic unit(VAU)
■ 128-bit compare move unit (CMU)
source https://en.wikichip.org/wiki/movidius/microarchitectures/shave_v2.0
Phase 1 Phase 2 Phase 3 Phase 4
- 39. Single Issue RISC v.s VLIW
source https://slideplayer.com/slide/4896590/
Phase 1 Phase 2 Phase 3 Phase 4
- 41. Robot Arm and Camera Calibration
source https://slideplayer.com/slide/7491989/
- 43. Farmharvestbot
FB: FarmBot Taiwan User Group-FBTUG
Doc: https://paper.dropbox.com/doc/FBTUG-FarmHarvestBot--ASlFVrNQa1XWOCWxktW_~JkgAg-w2FKkhc4ZTlj6knhOK43p
專案成員: https://paper.dropbox.com/doc/FBTUG-FarmHarvestBot--ASkpEA1~UFD9CQDm4hhPaWKMAg-9q9yQmaWjhkgh1EdB4EBc