Deep Reality Simulation for Automated Poacher Detection with Mark Hamilton and Anand Raman
•
1 recomendación•532 vistas
We present a novel deep learning approach to create a robust object detection network for use in an infra-red, UAV-based, poacher recognition system. More specifically we have used Microsoft AirSim to generate thousands of hours of simulated drone footage in the African Savannah. We then used deep domain adaptation to translate our simulation into a form that is adversarially indistinguishable from real infrared drone footage. This yields a programmable data generator that can be used to dramatically improve the accuracy of algorithms without requiring expensive human curated annotations. Furthermore, we extend this work and contribute a photorealism extension to AirSim, automating much of the domain specific expertise needed for computer graphics work, and enabling the generation of limitless quantities of photorealistic data for use in reinforcement learning and autonomous vehicles.
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Deep Reality Simulation for Automated Poacher Detection with Mark Hamilton and Anand Raman
Similar a Deep Reality Simulation for Automated Poacher Detection with Mark Hamilton and Anand Raman (20)
Más de Databricks
Más de Databricks (20)
Último
Último (20)
Deep Reality Simulation for Automated Poacher Detection with Mark Hamilton and Anand Raman
- 1. Mark Hamilton, Microsoft Anand Raman, Microsoft Unsupervised Object Detection using the Azure Cognitive Services on Spark #SAISExp4
- 2. Hamilton and Raman, #SAISExp4 2
- 3. Hamilton and Raman, #SAISExp4 3
- 5. Classifying all 1.3 million images will take 20k hours Hamilton and Raman, #SAISExp4 5
- 6. Hamilton and Raman, #SAISExp4 6
- 7. Deep Learning • Brain-like algorithms trained with gradient descent • Has become a recent favorite because: – Spectacular performance in many domains – Quick training – Low memory – Large space of possible model architectures – Automatic differentiation software makes it very easy Hamilton and Raman, #SAISExp4 7
- 8. 8Hamilton and Raman, #SAISExp4 Tire Dog Scotty Gu Input Images Low Level Features Mid Level Features High Level Features Output Classes Filters from Zeiler + Fergus 2013
- 9. 9Hamilton and Raman, #SAISExp4 Input Images Low Level Features Mid Level Features Logistic Regression Output Classes Filters from Zeiler + Fergus 2013 Any SparkML Model Other Leopard Predictions
- 10. ML • High level library for distributed machine learning • More general than SciKit-Learn • All models have a uniform interface – Can compose models into complex pipelines – Can save, load, and transport models Load Data Tokenizer Term Hashing Logistic Reg Evaluate Pipeline Evaluate Load Data ● Estimator ● Transformer Hamilton and Raman, #SAISExp4 10
- 11. GPU/CPU acceleration Automated gradient calculations Flexible language for defining huge space of models State of the art performance and accuracy ONNX Runtime Large scale parallelism Fault tolerance Auto-scaling / elasticity High throughput streaming Data source + orchestrator agnostic Distributed Computing Hamilton and Raman, #SAISExp4 11 Deep Learning
- 12. Microsoft ML for Apache Spark • Open Source Distributed ML library • Unifies major computing paradigms and Microsoft tools into a single library and API – Python, Java, Scala, R – Batch, Streaming, Serving – Local, Static Cluster, Dynamic Cluster, Serverless – Databricks, HDI, AZTK, Kubernetes, … www.aka.ms/spark Hamilton and Raman, #SAISExp4 12 Scalable Deep Learning Stress Free Serving Distributed Microservices Fast Gradient Boosting CNTK on Spark Spark Serving HTTP on Spark LightGBM on Spark MMLSpark
- 13. Training a Deep Transfer Learner: Hamilton and Raman, #SAISExp4 13
- 14. Performance With Deep Featurization, Augmentation, and Temporal Ensembling Without Deep Featurization Accuracy 65.6% Accuracy 94.7% Hamilton and Raman, #SAISExp4 14
- 15. … Time 1 2 3 4 5 6 Time 1 2 3 4 5 6 Deep Network … Hamilton and Raman, #SAISExp4 15
- 16. Real Time monitoring with PowerBI Stream from Kafka / Cosmos OpenCV on Spark CNTK on Spark SparkML Logistic Reg Stream Results to PowerBI Camera Image Upload in Kyrgyzstan MMLSpark Contribution 27 Lines of Code ~1k imgs/s throughput on CPU ~3 second latencies Hamilton and Raman, #SAISExp4 16
- 17. Simple APIs matter Hamilton and Raman, #SAISExp4 17
- 18. 18Hamilton and Raman, #SAISExp4 Lightning Fast Web Services on Any Spark Cluster • Sub-millisecond latencies • Fully Distributed • Spins up in seconds • Same API as Batch and Streaming • Scala, Python, R and Java • Fully Open Source www.aka.ms/spark JIRA: SPARK-25350 Serving 113 0.9 0 50 100 150 Latency (ms) Spark Serving Spark Continuous Serving Announcing: 100x Latency Reduction with MMLSpark v0.14
- 19. Web Serving with the Same API: Hamilton and Raman, #SAISExp4 19
- 20. • 3 main modes: – server: 1 service on the head node – distributedServer: 1 service per executor – continuousServer: 1 service per partition • Built on top of Spark Streaming • Each worker keeps a running service, and a routing table • Use HTTP on Spark objects to represent requests and responses in Spark SQL Spark Serving Architecture Hamilton and Raman, #SAISExp4 20
- 21. Going Further: No Labels, No Problem Hamilton and Raman, #SAISExp4 21
- 22. Bing Powered Snow Leopard Detection Pipeline Bing Photo Search of “Snow Leopard” + Related terms Logistic Regression Add Class + Union + Distinct Truncated ResNet50 Cloud Service CNTK on Spark Key Bing Image Search of random nouns Spark ML Hamilton and Raman, #SAISExp4 22
- 23. Bing Image Search API Hamilton and Raman, #SAISExp4 23
- 24. Announcing: Intelligent and Distributed Microservices in SparkML www.aka.ms/spark Microsoft Cognitive Services + Bing Text Vision Face Hamilton and Raman, #SAISExp4 24
- 25. Celebrity Quote Analysis Hamilton and Raman, #SAISExp4 25
- 26. HTTP on Spark • Allows Spark users to call into any HTTP service • Leverage parallel networking of cluster • Allows Spark to work with microservice architectures • Mini-batching, asynchrony, and buffering supported • Statically typed and usable from Spark, PySpark, and SparklyR Hamilton and Raman, #SAISExp4 26
- 27. Model Interpretability with LIME Source: Marco Tulio Ribeiro et al. Hamilton and Raman, #SAISExp4 27
- 28. Model Interpretability with LIME Source: Marco Tulio Ribeiro et al. Hamilton and Raman, #SAISExp4 28
- 29. Announcing: LIME on • Elastic and Distributed • Works with any image classification model • Example notebook instructions at: – www.aka.ms/spark Hamilton and Raman, #SAISExp4 29
- 30. Using LIME to generate bounding boxes 30Hamilton and Raman, #SAISExp4
- 31. Transferring LIME’s Knowledge to FasterRCNN 31Hamilton and Raman, #SAISExp4 𝑚𝑎𝑠𝑘 = 𝑢𝑛𝑖𝑜𝑛 𝑠𝑢𝑝𝑒𝑟𝑝𝑖𝑥𝑒𝑙𝑠 𝑡𝑜𝑝 70% 𝑜𝑓 𝑤𝑒𝑖𝑔ℎ𝑡𝑠) 𝑥1, 𝑦1 = min 𝑚𝑎𝑠𝑘 𝑥 , min 𝑚𝑎𝑠𝑘 𝑦 𝑥2, 𝑦2 = max 𝑚𝑎𝑠𝑘 𝑥 , max(𝑚𝑎𝑠𝑘 𝑦)
- 32. Results 32Hamilton and Raman, #SAISExp4 Human Labels Unsupervised FRCNN Outputs Human Labels Unsupervised FRCNN Outputs
- 33. 33Hamilton and Raman, #SAISExp4
- 34. Next Steps: Hotspotter 34Hamilton and Raman, #SAISExp4 Source: HotSpotter - Patterned Species Instance Recognition
- 35. In Conclusion www.aka.ms/spark Contributions Welcome! github.com/Azure/mmlspark Hamilton and Raman, #SAISExp4 35 • Big Releases in v0.14: – Cognitive Services on Spark – Sub-Millisecond latency distributed web services – Distributed Model Interpretability • Easy batch, streaming apps, PowerBI dashboards, or RESTful web services • We aim to give all work back to the community! • Easy to Get Started on Databricks: – 16 Jupiter notebook guide MMLSpark v0.14
- 36. Thanks to • You all! • Microsoft AI Development Acceleration Program: Abhiram Eswaran, Ari Green, Courtney Cochrane, Janhavi Suresh Mahajan, Karthik Rajendran, Minsoo Thigpen, Casey Hong, Soundar Srinivasan • MMLSpark Team: Sudarshan Raghunathan, Ilya Matiach, Eli Barzilay, Tong Wen, Ben Brodsky • The Snow Leopard Trust: Rhetick Sengupta, Koustubh Sharma, Jeff Brown, Michael Despines • Microsoft: Joseph Sirosh, Lucas Joppa, WeeHyong Tok Get in touch: marhamil@microsoft.com MMLSpark Website: aka.ms/spark Hamilton and Raman, #SAISExp4 36