from ai.backend import python @ pycontw2018
•
4 recomendaciones•889 vistas
Title: from ai.backend import python Author: Joe Tseng from Umbo CV
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a from ai.backend import python @ pycontw2018
Similar a from ai.backend import python @ pycontw2018 (20)
Más de Chun-Yu Tseng
Más de Chun-Yu Tseng (19)
Último
Último (20)
from ai.backend import python @ pycontw2018
- 1. Joe | AI Engineer
- 2. from ai.backend import python A (backend → AI) story and several thoughts about the future
- 3. Joe Tseng • Backend (AI) engineer • Tainan.py - organizer • PyConTW 2015/2016 - speaker
- 4. Outline About the story Umbo Computer Vision intro Develop and maintain CV services Build our machine learning pipelines What I have learned About the future My imagination
- 5. Outline About the story ➡ Umbo Computer Vision intro Develop and maintain CV services Build our machine learning pipelines What I have learned About the future My imagination
- 6. Autonomous video security systems
- 7. 400 million cameras ref: UK Police Scientific and Development Branch (PSDB) 12 or less cameras per operator - UK Police Scientific Department
- 8. Our Solution Existing cameras Umbo Learning Cameras Real-time alerts Umbo Light
- 9. Infra We use a lot of cloud services and have our own GPU clusters
- 10. Backend Languages All our AI team members speak Python ❤ …
- 11. Flow ./enter_dev_env.sh + write code with unit tests + code review + CI/CD
- 12. Monitoring We log, monitor and measure our services
- 13. Teams Hardware, Dev, CV, Infra, Research, PoC, QA, FAE …
- 14. Develop and maintain all computer vision and machine learning features The definition of an AI Engineer
- 15. 1. Develop and maintain CV services 2. Build our machine learning pipelines Our jobs
- 16. Backend → AI How ?
- 17. Outline About the story Umbo Computer Vision intro ➡ Develop and maintain CV services Build our machine learning pipelines What I have learned About the future My imagination
- 18. • On production in 2016 • From PoC projects to online services Previously on SmartCloud and Umbo Light
- 19. The Simplified CV Architecture Media Server CV Forwarder Load Balancer ZMQ Stream Router Stream Manager Stream Manager Stream Manager Load Balancer CV Worker CV Worker CV Worker HTTP Alert Endpoint
- 20. Where is AI?
- 23. How much can the backend do?
- 24. 87% How much the backend can do
- 25. • Infra • LB / ASG / VPC … • Monitoring • PaperTrail, Cloud Watch … • Development • Add new features • CI/CD • Maintenance • Upgrade packages • Refactor • Debug 87% MS CF LB SR SM SM SM LB CW CW CW AE
- 26. • Infra • LB / ASG / VPC … • Monitoring • PaperTrail, Cloud Watch … • Development • Add new features • CI/CD • Maintenance • Upgrade packages • Refactor • Debug 87% MS CF LB SR SM SM SM LB CW CW CW AE
- 27. • Stream Router/Manager was written by Python2 ❤ ❤ • Python2 => Python3 • byte string / decode • isinstance • format • API changes • Results • Python3 ❤ ❤ ❤ • Enable a lot of cool features • F-strings, typing, asyncio, tracemalloc, etc. 87%
- 28. • Refactor Stream Manager • Happy to use async/await • Use pipeline pattern • Still use run_in_executor/ ThreadPoolExecutor if needed • Results • Cleaner architecture • Performance boost 87%
- 29. • Refactor CV Worker • Lua => Python • Torch => PyTorch • Use aiohttp server • Results • Easier to maintain/upgrade • High performance • GPU resources are the bottleneck • Python package ecosystem 👍 87%
- 30. • Debug • Use pyflame to profile the program • Use tracemaclloc to find memory usage • Use Valgrind to check your C++ code • Results • Understand more and gotcha! 87%
- 31. Ping me if you want to listen this story (and also give us a talk) in Tainan.py 😼
- 32. 13% How much the backend cannot handle
- 33. • About Machine Learning • Not so easy to learn 13%
- 34. More than 13% In fact
- 35. • Computer Vision and Machine Learning • They are everywhere • The learning curve is steep More than 13% MS CF LB SR SM SM SM LB CW CW CW AE
- 36. You need to delve really deep into Computer Vision and Machine Learning to build a self-improving system
- 37. Outline About the story Umbo Computer Vision intro Develop and maintain CV services ➡ Build our machine learning pipelines What I have learned About the future My imagination
- 38. Why machine learning pipelines? Improve services => customers 😆
- 39. How to improve your services Measure first New models / heuristics approaches Need to do training or redesigning Need more (or specific) labeled data Training! Need to compare with the existing ones Get the best on production … Measure again 👷
- 41. • Meet Michelangelo: Uber’s Machine Learning Platform • Machine Learning Pipeline for Real- Time Forecasting @Uber Marketplace Michelangelo (Uber)
- 42. How much can backend do?
- 43. 0%~87%
- 44. Why 0%? 🐵 without domain knowledge
- 45. Why 87%? With domain knowledge + good communication with others + speaking the same language (Python!)
- 46. Catch a glimpse of our pipelines
- 47. • Can dispatch tasks to • Our own labelers • Third-party partners • MTurkers • Backend: • Flask + plugins • NumPy • Celery/Redis/Pymongo • uWSGI Labeling Platform source: https://github.com/nightrome/cocostuff
- 48. • Can train you a lot of new models with different parameters • Backend: • Airflow • Celery Training Pipeline
- 49. • Can visualize • The ground truth data • The evaluation results • Backend: • Flask + plugins • NumPy • SQLAlchemy Visualization Platform
- 50. A faster pipeline means a shorter turnaround time! Speed matters for a startup
- 51. Outline About the story Umbo Computer Vision intro Develop and maintain CV services Build our machine learning pipelines ➡ What I have learned About the future My imagination
- 54. Domain Knowledge Have a basic understanding of ML/CV with practical experience
- 56. Outline About the story Umbo Computer Vision intro Develop and maintain CV services Build our machine learning pipelines What I have learned About the future ➡ My imagination
- 57. from ai.backend import * Python, Golang, Java, C#, Javascript, PHP, Ruby, etc.
- 58. To build your pipelines, Python is the best default language Pipeline → Service
- 59. Every young engineer knows machine learning But some of them may have not heard of GIL 😥
- 60. High-quality (ai) backends are needed
- 61. Thank you
- 62. • Backend, CV, Front-end, App, QA, FAE, Firmware • joe.tseng@umbocv.com We are hiring