Azure Custom Vision allows you to build an image classifier that adjusts to your needs without needing an AI/ML background. In this session we will learn about Custom Vision and how you can train an image classifier model using its .NET SDK. A published model can then be exported to a variety of formats, such as TensorFlow and CoreML, which can perfectly be integrated into Android & iOS mobile apps with object recognition capabilities. In this session, the Custom Vision service will be explored and demonstrated with: a) a trained model using the COIL-100 image dataset and the Custom Vision .NET SDK. b) A mobile image classifier app which makes use of the model in a) in both online and offline scenarios

1. #GlobalAzureVirtual
Global Azure Virtual 2020 UK & Ireland
Create an image classifier
with Azure Custom Vision
.NET SDK
Luis Beltrán

2. Speaker Intro
#GlobalAzureVirtual
• Researcher at Tomas Bata University in Zlín, Czech Republic.
• Lecturer at Tecnológico Nacional de México in Celaya, Mexico.
• Passionate about Xamarin, Azure & AI

3. Agenda
#GlobalAzureVirtual
• What is Image Classification?
• Azure Cognitive Services
• Custom Vision service:
• Knowing the service
• Building an Image Classifier
• Exporting the model
• Custom Vision .NET SDK Demo
• Offline image classification demo with TensorFlow, CoreML and Xamarin
• Final thoughts
Download the slides: https://bit.ly/LuisCustomVisionGA

4. What is Image Classification?
#GlobalAzureVirtual
• The identification of the visual content in an
image.
• A topic of interest in the Computer Vision
area.
• A trivial task for human beings… quite a
challenge for computer applications

5. #GlobalAzureVirtual
Dog 0.02
Cat 0.98
Dog 0.99
Cat 0.01

6. Azure Cognitive Services
#GlobalAzureVirtual
Computer Vision
Face/Emotion Recognition
OCR/Handwriting
Custom Vision
Video Indexer
Text-to-Speech
Speech-to-Text
Translator
Custom Speech
Language Understanding
Text Translator
Text Analytics
QnA Maker
Bing Custom Search
Bing Visual Search
Bing Autosuggest
Bing Image Search
Bing News Search
microsoft.com/cognitive

7. Computer Vision
#GlobalAzureVirtual
Analyze an image
• Extract a rich set of visual features
based on the image content.
OCR
• Detect text in an image.
Generate thumbnails
• Scale and crop images, while retaining
key content.

8. Custom Vision Service
#GlobalAzureVirtual
Customize image recognition to fit your
business needs.
An AI service and end-to-end platform for
applying computer vision to your specific
scenario.

9. Building an Image Classifier
#GlobalAzureVirtual
• Create a Project
• Choose a domain
• Upload images
• Tag them
• Train the classifier
• Evaluate, publish & export
the model

10. Custom Vision .NET SDK
#GlobalAzureVirtual

11. Creating a project
#GlobalAzureVirtual
Custom Vision is organized hierarchically.
At the top level, there is a project, which
represents the data and model for a
specific task.
An image classifier is a model built with
Custom Vision Service using tagged
images.
Image Classifier = Project

12. #GlobalAzureVirtual

13. #GlobalAzureVirtual

14. Choosing a domain
#GlobalAzureVirtual
When you create a project, you select a
domain, which optimizes a classifier for a
specific set of objects in your images
Food
• Optimized for dishes you would see on a
restaurant menu
Landmark
• Optimized for recognizable landmarks,
both natural and artificial.
Retail
• Optimized for classifying images in a
shopping catalog/website.

15. #GlobalAzureVirtual

16. Uploading & tagging the images
#GlobalAzureVirtual
In order to create a high-precision
classifier, Custom Vision Service needs
several training images.
A training image is a picture of the image
that you want Custom Vision Service to
classify with a specific tag.

17. #GlobalAzureVirtual

18. #GlobalAzureVirtual

19. Training the classifier
#GlobalAzureVirtual
Every time the classifier is
(re)trained, a new iteration
of the model is created.

20. #GlobalAzureVirtual

21. Evaluate & publish the model
#GlobalAzureVirtual
After the model is trained, it can
be quickly tested with another
image, either local or from the
web. The evaluation uses the
most recently trained iteration.
If the model provides accurate
results, it can be published, which
allows the classifier:
• To be accessed via an HTTP
endpoint (or through the SDK)
for online image classification
(prediction).
• To be exported to a platform
for offline image classification.

22. #GlobalAzureVirtual

23. #GlobalAzureVirtual

24. #GlobalAzureVirtual

25. Deploy anywhere, from the
cloud to the edge!
#GlobalAzureVirtual
Run your models wherever you need them and according to your unique scenario &
requirements. Export your trained models to devices or to containers for low-latency
scenarios.

26. Exporting the model
#GlobalAzureVirtual

27. #GlobalAzureVirtual

28. #GlobalAzureVirtual

29. Demo: Custom Vision .NET SDK
#GlobalAzureVirtual
Open-source project available on GitHub: https://github.com/icebeam7/CustomVisionNetSDKDemo
Uploading training images
Project information
Adding tags to the project →

30. #GlobalAzureVirtual
Open-source project available on GitHub: https://github.com/icebeam7/CustomVisionNetSDKDemo
Evaluating the model with test images
Training the model

31. #GlobalAzureVirtual
Open-source project available on GitHub: https://github.com/icebeam7/CustomVisionNetSDKDemo
Exporting the model to different platforms

32. #GlobalAzureVirtual
Open-source project available on GitHub: https://github.com/icebeam7/CustomVisionNetSDKDemo
Project status in Custom Vision after using the app

33. #GlobalAzureVirtual
Open-source project available on GitHub: https://github.com/icebeam7/CustomVisionNetSDKDemo
Model exported to several platforms

34. Demo: Image classification with
Custom Vision and Xamarin
#GlobalAzureVirtual
Open-source project available on GitHub: https://github.com/icebeam7/MobileImageClassifierDemo

35. #GlobalAzureVirtual
Open-source project available on GitHub: https://github.com/icebeam7/MobileImageClassifierDemo

36. #GlobalAzureVirtual
Open-source project available on GitHub: https://github.com/icebeam7/MobileImageClassifierDemo

37. Final thoughts
#GlobalAzureVirtual
Optimization tips
• The best way to generate a quality classifier is
to add more varied tagged images (different
backgrounds, angles, object size, groups of
photos, and variants of types).
• Include images that truly represent what your
classifier will encounter in the real world.
• Thus, photos in context are better than photos
of objects in front of neutral backgrounds.
• Always train your model after you have
added new images.
• Use at least 30 images per tag.

38. Pricing
#GlobalAzureVirtual
https://azure.microsoft.com/en-us/pricing/details/cognitive-services/custom-vision-service/

39. Examples of Custom Vision scenarios
#GlobalAzureVirtual
Additional Scenarios
• Classify user submitted images to website
• Identifying elements – object counting, animal identification and lots more.
• Hazard detection/industrial safety – adding custom rules to videos
Production Line
Category Detection
Agriculture & Farming
Plant Health Detection
Categorize Items on
a Retail Website

40. Call to Action
#GlobalAzureVirtual
Custom Vision Service
https://azure.microsoft.com/en-us/services/cognitive-services/custom-vision-service/
Custom Vision Documentation
https://docs.microsoft.com/es-mx/azure/cognitive-services/custom-vision-
service/home
Microsoft Learn: Classify images with the Custom Vision service
https://docs.microsoft.com/en-us/learn/modules/classify-images-custom-vision/
Custom Vision and TensorFlow
https://docs.microsoft.com/es-mx/azure/cognitive-services/custom-vision-
service/export-model-python

41. Thank you for your attention!
#GlobalAzureVirtual

42. https://www.linkedin.com/in/luisantoniobeltran/
@darkicebeam
luis@luisbeltran.mx
https://github.com/icebeam7/
Let’s
connect!
#GlobalAzureVirtual