After a model has been deployed, it's important to understand how the model is being used in production, and to detect any degradation in its effectiveness due to data drift. This module describes tech- niques for monitoring models and their data.

1. Monitor Azure
Machine Learning

2. Eng Teong Cheah
Microsoft MVP

3. Monitor Azure Machine Learning
When you have critical applications and business processes relying on Azure resources,
you want to monitor those resources for their availability, performance, and operation.
This article describes the monitoring data generated by Azure Machine Learning and how
to analyze and alert on this data with Azure Monitor.

4. What is Azure Monitor?
Azure Machine Learning creates monitoring data using Azure Monitor, which is a full
stack monitoring service in Azure. Azure Monitor provides a complete set of features to
monitor your Azure resources. It can also monitor resources in other clouds and on-
premises.

5. Monitor data from Azure Machine Learning
Azure Machine Learning collects the same kinds of monitoring data as other Azure
resources that are described in Monitoring data from Azure resources.

6. Collection and routing
Platform metrics and the Activity log are collected and stored automatically, but can be
routed to other locations by using a diagnostic setting.
Resource Logs are not collected and stored until you create a diagnostic setting and route
them to one or more locations. When you need to manage multiple Azure Machine
Learning workspaces, you could route logs for all workspaces into the same logging
destination and query all logs from a single place.

7. Analyzing metrics
You can analyze metrics for Azure Machine Learning, along with metrics from other Azure
services, by opening Metrics from the Azure Monitor menu. See Getting started with
Azure Metrics Explorer for details on using this tool.

8. Filtering and splitting
For metrics that support dimensions, you can apply filters using a dimension value. For
example, filtering Active Cores for a Cluster Name of cpu-cluster.
You can also split a metric by dimension to visualize how different segments of the metric
compare with each other. For example, splitting out the Pipeline Step Type to see a
count of the types of steps used in the pipeline.

9. Transparency
When AI systems help inform decisions that have tremendous impacts on people's lives,
it's critical that people understand how those decisions were made. For example, a bank
might use an AI system to decide whether a person is creditworthy. A company might use
an AI system to determine the most qualified candidates to hire.
A crucial part of transparency is interpretability: the useful explanation of the behavior of
AI systems and their components. Improving interpretability requires stakeholders to
comprehend how and why AI systems function the way they do. The stakeholders can
then identify potential performance issues, fairness issues, exclusionary practices, or
unintended outcomes.

10. Analyzing logs
Using Azure Monitor Log Analytics requires you to create a diagnostic configuration and
enable Send information to Log Analytics.

11. References
Microsoft Docs