# Microsoft Azure Machine Learning Monitoring

- [Microsoft Azure Machine Learning - An Overview](https://www.manageengine.com/products/applications_manager/help/azure-machine-learning-monitoring-tools.html#azuremloveriew)
- [Creating a new Microsoft Azure Machine Learning Monitor](https://www.manageengine.com/products/applications_manager/help/azure-machine-learning-monitoring-tools.html#newmonitor)
- [Monitored Parameters](https://www.manageengine.com/products/applications_manager/help/azure-machine-learning-monitoring-tools.html#parameters)

## Microsoft Azure Machine Learning - An Overview

Azure Machine Learning is a cloud platform for training, deploying, and managing machine learning models at scale. Applications Manager's [Azure Machine Learning monitoring](https://www.manageengine.com/products/applications_manager/azure-machine-learning-monitoring.html) provides visibility into CPU and GPU utilization, memory, disk I/O, workspace run lifecycle, cluster node and core allocation, model deployment and registration activity, InfiniBand network traffic, and AI agent metrics.

## Creating a new Microsoft Azure Machine Learning Monitor

To learn how to create a new Microsoft Azure Machine Learning monitor, [click here](https://www.manageengine.com/products/applications_manager/help/microsoft-azure.html#newmonitor).

## Monitored Parameters

Navigate to the **Category View** by clicking the **Monitors** tab. Hover over 'Child Monitors' under **Microsoft Azure** in the Cloud Apps table, and then select the **Machine Learning** monitor from the displayed tooltip. This action will display the bulk configuration view for Microsoft Azure Machine Learning in three tabs:

- **Availability** tab gives the availability history for the past 24 hours or 30 days.
- **Performance** tab gives the health status and events for the past 24 hours or 30 days.
- **List view** tab enables you to perform [bulk admin configurations](https://www.manageengine.com/products/applications_manager/help/bulk-config.html).

Following are the list of metrics monitored in [Microsoft Azure Machine Learning Monitoring](https://www.manageengine.com/products/applications_manager/help/azure-machine-learning-monitoring-tools.html) in their corresponding tabs:

- [Performance Overview](https://www.manageengine.com/products/applications_manager/help/azure-machine-learning-monitoring-tools.html#performanceoverview)
- [Workspace Runs](https://www.manageengine.com/products/applications_manager/help/azure-machine-learning-monitoring-tools.html#workspaceruns)
- [Nodes & Cores](https://www.manageengine.com/products/applications_manager/help/azure-machine-learning-monitoring-tools.html#nodescores)
- [Network & Models](https://www.manageengine.com/products/applications_manager/help/azure-machine-learning-monitoring-tools.html#networkmodels)
- [AI Agent](https://www.manageengine.com/products/applications_manager/help/azure-machine-learning-monitoring-tools.html#aiagent)
- [Configuration](https://www.manageengine.com/products/applications_manager/help/azure-machine-learning-monitoring-tools.html#configuration)

### Performance Overview

| Parameter | Description |
|---|---|
| **CPU USAGE** |  |
| CPU Usage | The average of utilization of a CPU node in millicores for the Azure Machine Learning at the time of polling (in millicores). |
| CPU Capacity Usage | The average of capacity of a CPU node in millicores for the Azure Machine Learning at the time of polling (in millicores). |
| **CPU MEMORY** |  |
| CPU Memory Capacity Usage | The average of maximum memory utilization of a CPU node for the Azure Machine Learning at the time of polling (in MB). |
| CPU Memory Usage | The average of memory utilization of a CPU node for the Azure Machine Learning at the time of polling (in MB). |
| **CPU UTILIZATION** |  |
| CPU Utilization | The average of utilization on a CPU node for the Azure Machine Learning at the time of polling (in %). |
| CPU Memory Utilization | The average of memory utilization of a CPU node for the Azure Machine Learning at the time of polling (in %). |
| **GPU UTILIZATION** |  |
| GPU Utilization | The average utilization on a GPU node for the Azure Machine Learning at the time of polling (in %). |
| GPU Memory Utilization | The average memory utilization of a GPU device for the Azure Machine Learning at the time of polling (in %). |
| **GPU MEMORY** |  |
| GPU Memory Capacity Usage | The average of memory capacity of a GPU device for the Azure Machine Learning at the time of polling (in MB). |
| GPU Memory Usage | The average of memory utilization of a GPU device for the Azure Machine Learning at the time of polling (in MB). |
| **GPU USAGE** |  |
| GPU Usage | The average of utilization of a GPU device in milli-GPUs for the Azure Machine Learning at the time of polling (in milli-GPUs). |
| GPU Capacity | The average capacity of a GPU device in milli-GPUs of the Azure Machine Learning at the time of polling (in milli-GPUs). |
| **GPU ENERGY** |  |
| GPU Energy | The total number of interval energy in Joules on a GPU node for the Azure Machine Learning between the poll intervals. |
| **BLOB STORAGE API CALLS** |  |
| Storage API Success | The total number of Azure Blob Storage API calls success in the Azure Machine Learning between the poll intervals. |
| Storage API Failure | The total number of Azure Blob Storage API calls failure in the Azure Machine Learning between the poll intervals. |
| **DISK USAGE** |  |
| Disk Available | The average of available disk space for the Azure Machine Learning at the time of polling (in MB). |
| Disk Used | The average of disk space used by the Azure Machine Learning at the time of polling (in MB). |
| Disk Read | The average of data read from disk for the Azure Machine Learning at the time of polling (in MB). |
| Disk Write | The average of data written into disk for the Azure Machine Learning at the time of polling (in MB). |
| **QUOTA UTILIZATION** |  |
| Quota Utilization | The average of quota utilized for the Azure Machine Learning workspace at the time of polling (in %). |

### Workspace Runs

| Parameter | Description |
|---|---|
| **RUN STATUS** |  |
| Completed Runs | The total number of runs completed successfully for this workspace in the Azure Machine Learning between the poll intervals. |
| Failed Runs | The total number of runs failed for this workspace in the Azure Machine Learning between the poll intervals. |
| Cancelled Runs | The total number of runs cancelled for this workspace in the Azure Machine Learning between the poll intervals. |
| Started Runs | The total number of runs running for this workspace in the Azure Machine Learning between the poll intervals. |
| Cancel Requested Runs | The total number of runs where cancel was requested for this workspace in the Azure Machine Learning between the poll intervals. |
| Provisioning Runs | The total number of runs that are provisioning in the Azure Machine Learning between the poll intervals. |
| **ACTIVE RUNS** |  |
| Preparing Runs | The total number of runs that are preparing for this workspace in the Azure Machine Learning between the poll intervals. |
| Finalizing Runs | The total number of runs entered finalizing state for this workspace in the Azure Machine Learning between the poll intervals. |
| Starting Runs | The total number of runs started for this workspace in the Azure Machine Learning between the poll intervals. |
| Queued Runs | The total number of runs that are queued for this workspace in the Azure Machine Learning between the poll intervals. |
| Not Responding Runs | The total number of runs not responding for this workspace in the Azure Machine Learning between the poll intervals. |
| Not Started Runs | The total number of runs in Not Started state for this workspace in the Azure Machine Learning between the poll intervals. |
| **ERRORS AND WARNINGS** |  |
| Errors | The total number of run errors in this workspace for the Azure Machine Learning between the poll intervals. |
| Warnings | The total number of run warnings in this workspace in the Azure Machine Learning between the poll intervals. |

**Note:** Workspace Runs metrics are mapped under Performance Polling. To adjust the polling interval, navigate to **Settings** → **Performance Polling**. In the **Optimize Data Collection** tab, select **Azure Machine Learning** as the Monitor Type and **Workspace Runs** as the Metric Name. Then set Default Polling Status as required, by default opted as Never collect data.

### Nodes & Cores

| Parameter | Description |
|---|---|
| **NODES** |  |
| Total Nodes | The total number of nodes includes some of Active Nodes, Idle Nodes, Unusable Nodes, Preempted Nodes, Leaving Nodes in the Azure Machine Learning between the poll intervals. |
| Active Nodes | The total number of active nodes which are actively running a job in the Azure Machine Learning between the poll intervals. |
| Idle Nodes | The total number of idle nodes which are not running any jobs but can accept new job if available in the Azure Machine Learning between the poll intervals. |
| Leaving Nodes | The total number of leaving nodes which just finished processing a job and will go to Idle state in the Azure Machine Learning between the poll intervals. |
| Unusable Nodes | The total number of unusable nodes which are not functional due to some unresolvable issue in the Azure Machine Learning between the poll intervals. |
| Preempted Nodes | The total number of preempted nodes are the low priority nodes which are taken away from the available node pool in the Azure Machine Learning between the poll intervals. |
| **CORES** |  |
| Total Cores | The total number of cores in the Azure Machine Learning between the poll intervals. |
| Active Cores | The total number of active cores in the Azure Machine Learning between the poll intervals. |
| Idle Cores | The total number of idle cores in the Azure Machine Learning between the poll intervals. |
| Unusable Cores | The total number of unusable cores in the Azure Machine Learning between the poll intervals. |
| Preempted Cores | The total number of preempted cores in the Azure Machine Learning between the poll intervals. |
| Leaving Cores | The total number of leaving cores in the Azure Machine Learning between the poll intervals. |

**Note:** Nodes & Cores metrics are mapped under Performance Polling. To adjust the polling interval, navigate to **Settings** → **Performance Polling**. In the **Optimize Data Collection** tab, select **Azure Machine Learning** as the Monitor Type and **Nodes & Cores** as the Metric Name. Then set Default Polling Status as required, by default opted as Never collect data.

### Network & Models

| Parameter | Description |
|---|---|
| **NETWORK TRAFFIC** |  |
| Network Input | The total amount of Network data received for the Azure Machine Learning workspace between the poll intervals (in MB). |
| Network Output | The total amount of Network data sent for the Azure Machine Learning workspace between the poll intervals (in MB). |
| **INFINIBAND TRAFFIC** |  |
| InfiniBand Received | The total amount of network data received over InfiniBand of the Azure Machine Learning between the poll intervals (in MB). |
| InfiniBand Sent | The total amount of network data sent over InfiniBand of the Azure Machine Learning between the poll intervals (in MB). |
| **MODEL STATUS** |  |
| Model Deploy Succeeded | The total number of model deployments that succeeded in this workspace in the Azure Machine Learning between the poll intervals. |
| Model Deploy Failed | The total number of model deployments that failed in this workspace in the Azure Machine Learning between the poll intervals. |
| Model Deploy Started | The total number of model deployments started in this workspace in the Azure Machine Learning between the poll intervals. |
| Model Register Succeeded | The total number of model registrations that succeeded in this workspace in the Azure Machine Learning between the poll intervals. |
| Model Register Failed | The total number of model registrations that failed in this workspace in the Azure Machine Learning between the poll intervals. |

**Note:** Network & Models metrics are mapped under Performance Polling. To adjust the polling interval, navigate to **Settings** → **Performance Polling**. In the **Optimize Data Collection** tab, select **Azure Machine Learning** as the Monitor Type and **Network & Models** as the Metric Name. Then set Default Polling Status as required, by default opted as Never collect data.

### AI Agent

| Parameter | Description |
|---|---|
| **AGENTS** |  |
| Agents | The total number of events for AI Agents in the Azure Machine Learning between the poll intervals. |
| **RUNS** |  |
| Runs | The total number of runs by AI Agents in this workspace in the Azure Machine Learning between the poll intervals. |
| **TOKENS** |  |
| Tokens | The total number of tokens by AI Agents in the Azure Machine Learning between the poll intervals. |
| **THREADS** |  |
| Threads | The total number of events for AI Agent threads in the Azure Machine Learning between the poll intervals. |
| **MESSAGES** |  |
| Messages | The total number of events for AI Agent messages in the Azure Machine Learning between the poll intervals. |
| **INDEXED FILES** |  |
| Indexed Files | The total number of files indexed for file search in the Azure Machine Learning between the poll intervals. |
| **TOOLBOX CALLS** |  |
| Toolbox Calls | The total number of tool calls made by AI Agents in this workspace in the Azure Machine Learning between the poll intervals. |
| Toolbox Call Tool Calls | The total number of toolbox call_tool call events in the Azure Machine Learning between the poll intervals. |
| **TOOLBOX TOOLS** |  |
| Toolbox Tools Deferred | The total number of tools deferred by Tool Search in toolbox tools/list responses in the Azure Machine Learning between the poll intervals. |
| Toolbox Tools Visible | The total number of tools visible to the model in toolbox tools/list responses in the Azure Machine Learning between the poll intervals. |
| **TOOLBOX SEARCH** |  |
| Toolbox Tool Search Calls | The total number of toolbox tool_search call events in the Azure Machine Learning between the poll intervals. |
| Toolbox Tool Search Enabled | The total number of tool Search-enabled toolbox version creation events in the Azure Machine Learning between the poll intervals. |
| Toolbox Tool Search Results | The total number of tools returned by successful toolbox tool_search calls in the Azure Machine Learning between the poll intervals. |

**Note:** AI Agent metrics are mapped under Performance Polling. To adjust the polling interval, navigate to **Settings** → **Performance Polling**. In the **Optimize Data Collection** tab, select **Azure Machine Learning** as the Monitor Type and **AI Agent** as the Metric Name. Then set Default Polling Status as required, by default opted as Never collect data.

### Configuration

| Parameter | Description |
|---|---|
| **GENERAL CONFIGURATION** |  |
| Resource Group Name | The name of the Azure resource group that contains the Machine Learning workspace. |
| Location | The Azure region where the Machine Learning workspace is deployed. |
| Provisioning State | The provisioning state of the Machine Learning workspace. Possible values: Unknown, Updating, Creating, Deleting, Succeeded, Failed, Canceled. |
| SKU Name | The SKU name associated with the Machine Learning workspace. |
| Kind | The kind of Machine Learning workspace. |
| Public Network Access | Indicates whether public network access is enabled for the Machine Learning workspace. Possible values: Enabled, Disabled. |
| **RESOURCE METADATA** |  |
| Creation Time | The timestamp when the Machine Learning workspace was created. |
| Created By | The identity that created the Machine Learning workspace. |
| Creator Identity Type | The type of identity that created the workspace. Possible values: User, Application, ManagedIdentity, Key. |
| Last Modified Time | The timestamp when the Machine Learning workspace was last modified. |
| Last Modified By | The identity that last modified the Machine Learning workspace. |
| Last Modifier Identity Type | The type of identity that last modified the workspace. Possible values: User, Application, ManagedIdentity, Key. |
| **SECURITY AND NETWORK** |  |
| Allow Role Assignment on RG | Indicates whether role assignments can be created at the resource group level for the Machine Learning workspace. |
| HBI Workspace | Indicates whether the workspace is designated as an HBI (High Business Impact) workspace. |
| Data Isolation | Indicates whether data associated with the workspace is isolated. |
| Service Side CMK Encryption | Indicates whether service-side Customer Managed Key (CMK) encryption is enabled for the workspace. Possible values: Enabled, Disabled. |
| Software Bill of Materials | Indicates whether Software Bill of Materials (SBOM) generation is enabled for the workspace. |
| **WORKSPACE SETTINGS** |  |
| Provision Network Now | Indicates whether the managed virtual network is provisioned immediately. |
| Soft Delete | Indicates whether soft delete is enabled for the workspace. |
| Storage HNS | Indicates whether the storage associated with the workspace has hierarchical namespace (HNS) enabled. |
| System Datastores Auth Mode | The authentication mode used to access the workspace's system datastores. |
| V1 Legacy Mode | Indicates whether V1 legacy mode is enabled for the workspace. |