Microsoft Azure Data Factory (ADF) is a cloud-based data integration service that enables organizations to orchestrate and automate the movement and transformation of data across hybrid and cloud environments. Applications Manager’s Azure Data Factory Monitoring helps you track pipeline runs, activity performance, and failure trends to optimize data integration processes and ensure seamless workflow orchestration in your Azure environment.
To learn how to create a new Microsoft Azure Data Factory Monitor, click here.
Navigate to the Category View by clicking the Monitors tab. Hover over 'Child Monitors' under Microsoft Azure in the Cloud Apps table, and then select the Data Factories monitor from the displayed tooltip. The bulk configuration view will display metrics categorized under the following tabs:
The Microsoft Azure monitor provides a brief detail of the Azure Data Factory under the given subscription. Following are the list of metrics monitored in Azure Data Factory Monitoring in their corresponding tabs:
| Parameter | Description |
|---|---|
| SUCCEEDED PIPELINE RUNS | |
| Succeeded Pipeline Runs | The total number of pipeline runs that completed successfully between the poll interval. |
| FAILED PIPELINE RUNS | |
| Failed Pipeline Runs | The total number of pipeline runs that failed execution between the poll interval. |
| CANCELLED PIPELINE RUNS | |
| Cancelled Pipeline Runs | The total number of pipeline runs that were cancelled while they were running between the poll interval. |
| ELAPSED PIPELINE RUNS | |
| Elapsed Pipeline Runs | The total number of pipeline runs that exceeded the configured elapsed time value between the poll interval. |
| SUCCEEDED TRIGGER RUNS | |
| Succeeded Trigger Runs | The total number of trigger runs that successfully initiated pipelines between the poll interval. |
| FAILED TRIGGER RUNS | |
| Failed Trigger Runs | The total number of trigger runs that failed to initiate pipelines between the poll interval. |
| CANCELLED TRIGGER RUNS | |
| Cancelled Trigger Runs | The total number of trigger runs that were cancelled before starting a pipeline between the poll interval. |
| SUCCEEDED ACTIVITY RUNS | |
| Succeeded Activity Runs | The total number of activity runs that completed successfully between the poll interval. |
| FAILED ACTIVITY RUNS | |
| Failed Activity Runs | The total number of activity runs that failed execution between the poll interval. |
| CANCELLED ACTIVITY RUNS | |
| Cancelled Activity Runs | The total number of activity runs that were cancelled while running between the poll interval. |
| Parameter | Description |
|---|---|
| PIPELINE SUCCESS RATE | |
| Pipeline Success Rate | The percentage of successful pipeline runs based on the runs displayed. |
| TRIGGER SUCCESS RATE | |
| Trigger Success Rate | The percentage of successful trigger runs based on the runs displayed. |
| Pipeline Runs | |
| Pipeline Run Identifier | Identifier of a pipeline run. |
| Pipeline Name | The name of the pipeline. |
| Status | Status of the pipeline run. Possible values: Queued, InProgress, Succeeded, Failed, Canceling, Cancelled. |
| Start Time | The start time of the pipeline run. |
| End Time | The end time of the pipeline run. |
| Total Duration (sec) | The total execution duration of the pipeline run (in seconds). |
| Trigger Run Identifier | Identifier of the trigger that initiated the pipeline run. |
| Trigger Name | Name of the trigger. |
| Trigger Runs | |
| Trigger Run Identifier | Identifier of the trigger run. |
| Trigger Name | Name of the trigger. |
| Trigger Type | Type of the trigger. |
| Status | Status of the trigger run. Possible values: Succeeded, Failed, InProgress. |
| Scheduled Time | The time the trigger was scheduled to run. |
| Triggered Time | The time the trigger executed. |
| Delay | The delay before the trigger execution. |
| No. of Pipelines | The number of pipelines triggered by the trigger run. |
| Parameter | Description |
|---|---|
| Integration Runtime CPU Utilization | The average percentage of CPU utilized by the Integration Runtime at the time of polling. |
| Integration Runtime Available Memory | The average amount of available memory on the IR host machine at the time of polling (in MB). |
| Integration Runtime Available Nodes | The average number of available nodes capable of processing data integration activities at the time of polling. |
| Integration Runtime Queue Duration | The average time taken for an activity to begin execution after entering the queue between the poll interval (in seconds). |
| Integration Runtime Queue Length | The average number of activities waiting in the queue between the poll interval. |
| Parameter | Description |
|---|---|
| Resource Group Name | The name of the Resource Group containing the Data Factory. |
| Location | The location of the Azure Data Factory. |
| Provisioning State | The current provisioning state of the Data Factory. |
| Creation Time | The time when the Data Factory was created. |
| Public Network Access | Indicates whether public network access is enabled or disabled. |
Thank you for your feedback!
