Microsoft Azure Synapse Analytics Monitoring
Microsoft Azure Synapse Analytics - An Overview
Azure Synapse Analytics is a cloud-native integrated analytics solution that offers a robust framework tailored for big data and data warehousing requirements. With Azure Synapse Analytics, businesses can effortlessly handle data ingestion, preparation, management, and delivery to support their real-time business intelligence and machine learning demands.
Monitoring plays a vital role in overseeing and upholding the well-being, efficiency, and security of your analytics ecosystem within Azure Synapse Analytics. Applications Manager's Azure Synapse Analytics Monitoring tool offers a variety of monitoring and observability resources and functionalities designed to provide you with valuable perspectives into the condition of your Synapse Analytics tasks.
Creating a new Microsoft Azure Synapse Analytics Monitor
To learn how to create a new Microsoft Azure Synapse Analytics Monitor, click here.
Monitored Parameters
Navigate to the Category View by clicking the Monitors tab. Hover over 'Child Monitors' under Microsoft Azure in the Cloud Apps table, and then select the Synapse Analytics monitor from the displayed tooltip. This action will display the bulk configuration view for Azure Synapse Analytics in three tabs:
- Availability tab gives the Availability history for the past 24 hours or 30 days.
- Performance tab gives the Health Status and events for the past 24 hours or 30 days.
- List view enables you to perform bulk admin configurations.
The Microsoft Azure monitor provides a brief detail of the Azure Synapse Analytics under the given subscription. Following are the list of metrics monitored in Azure Synapse Analytics Monitoring in their corresponding tabs:
Overview
| Parameter | Description |
|---|
| DATA PROCESSED |
| Data Processed Rate | The amount of data processed by queries per minute between the poll interval (in MB/min). |
| Total Data Processed | The total amount of data processed by queries between the poll interval (in GB). |
| LOGIN ATTEMPTS |
| Login Attempts | The total number of login attempts that succeeded or failed between the poll interval. |
| REQUESTS ENDED |
| Requests Ended | The total number of requests that succeeded, failed or were cancelled between the poll interval. |
| PIPELINE RUNS ENDED |
| Pipeline Runs Ended | The total number of integration pipeline runs that succeeded, failed or were cancelled between the poll interval. |
| ACTIVITY RUNS ENDED |
| Activity Runs Ended | The total number of integration activities that succeeded, failed or were cancelled between the poll interval. |
| TRIGGER RUNS ENDED |
| Trigger Runs Ended | The total number of integration triggers that succeeded, failed or were cancelled between the poll interval. |
| LINK CONNECTION EVENTS |
| Link Connection Events | The total number of Synapse Link connection events including start, stop and failure between the poll interval. |
| LINK TABLE EVENTS |
| Link Table Events | The total number Synapse Link table events including snapshot, removal and failure between the poll interval. |
| LINK PROCESSED ROWS |
| Link Processed Rows | The total number of changed rows processed by Synapse Link between the poll interval. |
| TOTAL LINK DATA PROCESSED |
| Link Data Processed Rate | The data volume processed by the Synapse Link per minute between the poll interval (in MB/min). |
| Total Link Data Processed | The total data volume processed by the Synapse Link between the poll interval (in GB). |
| LINK LATENCY |
| Link Latency | The maximum latency taken for Synapse Link data processing between the poll interval (in seconds). |
SQL Pools
| Parameter | Description |
|---|
| SQL Pools Performance |
| SQL Pool Name | The name of the dedicated SQL Pool. |
| DWU Used Percent | The maximum usage of DWUs between the poll interval (in %). Represents a high-level overview of usage within the SQL pool that is measured by taking the maximum of either CPU percentage or Data IO percentage |
| CPU Utilization | The maximum CPU utilization across all nodes in the SQL pool between the poll interval (in %). |
| Memory Utilization | The maximum memory utilization across all nodes in the SQL pool between the poll interval (in %). |
| Active Queries | The total number of active queries between the poll interval. Using this metric unfiltered and unsplit displays all the active queries running in the system. |
| Queued Queries | The total number of requests queued after the max concurrency limit was reached between the poll interval. |
| Adaptive Cache Hit Percent | The maximum cache hit percentage between the poll interval (in %). This metric measures the efficiency of workloads in utilizing the adaptive cache. Use this metric along with the cache hit percentage metric to determine whether to scale for additional capacity or rerun workloads to refresh the cache. |
| Adaptive Cache Used Percent | The maximum cache used percentage between the poll interval (in %). This metric measures the efficiency of workloads in utilizing the adaptive cache. Use this metric along with the cache used percentage metric to determine whether to scale for additional capacity or rerun workloads to refresh the cache. |
| No of Logins | The total number of logins to the SQL Pool between the poll interval. |
| Connections Blocked by Firewall | The total number of connections blocked by firewall rules between the poll interval. Revisit access control policies for your SQL pool and monitor these connections if the count is high. |
| SQL Pools Configuration |
| SQL Pool Name | The name of the SQL Pool. |
| Performance Level | The performance level of the SQL Pool. |
| DWU Limit | The maximum limit of DWU of the SQL Pool between the poll interval. |
| DWU Used | The maximum DWUs used by the SQL Pool between the poll interval. Represents a high-level representation of usage across the SQL pool. Measured as DWU limit * DWU percentage. |
| SQL Pool Status | The status of the SQL Pool. |
| Storage Account Type | The storage account type used to store backups for this SQL Pool. Possible values: GRS/LRS. |
| SQL Pool Provisioning State | The provisioning state of the SQL Pool. |
| Maximum Size | The maximum size of the SQL Pool in GB. |
| Creation Time | Date the SQL Pool was created. |
Note: Kindly note that the monitor only displays the first 20 SQL Pools from the API.
Big Data Pools
| Parameter | Description |
|---|
| Big Data Pools Performance |
| Big Data Pool Name | Name of the Big Data Pool. |
| vCores Allocated | The maximum number of vCores allocated for an Apache Spark Pool between the poll interval. |
| Memory Allocated | The maximum allocated memory for an Apache Spark Pool between the poll interval (in GB). |
| Active Spark Applications | The maximum Active Apache Spark Pool applications between the poll interval. |
| Ended Spark applications | The total number of Apache Spark pool applications that ended between the poll interval. |
| Node Count | The total number of nodes in the Big Data pool. |
| Cache Size | The cache size of the Big Data pool (in GB). |
| AutoPause Delay | The amount of idle time before the Big Data pool is automatically paused (in minutes). |
| Big Data Pool Provisioning State | The provisioning state of the Big Data Pool. |
| Node Size | The size of the nodes in the Big Data pool. Possible values:- Small
- Medium
- Large
- XLarge
- XXlarge
- XXXLarge (Isolated Compute)
|
| Big Data Pools Configuration |
| Big Data Pool Name | Name of the Big Data Pool. |
| Apache Spark Version | The version of the Azure Spark Pool. |
| Node Size Family | The type of nodes the Big Data Pool provides. |
| Autoscale Enabled | Indicates whether automatic scaling is enabled for the Big Data pool or not. Possible values: true/false. |
| Autoscale Min Nodes | The minimum number of nodes the Big Data pool can support. |
| Autoscale Max Nodes | The maximum number of nodes the Big Data pool can support. |
| Session Level Packages Enabled | Indicates whether the session level packages automatic scaling is enabled for the Big Data pool or not. Possible values: true/false. |
| Creation time | The time the Big Data Pool was created. |
| Last Succeeded Time | The time when the Big Data pool was last updated successfully. |
Note: Kindly note that the monitor only displays the first 20 Big Data Pools from the API.
Configuration
| Parameter | Description |
|---|
| CONFIGURATION |
| Resource Group Name | The name of the Resource Group. |
| Location | The location of the resource. |
| Provisioning State | The provisioning state of the resource. |
| Managed Resource Group Name | The name of the workspace managed resource group. |
| SQL Admin login | The username of the SQL DB administrator. |
| AD Only Authentication | Indicates whether Microsoft Entra ID Authentication is enabled or not on all workspace subresources. Possible values: True/False. |
| Public Network Access | Indicates whether Public network access is enabled or not. Possible values: Enable/Disable. |
| Managed Virtual network | Setting this to 'default' will ensure that all compute for the synapse workspace is in a virtual network managed on behalf of the user. |
| DATA LAKE STORAGE ACCOUNT DETAILS |
| Storage Account Name | The name of the Azure Data Lake Storage Account. |
| FileSystem Name | The name of Azure Data Lake Storage Account's filesystem. |
| Storage Account URL | The url of the Azure Data Lake Storage Account. |
| ENDPOINTS DETAILS |
| Web Endpoint | The connectivity endpoints for the Workspace. |
| SQL Endpoint | The connectivity endpoints for SQL. |
| SQL On Demand Endpoint | The connectivity endpoints for On Demand SQL. |
| Dev Endpoint | The connectivity endpoints for Dev. |
