Amazon Web Services Monitoring



Amazon Web Services (AWS) - Overview

Amazon Web Services (AWS) is a comprehensive cloud computing platform by Amazon. AWS offers an array of cloud computing services to build on from data centers spread across availability zones in regions across the world. But you still need to be able to understand what’s going on when you deploy and consume such a wide-ranging set of services.

From your infrastructure and applications to your end-user experience, Applications Manager performs robust monitoring of several core Amazon Web Services under your Amazon account:


Creating a new Amazon Monitor

Prerequisites for setting up Amazon monitoring: Click here.

To create a new Amazon monitor, follow the steps given below:

  1. Click on New Monitor link.

  2. Select Amazon under Cloud Apps category.

  3. Specify the Display Name of the Amazon server

  4. Select the Amazon Services you wish to discover and monitor from the drop-down menu.

  5. Enter the Access Key Id of the AWS for accessing the AWS through the API. The access key has 20 alpha-numeric characters.

  6. Enter the Secret Access Key of the AWS. The secret key should be 40 alpha-numeric characters long.

  7. Choose the Account Type. The default Account Type when adding a new Amazon monitor is AWS Global. Choose AWS China as the Account Type to monitor services in the AWS China regions.

  8. Specify the Polling Interval in minutes.

  9. If you are adding a new monitor from an Admin Server, select a Managed Server.

  10. Choose the Monitor Group with which you want to associate the Amazon monitor to, from the combo box (optional). You can choose multiple groups to associate your monitor.

  11. Click Add Monitor(s). This discovers the Amazon server from the network and starts monitoring it.

Note: Under the Admin -> Performance Polling-> Amazon -> "On Modifying Amazon Services" option, you can choose to either 'Stop discovering' or 'delete' the existing added child monitors (Amazon EC2, RDS, DynamoDB etc) when a user edits and removes existing services. By default, this option is set to 'Stop discovering'

Amazon Data Collection Settings

You can configure data collection settings for your Amazon monitor by going to the Admin tab, selecting Performance Polling under Discovery and then the Amazon tab. Here are the settings that you can customize:

  • You can set the Unique Identifier for EC2 instances to be either the instance ID or a tag name.

    Caution: Once set, changing the Unique Identifier will result in the existing monitors being deleted. If a new instance with Tag name of an already terminated instance is launched, it will be merged with the former instance.
  • Enable the Alert for Terminated EC2 Instance option to receive notification when an instance is stopped.
  • Enabling the Remove terminated EC2 Instances option completely removes terminated EC2 Instances from the Applications Manager console in the next poll. By default, this option is disabled. Enable it if you find a terminated EC2 instance in the Applications Manager console, after it was removed from the AWS console (If a user terminates an EC2 instance it will be removed from the AWS console after 60 minutes.)
Note: Make sure you have a working internet connecion to add or fetch data from the cloud services. If your environment requires the use of a proxy server to access external Websites, you can configure the proxy settings under the 'Admin' tab.

Use Add Monitor API to add a new Amazon Monitor

Syntax

http://[Host]:[Port]/AppManager/xml/AddMonitor?apikey=[APIKEY]&type=[TYPE]&AmazonServices=[AmazonServices]&accessKey=[ACCESSKEY]&SecretAccessKey=[SECRETACCESSKEY]&displayname=[DISPLAYNAME]&AccountType=[account type]

Request Parameters:

The parameters involved in the API request are described below. Also, refer the list of common Request Parameters.

ParameterDescription
type The type of the monitor you want to add. Value should be Amazon
accessKey The Access Key Id of the AWS for accessing the AWS through the API.
SecretAccessKey The secret access key of the AWS.
AccountType The account type. Possible Values are
  • AwsGlobal
  • AwsChina
AmazonServices The Amazon Services that you want to monitor. Possible values are:
  • EC2 (for Elastic Compute Cloud)
  • RDS (for Relational Database Services)
  • S3 (for S3 Buckets)
  • SNS (for Simple Notification Services)
  • DYNAMODB,EC2,RDS,S3,SNS (for multiple services)

Sample Request:

http://localhost:9090/AppManager/xml/AddMonitor?apikey=15212eb94d705a3b0993e30709818c4e&type=Amazon&AmazonServices=DYNAMODB,EC2,RDS,S3,SNS&accessKey=AKIAIV2PL2VNMIAUOXCA&SecretAccessKey=FECP%2BgNtMcTlWSs05S6ubC/hl6oGwSowmOBP2uNx&displayname=awsrestapi&AccountType=AwsGlobal

Monitored Parameters

Amazon accounts are monitored based on the parameters or the attributes listed below. These attributes provide detailed information about the functioning of the Amazon account. You can also configure thresholds to the numerical attributes based on these details and get notified when the thresholds are violated.

Go to the Monitors Category View by clicking the Monitors tab. Click on the Amazon monitor under the Cloud Apps Table. Displayed is the Amazon bulk configuration view distributed into three tabs:

  • Availability tab displays the Availability history for the past 24 hours or 30 days.
  • Performance tab displays the Health Status and events for the past 24 hours or 30 days along with key performance indicators of the Amazon account such as Total EC2 Instances running and Total RDS Instances along with heat charts for these attributes.
  • List view enables you to perform bulk admin configurations and displays all the Amazon instances along with an overall idea of their availability and health status.

Click on the monitor name to see all the AWS metrics listed under the following tabs:


Instances

This tab provides a high-level overview of your Amazon account as well as information about the EC2 instances present in this account. Know more about EC2 Monitoring.

ParameterDescription
Monitor Information
Name The name of the Amazon account.
Type The type you are monitoring.
Health The health (Clear, Warning, Critical) status of the Amazon account.
Total EC2 Instances Running The number of EC2 instances running in the account
Total EBS Volumes in use The number of EBS storage volumes currently in use
Total RDS Instances The total number of RDS instances present in the account
Last Polled at The time at which the last poll was performed
Next Poll at The time at which the next poll is scheduled
Today's Availability The overall availability status of the account for the day. You can also view 7/30 reports and the current availability status of the account
EC2 Instances
Instance ID The unique identifier of the EC2 instance
Region Name The region where this EC2 instance is running (Supported regions).
State The current state of the instance. The values include running, stopped, shutdown and terminated
Platform The OS on which the instance is running
Public DNS Name The DNS name associated with the instance

You can perform the following admin actions on the EC2 instances:

  • Delete: Delete the EC2 instance from the account.
  • Assign Platform: Assign platforms such as Windows, Mac OS, etc. to instances
  • Start Instances: Option to start the EC2 instance from Applications Manager.
  • Stop Instances: Option to stop EC2 instances from Applications Manager
  • Reboot Instances: Option to reboot the EC2 instances from within Applications Manager.

RDS Instances

This tab provides details about the RDS Instances present in the Amazon account. More on RDS Instances.

ParameterDescription
Instance ID The unique identifier of the instance.
Region Name The region in which the RDS instance is running.
State The current state of this instance. The possible values for this field are available, creating, failed, rebooting, etc.
DB Engine Name The name of the database engine associated with this instance
Allocated Storage The storage space allocated to this instance in Giga Bytes
Aurora Cluster Name The name of the Aurora DB cluster.
Cluster Instance Role The role that is associated with the Aurora DB instance.
Cluster Group status The current state of this DB cluster group.

S3 Buckets

This tab provides details about the S3 buckets present in the Amazon account.

ParameterDescription
Bucket Name The unique name of the S3 bucket.
Bucket Size The size of the S3 bucket in megabytes.
Bucket Location The geographical region where Amazon has stored this bucket. Regions currently supported by Amazon are US-Standard, US-West (Northern California), EU (Ireland) and APAC-Singapore.
Creation Time The time when the bucket was created.
Virtual Folders The number of folders present in this S3 bucket.
Number of Objects The number of objects stored in this S3 bucket.

You can perform the following admin actions on the S3 buckets from within Applications Manager.

  • Disable: Disable the monitoring of S3 bucket.
  • Enable: Enable the monitoring of S3 bucket.
  • Delete: Delete the S3 bucket from Applications Manager.Use this option when S3 bucket is deleted from your Amazon account.

You can also view comparison reports based on attributes such as bucket size, virtual folders and number of object


SNS Services

In this tab you can monitor region and topic-wise metrics like notification status (delivery and failure), SMS deliveries, messages throughput - sent and failed counts, subscription count and more. More on SNS Service monitoring.

ParameterDescription
Total Messages Published The total number of messages published throughout all the AWS supported regions.
Total Notifications Delivered The total number of messages delivered throughout all the AWS supported regions.
Total Notifications Failed The total number of messages failed throughout all the AWS supported regions.
Total Published Size (KB) The total size of messages published throughout all the AWS supported regions.
Notifications Delivered (%) The percentage of total delivered notifications.
Notifications Failed (%) The percentage of total failed notifications.
SMS Month-To-Date charges The total charges accrued since the start of the current calendar month for sending SMS messages in AWS supported regions. Unit - USD

DynamoDB

In this tab, you can gather data for performance metrics of you Amazon DynamoDB like latency, request throughput and throttling errors. Optimize resource usage and improve application performance of your Amazon Dynamodb database. More on DynamoDB monitoring.

ParameterDescription
Read Capacity Units
Provisioned Read The number of provisioned read capacity units for a table or a global secondary index between the poll interval time.
Provisioned Read - Average The average provisioned read capacity.
Provisioned Read - Minimum The lowest setting for provisioned read capacity.
Provisioned Read - Maximum The highest setting for provisioned read capacity.
Consumed Read The number of read capacity units consumed over the poll interval time. This will help you track how much of your provisioned throughput is used.
Consumed Read - Average The maximum number of read capacity units consumed by any individual request to the table or index.
Consumed Read - Minimum The minimum number of read capacity units consumed by any individual request to the table or index.
Consumed Read - Maximum The average per-request read capacity consumed.
Write Capacity Units
Provisioned Write The number of provisioned write capacity units for a table.
Provisioned Write - Average The average provisioned read capacity.
Provisioned Write - Minimum The lowest setting for provisioned read capacity.
Provisioned Write - Maximum The highest setting for provisioned read capacity.
Consumed Write The number of write capacity units consumed for the global secondary index between the poll interval time.
Consumed Write - Average The average per-request write capacity consumed.
Consumed Write - Minimum The minimum number of write capacity units consumed by any individual request to the table or index.
Consumed Write - Maximum The maximum number of write capacity units consumed by any individual request to the table or index.
Read Throttled Requests
Throttled Read Requests If your read or write requests exceed the throughput settings for a table and tries to consume more than the provisioned capacity units or exceeds for an index, DynamoDB can throttle that request. Based on the type of operation (Get, Scan, Query, BatchGet) performed on the table, throttled request data can be populated.
Get Throttled Requests The Get operation returns a set of attributes for the request with the given primary key.
Scan Throttled Requests The Scan operation returns one or more requests and request attributes by accessing every request in a table or a secondary index.
Query Throttled Requests The Query operation finds requests based on primary key values. You can query any table or secondary index that has a composite primary key (a partition key and a sort key).
BatchGet Throttled Requests The BatchGet operation returns the attributes of one or more requests from one or more tables.
Write Throttled Requests
Put Throttled Requests The Put operation creates a new request, or replaces an old request with a new one.
Update Throttled Requests The Update operation edits an existing request's attributes, or adds a new request to the table if it does not already exist. You can put, delete, or add attribute values.
Delete Throttled Requests The Delete operation deletes a single request in a table by primary key.
BatchWrite Throttled Requests The BatchWrite operation puts or deletes multiple requests in one or more tables.
Read Throttled Events
Table Read Throttled Events The number of throttled write events during the time period. Throttled events occurs when any particular event or any single event in any batch operation fails.
Write Throttled Events
Table Write Throttled Events The number of throttled read events during the time period. Throttled events occurs when any particular event or any single event in any batch operation fails.

Supported Regions

AWS Global Regions

From Version 13600 and above, we support these global AWS regions:
    1. US East (N. Virginia)
    2. US East (Ohio)
    3. US West (N. California)
    4. US West (Oregon)
    5. Canada (Central)
    6. EU (Frankfurt)
    7. EU (Ireland)
    8. EU (London)
    9. EU (Paris)
    10. EU (Stockholm)
    11. Asia Pacific (Tokyo)
    12. Asia Pacific (Seoul)
    13. Asia Pacific (Mumbai)
    14. Asia Pacific (Singapore)
    15. Asia Pacific (Sydney)
    16. Canada (Central)
    17. South America (Sao Paulo)

AWS China Regions

The China regions are isolated from all other AWS (Global) regions and operated by AWS's Chinese partners separately from all other AWS regions. While the cloud services offered in both AWS China regions are the same as those available in other AWS (Global) regions. From Version 13620, we support monitoring of Amazon Web Services in AWS China regions. Here is a list of regions available in the AWS China account:
    1. China (Beijing)
    2. China (Ningxia)

The default Account Type when adding a new Amazon monitor is AWS Global. Choose AWS China as the Account Type to monitor services in the AWS China regions.


Troubleshooting

Some instances of EC2, RDS, S3, SNS are not being discovered for data collection:

There are two ways of authenticating information in the requests made to AWS to collect data:
    • Signature version 2 (S2) and
    • Signature version 4 (S4)

In the following regions, aws supports only signature version 4 signing process: [US East (Ohio), Canada (Central), Asia Pacific (Mumbai), Asia Pacific (Seoul), EU (Frankfurt), EU (London) ]. If you are using AWS services from the regions which supports only S4, Applications Manager will not discover them as they require S4 signing process.

Solution: Upgrade to Applications Manager Version 13600 (or newer). We have upgraded to the latest aws jar and support both signing processes in AWS. From Version 13600, we also support all 15 regions available currently. (Know more)

Note: Make sure you have a working internet connecion to add or fetch data from the cloud services. If your environment requires the use of a proxy server to access external Websites, you can configure the proxy settings under the 'Admin' tab.