Configuring CloudWatch Agent in an EC2 instance

To collect operating system-level metrics such as Memory and Disk, you need to deploy the CloudWatch Agent inside an EC2 instance. The agent will then send your data to CloudWatch, from where Applications Manager fetches and displays it in the console. For more information about the CloudWatch agent and the supported operating systems, refer to this document.

Before we go into the prerequisites, let's explore the two potential methods for installing a CloudWatch agent on the EC2 instance:

1. Using Systems Manager
   • There's no need to access the instance as this can be done directly in the portal itself.
   • Involves more steps than the command line method.
   • Recommended for bulk installation of CloudWatch agents.
2. Using Command line
   • Access to the instance is required, especially for command-line usage.
   • Involves fewer steps than the Systems Manager method.
   • Recommended for installing CloudWatch agents in lesser number of instances.

Note: Consider the pros and cons mentioned above before choosing your preferred method and follow the steps accordingly.

Overview of prerequisites

1. Create the IAM role: Create an IAM role to attach to the EC2 Instance.
2. Attach IAM role: Attach the created IAM role to the EC2 instance. This IAM role will provide the necessary permissions to the CloudWatch agent residing on the EC2 instance to push metrics to the CloudWatch console.
3. Create the agent configuration file: Create the configuration file for the CloudWatch agent to specify the list of metrics to be pushed to the CloudWatch console.
4. Install agent: Download and install the CloudWatch agent. We can do it in either of the following ways:
   • Use Systems Manager to install the agent
   • Use the command line to install the agent manually
5. Start the agent using config file: Start the CloudWatch agent using the JSON configuration.
6. Verify CloudWatch metrics: Verify the metrics in the CloudWatch console.

Prerequisites

• Create the IAM role
• Attach the role to an EC2 Instance
• Create the configuration file
• Install the CloudWatch agent
• Start the CloudWatch agent

1. Create the IAM role

Follow the steps given below to create an IAM role to attach to the EC2 Instance:

1. Sign in to the AWS Management Console and open the IAM console.
2. In the navigation panel, choose Roles and then choose Create role.
3. Under 'Select type of trusted entity', choose AWS service and choose EC2 from the drop-down under Use case and click on Next.
4. In the list of policies, select the check box next to CloudWatchAgentServerPolicy. If necessary, use the search box to find the policy.
5. [For Systems Manager] To use Systems Manager to install or configure the CloudWatch agent, select the check box next to AmazonSSMManagedInstanceCore.

   Note: This AWS-managed policy allows an instance to utilize the core functionality of the Systems Manager service. It becomes unnecessary if you exclusively initiate and set up the agent using the command line.

6. For the Role name, enter a name for your new role, such as CloudWatchAgentServerRole.
7. Verify that the selected policies from above are listed in the Permissions Policy Summary.
8. On clicking Create role, the role will be successfully created.

For further information on creating an IAM Role, refer to this document .

2. Attach the role to an EC2 Instance

The IAM role will provision the required permission to CloudWatch agent resided in EC2 Instance to push metrics to CloudWatch console. Follow the steps given below to attach the created IAM role to the EC2 Instance:

1. Open the Amazon EC2 console.
2. Click on Instances from the navigation pane.
3. Click on the Instance ID of the required instance and navigate to Actions → Security → Modify IAM role.
4. Select the IAM role created in the previous step and attach it to your instance. Click on Update IAM role to finish.

3. Create the configuration file

Generate the configuration file for the CloudWatch agent to specify the list of metrics that will be pushed to the CloudWatch console

Note: The utilization of Parameter Store is obligatory for Systems Manager. Alternatively, when employing the command line, one can opt for either the use of Parameter Store or directly create a JSON file on the EC2 Instance.

Parameter Store (Recommended)

The Parameter Store serves as a configuration management service capable of storing configuration strings. It operates on a region-specific basis, facilitating the utilization of the same configuration for multiple instances.

1. Go to the parameter store.
2. Click Create Parameter and follow the steps below:
   • Enter a Name (For example: CwAgent_Windows_Config Or CwAgent_Linux_Config) and Description for your parameter.
   • Choose the values 'Standard' for the Tier and 'String' for the Type.
   • Select 'text' for the Data type and enter the JSON Content given below, based on the EC2 Instance OS type, in the Value text field.
   • Click on the Create parameter button.

Directly Within the EC2 Instance

This method is solely applicable to the command-line approach and can be disregarded if you have already created the configuration JSON in the Parameter store for the command-line method.

1. Access the EC2 instance where you will install the CloudWatch agent.
2. Create a JSON file with the provided content and save it. Below are the recommended paths for Linux and Windows (Create the directory before creating the file):

   Linux:

   /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json

   Windows:

   $Env:ProgramData\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent.json

JSON Content

Windows OS:

{ "metrics": { "append_dimensions": { "InstanceId": "${aws:InstanceId}" }, "metrics_collected": { "LogicalDisk": { "measurement": [ "% Free Space", "Free Megabytes" ], "metrics_collection_interval": 300, "resources": [ "*" ] }, "Memory": { "measurement": [ "% Committed Bytes In Use", "Available MBytes" ], "metrics_collection_interval": 300 } } } }

Linux OS:

{ "metrics": { "append_dimensions": { "InstanceId": "${aws:InstanceId}" }, "metrics_collected": { "mem": { "measurement": [ "mem_total", "mem_used" ], "metrics_collection_interval": 300 }, "disk": { "resources": [ "/", "/run" ], "measurement": [ "disk_total", "disk_used" ], "metrics_collection_interval": 300 }, "swap": { "measurement": [ "swap_used_percent" ], "metrics_collection_interval": 300 } } } }

4. Install the CloudWatch agent

Users can choose to download and install the CloudWatch agent in either of the following ways:

• Using Systems Manager to install the agent

  To procure and install the CloudWatch agent using Systems Manager, please adhere to the instructions outlined in this guide.

• In the EC2 instance (command line) directly
  1. Install directly from package manager
  2. The CloudWatch agent is available as a package in Linux package manager, install the package by entering the following command.

     sudo yum install amazon-cloudwatch-agent

  3. (OR)Download package and install manually
     1. Download the CloudWatch agent package from the this document corresponding to your respective OS. For example:
        • Amazon Linux (x86-64): (rpm file)
        • Ubuntu/Debian (x86-64): (deb file)
        • Windows (x86-64): (msi file)
     2. Install the downloaded package using the respective command:
        • rpm file: sudo rpm -U ./amazon-cloudwatch-agent.rpm
        • deb file: sudo dpkg -i -E ./amazon-cloudwatch-agent.deb
        • exe file: msiexec /i amazon-cloudwatch-agent.msi

5. Start the CloudWatch agent

This step requires the JSON configuration created in the third step.

Note: Restarting the CloudWatch agent is mandatory whenever the configuration is changed.

For Systems Manager

• To start the CloudWatch agent with a parameter store configuration using Systems Manager, please refer to the AWS CloudWatch documentation's section 'Step 2: Start the CloudWatch agent using your agent configuration file' & follow the steps outlined.

For Command line

Linux:

• Using parameter store configuration

  sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c ssm:configuration-parameter-store-name

• Using JSON file

  sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:configuration-file-path

Windows (PowerShell):

• Using parameter store configuration

  & "C:\Program Files\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1" -a fetch-config -m ec2 -s -c ssm:configuration-parameter-store-name

• Using JSON file

  & "C:\Program Files\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1" -a fetch-config -m ec2 -s -c file:configuration-file-path

Check CloudWatch agent running status

After performing all the steps to verify if the CloudWatch agent is running , please follow the below steps.

For Systems Manager:

• Follow the steps mentioned in this document to verify the status of the CloudWatch agent.

For Command line:

• Linux:

  sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status

• Windows (PowerShell):

  & $Env:ProgramFiles\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1 -m ec2 -a status