# Configuring CloudWatch Agent in an EC2 instance

To collect operating system-level metrics such as Memory and Disk, you need to deploy the CloudWatch Agent inside an EC2 instance. The agent will then send your data to CloudWatch, from where Applications Manager fetches and displays it in the console. For more information about the CloudWatch agent and the supported operating systems, refer to [this document](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html).

Before we go into the prerequisites, let's explore the two potential methods for installing a CloudWatch agent on the EC2 instance:

1. **Using Systems Manager**
   - There's no need to access the instance, as this can be done directly in the portal itself.
   - Involves more steps than the command line method.
   - Recommended for bulk installation of CloudWatch agents.
2. **Using Command line**
   - Access to the instance is required, especially for command-line usage.
   - Involves fewer steps than the Systems Manager method.
   - Recommended for installing CloudWatch agents in a smaller number of instances.

**Note:** Consider the pros and cons mentioned above before choosing your preferred method and follow the steps accordingly.

## Overview of prerequisites

1. **Create the IAM role**: Create an IAM role to attach to the EC2 Instance.
2. **Attach IAM role**: Attach the created IAM role to the EC2 instance. This IAM role will provide the necessary permissions to the CloudWatch agent residing on the EC2 instance to push metrics to the CloudWatch console.
3. **Create the agent configuration file**: Create the configuration file for the CloudWatch agent to specify the list of metrics to be pushed to the CloudWatch console.
4. **Install agent**: Download and install the CloudWatch agent. We can do it in either of the following ways:
   - Use Systems Manager to install the agent
   - Use the command line to install the agent manually
5. **Start the agent using config file**: Start the CloudWatch agent using the JSON configuration.
6. **Verify CloudWatch metrics**: Verify the metrics in the CloudWatch console.

## Prerequisites

- [Create the IAM role](https://www.manageengine.com/products/applications_manager/help/configuring-cloud-watch-agent.html#IAM)
- [Attach the role to an EC2 Instance](https://www.manageengine.com/products/applications_manager/help/configuring-cloud-watch-agent.html#EC2)
- [Create the configuration file](https://www.manageengine.com/products/applications_manager/help/configuring-cloud-watch-agent.html#configurationfile)
- [Install the CloudWatch agent](https://www.manageengine.com/products/applications_manager/help/configuring-cloud-watch-agent.html#install)
- [Start the CloudWatch agent](https://www.manageengine.com/products/applications_manager/help/configuring-cloud-watch-agent.html#start)

### 1. Create the IAM role

Follow the steps given below to create an IAM role to attach to the EC2 Instance:

1. Sign in to the AWS Management Console and open the [IAM console](https://console.aws.amazon.com/iam/index.html).
2. In the navigation panel, choose **Roles** and then choose **Create role**.
3. Under "Select type of trusted entity", choose **AWS service** and choose **EC2** from the drop-down under Use case and click on **Next**.
4. In the list of policies, select the checkbox next to **CloudWatchAgentServerPolicy**. If necessary, use the search box to find the policy.
5. ***[For Systems Manager]*** To use Systems Manager to install or configure the CloudWatch agent, select the check box next to **AmazonSSMManagedInstanceCore**.

   **Note:** This AWS-managed policy allows an instance to utilize the core functionality of the Systems Manager service. It becomes unnecessary if you exclusively initiate and set up the agent using the command line.

6. For the **Role name**, enter a name for your new role, such as **CloudWatchAgentServerRole**.
7. Verify that the selected policies from above are listed in the Permissions Policy Summary.
8. On clicking **Create role**, the role will be successfully created.

For further information on creating an IAM Role, [refer to this document](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-iam-roles-for-cloudwatch-agent.html).

### 2. Attach the role to an EC2 Instance

The IAM role will provision the required permission to CloudWatch agent resided in EC2 Instance to push metrics to CloudWatch console. Follow the steps given below to attach the created IAM role to the EC2 Instance:

1. Open the Amazon [EC2 console](https://console.aws.amazon.com/ec2).
2. Click on **Instances** from the navigation pane.
3. Click on the Instance ID of the required instance and navigate to **Actions** → **Security** → **Modify IAM role**.
4. Select the IAM role created in the previous step and attach it to your instance. Click on **Update IAM role** to finish.

### 3. Create the configuration file

Generate the configuration file for the CloudWatch agent to specify the list of metrics that will be pushed to the CloudWatch console.

**Note:** The utilization of Parameter Store is obligatory for Systems Manager. Alternatively, when employing the command line, one can opt for either the use of Parameter Store or directly create a JSON file on the EC2 Instance.

#### Parameter Store (Recommended)

The Parameter Store serves as a configuration management service capable of storing configuration strings. It operates on a region-specific basis, facilitating the utilization of the same configuration for multiple instances.

1. Go to the [parameter store](https://console.aws.amazon.com/systems-manager/parameters).
2. Click **Create Parameter** and follow the steps below:
   - Enter a **Name** (**For example**: CwAgent_Windows_Config Or CwAgent_Linux_Config) and **Description** for your parameter.
   - Choose the values "Standard" for the **Tier** and "String" for the **Type**.
   - Select "text" for the **Data type** and enter the [JSON content given below](https://www.manageengine.com/products/applications_manager/help/configuring-cloud-watch-agent.html#JSON), based on the EC2 Instance OS type, in the **Value** text field.
   - Click on the **Create parameter** button.

#### Directly Within the EC2 Instance

This method is solely applicable to the command-line approach and can be disregarded if you have already created the configuration JSON in the Parameter Store for the command-line method.

1. Access the EC2 instance where you will install the CloudWatch agent.
2. Create a JSON file with the [provided content](https://www.manageengine.com/products/applications_manager/help/configuring-cloud-watch-agent.html#JSON) and save it. Below are the recommended paths for Linux and Windows (Create the directory before creating the file):

   **Linux:**

   `/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json`

   **Windows:**

   `$Env:ProgramData\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent.json`

#### JSON Content

##### Till Applications Manager version 181800

**Windows OS:**

```json
{
  "metrics": {
    "append_dimensions": {
      "InstanceId": "${aws:InstanceId}"
    },
    "metrics_collected": {
      "LogicalDisk": {
        "measurement": [
          "% Free Space",
          "Free Megabytes"
        ],
        "metrics_collection_interval": 300,
        "resources": [
          "*"
        ]
      },
      "Memory": {
        "measurement": [
          "% Committed Bytes In Use",
          "Available MBytes"
        ],
        "metrics_collection_interval": 300
      }
    }
  }
}
```

**Linux OS:**

```json
{
  "metrics": {
    "append_dimensions": {
      "InstanceId": "${aws:InstanceId}"
    },
    "metrics_collected": {
      "mem": {
        "measurement": [
          "mem_total",
          "mem_used"
        ],
        "metrics_collection_interval": 300
      },
      "disk": {
        "resources": [
          "/",
          "/run"
        ],
        "measurement": [
          "disk_total",
          "disk_used"
        ],
        "metrics_collection_interval": 300
      },
      "swap": {
        "measurement": [
          "swap_used_percent"
        ],
        "metrics_collection_interval": 300
      }
    }
  }
}
```

##### From Applications Manager version 181900

The following configuration includes network monitoring metrics in addition to the OS-level metrics.

**Windows OS:**

```json
{
  "metrics": {
    "append_dimensions": {
      "InstanceId": "${aws:InstanceId}"
    },
    "metrics_collected": {
      "LogicalDisk": {
        "measurement": [
          "% Free Space",
          "Free Megabytes"
        ],
        "metrics_collection_interval": 300,
        "resources": [
          "*"
        ]
      },
      "Memory": {
        "measurement": [
          "% Committed Bytes In Use",
          "Available MBytes"
        ],
        "metrics_collection_interval": 300
      },
      "Network Interface": {
        "measurement": [
          "Bytes Received/sec",
          "Bytes Sent/sec",
          "Packets Received/sec",
          "Packets Sent/sec",
          "Packets Received Errors",
          "Packets Outbound Errors",
          "Packets Received Discarded",
          "Packets Outbound Discarded"
        ],
        "metrics_collection_interval": 300,
        "resources": [
          "*"
        ]
      },
      "TCPv4": {
        "measurement": [
          "Connections Established",
          "Connections Active",
          "Connections Passive"
        ],
        "metrics_collection_interval": 300
      },
      "UDPv4": {
        "measurement": [
          "Datagrams/sec"
        ],
        "metrics_collection_interval": 300
      },
      "ENA Packets Shaping": {
        "measurement": [
          "Aggregate inbound BW allowance exceeded",
          "Aggregate outbound BW allowance exceeded",
          "Connection tracking allowance available",
          "Connection tracking allowance exceeded",
          "Link local packet rate allowance exceeded",
          "PPS allowance exceeded"
        ],
        "resources": [
          "*"
        ],
        "metrics_collection_interval": 300
      }
    }
  }
}
```

**Linux OS:**

```json
{
  "metrics": {
    "append_dimensions": {
      "InstanceId": "${aws:InstanceId}"
    },
    "metrics_collected": {
      "mem": {
        "measurement": [
          "mem_total",
          "mem_used"
        ],
        "metrics_collection_interval": 300
      },
      "disk": {
        "resources": [
          "/",
          "/run"
        ],
        "measurement": [
          "disk_total",
          "disk_used"
        ],
        "metrics_collection_interval": 300
      },
      "swap": {
        "measurement": [
          "swap_used_percent"
        ],
        "metrics_collection_interval": 300
      },
      "net": {
        "measurement": [
          "net_bytes_recv",
          "net_bytes_sent",
          "net_packets_recv",
          "net_packets_sent",
          "net_err_in",
          "net_err_out",
          "net_drop_in",
          "net_drop_out"
        ],
        "metrics_collection_interval": 300,
        "resources": [
          "*"
        ]
      },
      "netstat": {
        "measurement": [
          "tcp_established",
          "tcp_time_wait",
          "tcp_close_wait",
          "tcp_listen",
          "tcp_syn_sent",
          "udp_socket"
        ],
        "metrics_collection_interval": 300
      },
      "ethtool": {
        "metrics_include": [
          "bw_in_allowance_exceeded",
          "bw_out_allowance_exceeded",
          "conntrack_allowance_available",
          "conntrack_allowance_exceeded",
          "linklocal_allowance_exceeded",
          "pps_allowance_exceeded",
          "ena_srd_mode",
          "ena_srd_eligible_tx_pkts",
          "ena_srd_tx_pkts",
          "ena_srd_rx_pkts",
          "ena_srd_resource_utilization"
        ]
      }
    }
  }
}
```

### 4. Install the CloudWatch agent

Users can choose to download and install the CloudWatch agent in either of the following ways:

- **Using Systems Manager to install the agent**

  To procure and install the CloudWatch agent using Systems Manager, please adhere to the instructions [outlined in this guide](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/download-CloudWatch-Agent-on-EC2-Instance-SSM-first.html#install-CloudWatch-Agent-on-EC2-Instance-fleet).

- **In the EC2 instance (command line) directly**

  1. Install directly from the package manager.
  2. The CloudWatch agent is available as a package in the Linux package manager; install the package by entering the following command.

     ```bash
     sudo yum install amazon-cloudwatch-agent
     ```

  3. **(OR)** Download package and install manually.

     1. Download the CloudWatch agent package from [this document](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/download-cloudwatch-agent-commandline.html#download-CloudWatch-Agent-on-EC2-Instance-commandline-first) corresponding to your respective OS. For example:
        - Amazon Linux (x86-64): ([rpm file](https://amazoncloudwatch-agent.s3.amazonaws.com/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm))
        - Ubuntu/Debian (x86-64): ([deb file](https://amazoncloudwatch-agent.s3.amazonaws.com/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb))
        - Windows (x86-64): ([msi file](https://amazoncloudwatch-agent.s3.amazonaws.com/windows/amd64/latest/amazon-cloudwatch-agent.msi))
     2. Install the downloaded package using the respective command:
        - **rpm file**: `sudo rpm -U ./amazon-cloudwatch-agent.rpm`
        - **deb file**: `sudo dpkg -i -E ./amazon-cloudwatch-agent.deb`
        - **exe file**: `msiexec /i amazon-cloudwatch-agent.msi`

### 5. Start the CloudWatch agent

This step requires the JSON configuration created in the [third step](https://www.manageengine.com/products/applications_manager/help/configuring-cloud-watch-agent.html#configurationfile).

**Note:** Restarting the CloudWatch agent is mandatory whenever the configuration is changed.

#### For Systems Manager

- To start the CloudWatch agent with a parameter store configuration using Systems Manager, please refer to the AWS CloudWatch documentation's section "[Step 2: Start the CloudWatch agent using your agent configuration file](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/download-CloudWatch-Agent-on-EC2-Instance-SSM-first.html#install-CloudWatch-Agent-on-EC2-Instance-fleet)" and follow the steps outlined.

#### For Command line

**Linux:**

- Using parameter store configuration

  ```bash
  sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c ssm:configuration-parameter-store-name
  ```

- Using a JSON file

  ```bash
  sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:configuration-file-path
  ```

**Windows (PowerShell):**

- Using parameter store configuration

  ```powershell
  & "C:\Program Files\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1" -a fetch-config -m ec2 -s -c ssm:configuration-parameter-store-name
  ```

- Using a JSON file

  ```powershell
  & "C:\Program Files\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1" -a fetch-config -m ec2 -s -c file:configuration-file-path
  ```

**Note:**

- Replace **configuration-parameter-store-name** with the parameter store name (**Eg**: CwAgent_Windows_Config / CwAgent_Linux_Config) created in the third step.
- Replace **configuration-file-path** with the absolute path to the json configuration file (**Eg**: `C:\Program Files\Amazon\AmazonCloudWatchAgent\config.json`).
- If the agent starts successfully, wait for 5 minutes and verify the presence of the CWAgent namespace in the CloudWatch console, ensuring that values are populated for the metrics.

#### Check CloudWatch agent running status

After performing all the steps to verify if the CloudWatch agent is running, please follow the steps below.

**For Systems Manager:**

- Follow the steps mentioned in [this document](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/troubleshooting-CloudWatch-Agent.html#CloudWatch-Agent-troubleshooting-verify-running) to verify the status of the CloudWatch agent.

**For Command line:**

- **Linux:**

  ```bash
  sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
  ```

- **Windows (PowerShell):**

  ```powershell
  & $Env:ProgramFiles\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1 -m ec2 -a status
  ```