Configure AWS data sources: CloudTrail and S3 server Access logs

1. CloudTrail

What is CloudTrail?

CloudTrail is an API log monitoring web service offered by AWS. It enables AWS customers to record API calls and sends these log files to Amazon S3 buckets for storage. The service provides details of API activity such as the identity of the API caller, the time of the API call, the source IP address of the API caller, the requests made and response elements returned by the AWS service. In addition, it captures a few non-API events (AWS service events and AWS Console sign-in events).

CloudTrail can also be configured to publish a notification for every log file that is delivered, allowing users to take action upon log file delivery.

Permissions required by Cloud Security Plus to collect CloudTrail logs

• Required permissions for the S3 bucket that collects your CloudTrail logs: Get*, List*, Delete*
• Required permissions for the SQS subscribed to the S3 bucket that collects CloudTrail logs: GetQueueAttributes, ListQueues, ReceiveMessage, GetQueueUrl, DeleteMessage
• Required permission for the CloudTrail service: DescribeTrails
• Required permission for SNS: ListSubscriptions

How to configure CloudTrail in the AWS console?

CloudTrail can be configured to publish a notification using Simple Notification Service (SNS) for every log file that is delivered. Cloud Security Plus uses a Simple Queue Service (SQS) that subscribes to this SNS and collects logs from CloudTrail.

Configure CloudTrail to produce these notifications then create an SQS in the same region as the S3 bucket collecting logs. Cloud Security Plus will take care of the rest.

The steps to configure CloudTrail depends on three possible scenarios viz.,

• CloudTrail is not configured.
• CloudTrail is configured but SNS is not configured.
• CloudTrail and SNS are both configured.

Check which scenario is applicable to you and then proceed with the corresponding steps.

Scenario 1: CloudTrail is not configured.

• Enabling CloudTrail: For more information, refer the given link:

  http://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html.

• Create an S3 Bucket to store the CloudTrail events.
• Enabling SNS Notifications: See the AWS documentation for instructions:

  http://docs.aws.amazon.com/awscloudtrail/latest/userguide/getting_notifications_top_level.html

  

• Create a new SQS in the same region as the S3 bucket collecting the logs.

  

• Subscribe the SQS to the SNS Notifications that you enabled in step 3.

  

Scenario 2: CloudTrail is configured but SNS isn't configured.

• Select the trail you wish to use.
• Click on “S3” to view the “Advanced” options.
• Enable SNS notifications for log delivery.

• Create a new SNS topic with the suitable name and save the changes made.

  
• Create a new SQS (The same as Scenario 1).
• Subscribe the SQS to the SNS Notifications that you enabled in step 5 (The same as Scenario 1).

Note: If log file encryption is enabled for the CloudTrail, include the decrypt permissions in the policy for the IAM user designated to Cloud Security Plus.

Scenario 3: CloudTrail and SNS are both configured.

• If both CloudTrail and SNS are configured all you have to do is create a new SQS and subscribe that SQS to the SNS notifications. (The same as Scenario 1).

Note: If log file encryption is enabled for the CloudTrail, include the decrypt permissions in the policy for the IAM user designated to Cloud Security Plus.

2. Access Logging for S3 buckets

What is S3 server access logging?

Requests to access S3 bucket can be tracked via access logging. Each access log record provides details about a single access request, such as the requester, bucket name, request time, request action, response status, and error code, if any. This access log information can be useful in identifying the nature of traffic.

Permissions required by Cloud Security Plus to collect S3 server access logs

Cloud Security Plus creates a dedicated bucket to store the server access logs. Therefore, the designated IAM user will need permissions to create a new bucket and read from the bucket.

Required permissions for S3 buckets and objects: ListBucket, GetObject, ListAllMyBuckets, CreateBucket, DeleteBucket, GetBucketLocation, SetRegion, GetBucketAcl, SetBucketAcl, SetBucketLoggingConfiguration, ListObjects*

How to configure S3 server access logs in the AWS console?

Enabling server access logging for an S3 bucket doesn’t require additional configuration in the AWS console. However, Cloud Security Plus creates a dedicated bucket to store the server access logs. Therefore, the designated IAM user will need permissions to create a new bucket and read from the bucket.