Amazon Bedrock Monitoring
Amazon Bedrock - Overview
Amazon Bedrock is a fully managed service that provides access to high-performing foundation models (FMs) from leading AI companies through a single API. It enables developers to build and scale generative AI applications using techniques such as fine-tuning, retrieval-augmented generation (RAG), and agent orchestration - without managing the underlying infrastructure.
Monitoring Amazon Bedrock is essential for maintaining application performance, controlling costs, and ensuring reliable AI inference operations. Applications Manager's Amazon Bedrock monitoring tool provides real-time visibility into key metrics such as invocation latency, token consumption, throttling events, and log delivery status, enabling you to detect performance bottlenecks, track quota usage, and ensure successful delivery of model invocation logs.
Creating a new Amazon Bedrock monitor
To learn how to create a new Amazon Bedrock monitor, refer here.
Monitored Parameters
Go to the Monitors Category View by clicking the Monitors tab. Click on the Bedrock instance available under Amazon in the Cloud Apps section. Displayed below is the Amazon Bedrock bulk configuration view distributed into three tabs:
- Availability tab gives the availability history for the past 24 hours or 30 days.
- Performance tab gives the health status and events for the past 24 hours or 30 days.
- List view tab enables you to perform bulk admin configurations.
By clicking a monitor from the list, you'll be taken to the Amazon Bedrock dashboard which includes the following tabs:
Performance Overview
| Parameter | Description |
|---|
| ERROR TRACKING |
| Invocation Client Errors | The total number of 4xx errors returned when a request is invalid between the poll interval. |
| Invocation Server Errors | The total number of 5xx errors returned due to internal service issues between the poll interval. |
| Throttled Invocations | The total number of requests throttled due to exceeding service quotas between the poll interval. |
| ESTIMATED TPM QUOTA USAGE |
| Estimated TPM Quota Usage | The average estimated tokens per minute (TPM) quota consumption across the Converse, ConverseStream, InvokeModel, and InvokeModelWithResponseStream API operations between the poll interval. |
| INVOCATION LATENCY |
| Invocation Latency | The average time for a model to respond to a request between the poll interval (in ms). |
| TIME TO FIRST TOKEN |
| Time to First Token | The average time until the first chunk of a streaming response is received between the poll interval (in ms). |
| INPUT TOKENS |
| Input Tokens | The total number of input tokens processed by the models between the poll interval. |
| OUTPUT TOKENS |
| Output Tokens | The total number of tokens generated by the models between the poll interval. |
| IMAGES GENERATED |
| Images Generated | The total number of images generated by image-generation models between the poll interval. |
| REQUEST TRAFFIC |
| Invocations | The total number of model invocation requests between the poll interval. |
Log Delivery
| Parameter | Description |
|---|
| CLOUDWATCH LOGGING STATUS |
| Successful Log Deliveries (CloudWatch Logs) | The total number of successful log deliveries to CloudWatch Logs between the poll interval. |
| Failed Log Deliveries (CloudWatch Logs) | The total number of failed log deliveries to CloudWatch Logs between the poll interval. |
| S3 LOGGING STATUS |
| Successful Log Deliveries (S3) | The total number of successful log deliveries to the standard S3 bucket between the poll interval. |
| Failed Log Deliveries (S3) | The total number of failed log deliveries to the standard S3 bucket between the poll interval. |
| Successful Log Deliveries (S3 Large Data) | The total number of successful deliveries for large data logs to S3 between the poll interval. |
| Failed Log Deliveries (S3 Large Data) | The total number of failed deliveries for large data logs to S3 between the poll interval. |
