Kubernetes Monitoring User Guide



Kubernetes - An Overview

Kubernetes (or k8s) is an open-source container orchestration system for automating deployment, scaling and management of application containers across clusters of hosts. Kubernetes clusters can span hosts across public, private, or hybrid clouds. K8s orchestration allows users to build application services across multiple containers, schedule those containers across a cluster, scale those containers, and manage the health of those containers over time.

Monitoring Kubernetes Clusters - What We do

Applications Manager lets administrators adapt monitoring strategies to account for the new infrastructure layers introduced (when adopting containers and the container orchestration) with a distributed Kubernetes environment.

  • Auto-discover the parts and map relationships between objects in the cluster- Kubernetes nodes, namespaces, deployments, replica sets, pods, and containers.
  • Track the capacity and resource utilization of your cluster and be able to drill into specific parts of the cluster.
  • Identify if you have enough nodes in your cluster and resource allocations to existing nodes is sufficient for deployed applications.
  • Ensure all nodes on the cluster are healthy - monitor the CPU and memory for Kubernetes nodes (workers and masters).
  • Ensure all desired pods in a deployment are running and not in a restart loop.
  • Set up alerts for Container restarts to identify issues with either a container or its host that affect performance of their applications.
  • Monitor the performance outliers of the Kubernetes-hosted applications running inside your cluster and track down any individual errors.
  • View the status of Kubernetes Master and Node components – API Server, the Etcd key/value store, Scheduler and Controller.
  • Monitor the Persistent Volume storage that a pod might consume and the Persistent Volume Claim that grants exclusive usage to a pod for storage.

Adding a new Kubernetes monitor

Prerequisites for setting up Kubernetes monitoring: kubectl should be installed on the machine where Kubernetes is installed.

Follow the steps given below to create a new Kubernetes monitor:

  1. Click on New Monitor link. 
  2. Select Kubernetes under Virtualization category.
  3. Specify the Display Name of the Kubernetes Server.
  4. Enter the Cluster hostname/ IP address of the server where Kubernetes is running. 
  5. Enter the credential details like user name and password for authentication, or select the required credentials from the Credential Manager list after enabling the Select from Credential list option.
  6. Check the box to enable Public Key Authentication (Supported for SSH2 only), the SSH Key for SSH authentication.
  7. Specify the command prompt value, which is the last character in your command prompt. Default value is $ and possible values are >, #, etc.
  8. Enter the SSH port. Default SSH port used is 22.
  9. Specify the Polling Interval in minutes.
  10. Choose the Monitor Group with which you want to associate the Kubernetes to, from the combo box (optional). You can choose multiple groups to associate your monitor.
  11. Click Add Monitor(s). This discovers the Kubernetes from the network and starts monitoring it.

Monitored Parameters

Go to the Monitors Category View by clicking the Monitors tab. Click on Kubernetes under the Virtualization table. Displayed is the Kubernetes bulk configuration view distributed into three tabs:

  • Availability tab gives the Availability history for the past 24 hours or 30 days.
  • Performance tab gives the Health Status and events for the past 24 hours or 30 days.
  • List view enables you to perform bulk admin configurations.

On clicking a monitor from the list, you'll be taken to the Kubernetes monitor dashboard. It has five tabs -

Overview

ParameterDescription
NAMESPACE DETAILS
Namespace Name Name of Namespace
Resource Version The version number of Namespace
Namespace Availability Availability of Namespace
Namespace CreatedTime Time at which the Namespace was created
PODS
Used Pods Count Number of pods used
Maximum Pods Count Maximum number of pods
Used Pods Count per Node Number of pods in a single node
CLUSTER USAGE DETAILS
Average Cluster CPU Usage Average CPU used by the cluster
Average Cluster Memory Usage Average memory used by the cluster
NODE
Master Node Count Number of master nodes
Worker Node Count Number of worker nodes
CLUSTER DETAILS Contains detailed information about the cluster
COMPONENT DETAILS Contains detailed information about the component

Node

ParameterDescription
NODE MEMORY DETAILS
Node Memory Limit Maximum limit of Node memory in GiB
Node Memory Requests Number of memory requests
NODE CPU DETAILS
Node CPU Limit Maximum limit of CPU
Node CPU Request Number of CPU requests
NODE MEMORY AND CPU DETAILS
Name Name of the node
Allocatable Memory(GiB) The CPU resources of a node that are available for scheduling in GiB
Memory Limit(%) The maximum limit of memory resource which can be used
Memory Request(%) Number of memory requests in %
Allocatable CPU Processor Count The number of CPU processes that are available
CPU Limit(%) The maximum limit of CPU resource which can be used
CPU Request(%) Number of CPU requests in %
NODE POD DETAILS
Name Name of the pod
Pod Usage Details Total number of pods available with used and free pods split-up.
Kube-system Pod Count Number of Kube state pods
Non-Kube-system Pod Count Number of non-Kube state pods
Image Count Number of images in the node
Used Pod Count Total num of pods present in Kubernetes
Allocatable Pod Count Number of pods that are available
NODE DETAILS
Name Name of the node
OSImage OSImage name
OS Name of the OS in which the container is deployed
Architecture Architecture details
Type Type of node
Kubelet Version The version of Kubelet used
Allocatable Ephemeral Storage(GiB) Size of temporary memory available in GiB
Created Time Time at which the node was created

Pods

ParameterDescription
POD DETAILS
Pod Name Name of the pod
Pod Namespace Namespace in which the pod resides
Pod Node Name Name of the pod-node
Pod Application Name of the pod application.
Pod Type Type of pod.
Pod created The means by which the pod was created.
Pod Running Status Status of the pod.
Pod Start Time The start time of the pod.
Pod Created Time Time at which the pod was created.
POD MEMORY DETAILS
Pods Memory Limit Maximum limit of memory.
Pods Memory Request The number of memory requests.
POD MEMORY AND CPU DETAILS
Pod Name Name of the pod.
Number of Containers The number of containers run by the pod.
Pod CPU Limit(%) The maximum limit of CPU resource which can be used.
Pod CPU Request (%) The number of CPU requests by pod in %.
Pod Memory Limit(%) The maximum limit of memory resource that can be used.
Pod Memory Request(%) The number of memory requested in %.
Pod created The means by which the pod was created.
Pod Persistent Volumes Claim Name of the Claim through which a pod can access the persistent volume.
CONTAINER DETAILS
Container Name Name of the container.
Container Image Name of the container image.
Container Pod Name Name of the container pod.
Container Restart Count The number of times the container has restarted.
Container Running Status Status of the container.
Container Start Time Start time of the container.

Services

ParameterDescription
SERVICE DETAILS
Services Name Name of the service.
Services Namespace Name of the Namespace in which the service resides.
Services Application Name of the Service application.
Service Type Type of the service.
Service Protocol Name of the service protocol.
Service Target Port Name of the port that connects with the service.
Service Created Time Creation time of the service.
DEPLOYMENT DETAILS
Deployment Name Name of the deployment.
Deployment Namespace Namespace where the deployment exists.
Deployment Replica Count The number of replicas in a deployment.
Deployment Available Replica Count Number of available replicas in a deployment.
Deployment Availability Availability of the deployment.

Persistent Volumes

ParameterDescription
PERSISTENT VOLUMES DETAILS
PV Name Name of the Persistent Volume.
PV Status Status of the Persistent Volume.
PV Claim Name of the Persistent Volume Claim.
PV Access Mode The mode through which you can access the Persistent Volume.
PV Storage Class Name of the Persistent Volume storage class.
PV Capacity(GiB) The capacity of the Persistent Volume in GiB.
PV Created Time Creation time of the Persistent Volume.
PERSISTENT VOLUMES CLAIM DETAILS
PVC Name Name of the Persistent Volume Claim.
PVC Namespace Name of the Namespace in which the Claim exists.
PVC Status Status of the Persistent Volume Claim.
PVC Volume Volume of the Persistent Volume Claim.
PV Access Mode The mode through which you can access the Persistent Volume.
PV Storage Class Name of the Persistent Volume storage class.
PVC Requests(GiB) Number of Persistent Volume Claim requests in GiB.
PVC Created Time Creation time of Persistent Volume Claim.