Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,33 @@ ClusterRole with the `get` verb for the `/metrics/resources` non-resource URL.
On Kubernetes 1.21 you must use the `--show-hidden-metrics-for-version=1.20`
flag to expose these alpha stability metrics.

### kubelet Pressure Stall Information (PSI) metrics

{{< feature-state for_k8s_version="v1.33" state="alpha" >}}

As an alpha feature, Kubernetes lets you configure kubelet to collect Linux kernel
[Pressure Stall Information](https://docs.kernel.org/accounting/psi.html)
(PSI) for CPU, memory and IO usage.
The information is collected at node, pod and container level.
The metrics are exposed at the `/metrics/cadvisor` endpoint with the following names:

```
container_pressure_cpu_stalled_seconds_total
container_pressure_cpu_waiting_seconds_total
container_pressure_memory_stalled_seconds_total
container_pressure_memory_waiting_seconds_total
container_pressure_io_stalled_seconds_total
container_pressure_io_waiting_seconds_total
```

You must enable the `KubeletPSI` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
to use this feature. The information is also exposed in the
[Summary API](/docs/reference/instrumentation/node-metrics#psi).

#### Requirements

Pressure Stall Information requires [Linux kernel versions 4.20 or later](/docs/reference/node/kernel-version-requirements#requirements-psi).

## Disabling metrics

You can explicitly turn off metrics via command line flag `--disabled-metrics`. This may be
Expand Down Expand Up @@ -223,4 +250,4 @@ is encountered that is not allowed with respect to the allow-list constraints.
* Read about the [Prometheus text format](https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md#text-based-format)
for metrics
* See the list of [stable Kubernetes metrics](https://github.com/kubernetes/kubernetes/blob/master/test/instrumentation/testdata/stable-metrics-list.yaml)
* Read about the [Kubernetes deprecation policy](/docs/reference/using-api/deprecation-policy/#deprecating-a-feature-or-behavior)
* Read about the [Kubernetes deprecation policy](/docs/reference/using-api/deprecation-policy/#deprecating-a-feature-or-behavior)
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
title: KubeletPSI
content_type: feature_gate
_build:
list: never
render: false

stages:
- stage: alpha
defaultValue: false
fromVersion: "1.33"
---
Enable kubelet to surface Pressure Stall Information (PSI) metrics in the Summary API and Prometheus metrics.
18 changes: 17 additions & 1 deletion content/en/docs/reference/instrumentation/node-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ description: >-
The [kubelet](/docs/reference/command-line-tools-reference/kubelet/)
gathers metric statistics at the node, volume, pod and container level,
and emits this information in the
[Summary API](https://github.com/kubernetes/kubernetes/blob/7d309e0104fedb57280b261e5677d919cb2a0e2d/staging/src/k8s.io/kubelet/pkg/apis/stats/v1alpha1/types.go).
[Summary API](/docs/reference/config-api/kubelet-stats.v1alpha1/).

You can send a proxied request to the stats summary API via the
Kubernetes API server.
Expand Down Expand Up @@ -43,6 +43,22 @@ in your cluster, and you use a container runtime that supports statistics access
{{< glossary_tooltip term_id="cri" text="Container Runtime Interface">}} (CRI), then
the kubelet [fetches Pod- and container-level metric data using CRI](/docs/reference/instrumentation/cri-pod-container-metrics), and not via cAdvisor.

## Pressure Stall Information (PSI) {#psi}

{{< feature-state for_k8s_version="v1.33" state="alpha" >}}

As an alpha feature, Kubernetes lets you configure kubelet to collect Linux kernel
[Pressure Stall Information](https://docs.kernel.org/accounting/psi.html)
(PSI) for CPU, memory and IO usage. The information is collected at node, pod and container level.
See [Summary API](/docs/reference/config-api/kubelet-stats.v1alpha1/) for detailed schema.
You must enable the `KubeletPSI` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
to use this feature. The information is also exposed in
[Prometheus metrics](/docs/concepts/cluster-administration/system-metrics#psi-metrics).

### Requirements

Pressure Stall Information requires [Linux kernel versions 4.20 or later](/docs/reference/node/kernel-version-requirements#requirements-psi).

## {{% heading "whatsnext" %}}

The task pages for [Troubleshooting Clusters](/docs/tasks/debug/debug-cluster/) discuss
Expand Down
4 changes: 4 additions & 0 deletions content/en/docs/reference/node/kernel-version-requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,10 @@ In [Linux 5.8](https://github.com/torvalds/linux/commit/4a7e89c5ec0238017a757131

In runc document, Kernel older than 5.2 is not recommended due to lack of freezer.

## Pressure Stall Information (PSI) {#requirements-psi}

[Pressure Stall Information](/docs/reference/instrumentation/node-metrics#psi) is supported in Linux kernel versions 4.20 and up.

## Other kernel requirements {#requirements-other}

Some features may depend on new kernel functionalities and have specific kernel requirements:
Expand Down