fix(utils): honor the cgroup v2 CPU limit in cpu_count() by charan-rathore · Pull Request #6347 · kserve/kserve

charan-rathore · 2026-10-04T15:32:00Z

What this PR does / why we need it:

kserve.utils.utils.cpu_count() only read the cgroup v1 quota files (/sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us). On a cgroup v2 host those files do not exist, so the open raised inside a bare except and the CPU limit was silently dropped. The function returned the node CPU count instead of the limit, and ModelServer sizes its default worker pool from it.

This change reads the cgroup v2 limit first:

cpu.max of the process cgroup (path from /proc/self/cgroup, line 0::<path>), then /sys/fs/cgroup/cpu.max as the fallback, which is the process cgroup when it runs in its own cgroup namespace.
max <period> means no limit, so the host count and the affinity count stay in effect.
The quota is turned into a whole number of cpus the same way the v1 branch does (int(quota / period)), and a result of zero or less is ignored as before.
If no v2 limit is found, the existing v1 logic runs unchanged.

Which issue(s) this PR fixes:
Fixes #6344

Feature/Issue validation/testing:

New python/kserve/test/test_cpu_count.py with 8 tests, using mocked file reads: v2 quota from the mount root, v2 quota from the process cgroup path, max as unlimited, a limit above the host count, v1 quota still applied, v1 -1 ignored, no cgroup files, and CPU affinity still respected.
Without the change the two v2 tests fail and the other six pass. With the change all 8 pass.
test_cpu_count.py and test_model_server.py together: 13 passed (Python 3.12). black is clean on the two changed files.

Not run: the rest of the python test suite, make precommit, and a real cgroup v2 host or cluster. The tests use mocked file contents, so the parsing follows the kernel cpu.max format as documented but was not checked against a live host.

Special notes for your reviewer:

The cgroup v2 path only checks the process's own cgroup, not its ancestors, which matches how the v1 branch reads a single file. Limits set only on a parent cgroup are not picked up.

Checklist:

Have you added unit/e2e tests that prove your fix is effective or that this feature works?
Has code been commented, particularly in hard-to-understand areas?
Have you made corresponding changes to the documentation? Not needed, the docstring already described this behavior.

Release note:

Fix `kserve.utils.cpu_count()` to honor the cgroup v2 CPU limit, so the default worker count follows the container limit on cgroup v2 nodes.

chrikrah

@charan-rathore this fixes #6344. It needs a rebase first, signed off (git commit -s) so the DCO check passes: #5910 landed on master yesterday and adds its own python/kserve/test/test_cpu_count.py, so GitHub shows a conflict.

The #6344 repro under a real cgroup v2 quota, at head 31f8428 and base 5be27cf:

# head 31f8428, from python/kserve
$ systemd-run --user --scope -q -p CPUQuota=200% python -c 'from kserve.utils.utils import cpu_count; print(cpu_count())'
2
# base 5be27cf, from python/kserve
$ systemd-run --user --scope -q -p CPUQuota=200% python -c 'from kserve.utils.utils import cpu_count; print(cpu_count())'
32
$ python -m pytest test/test_cpu_count.py -q
8 passed                     # head
2 failed, 6 passed           # utils.py from base, tests kept: the two v2 tests
$ python -m pytest test/test_model_server.py -q
5 passed                     # head and base

blocking: the add/add conflict on test_cpu_count.py. Joining both files on top of 7a8b7da gives 15 passed with test_model_server.py.

non-blocking: after that join, test_cpu_count_respects_affinity from #5910 reads the real cpu.max of the test process. Under CPUQuota=100% it fails with assert 1 == 2. Your fake_files fixture in that test would close it.

non-blocking: a quota under one CPU still returns the node count, and CPUQuota=50% gives 32. int(quota / period) is 0 there and cpu_count() skips it, as the v1 branch did.

@Jooho could you take the review once the rebase is pushed?

cpu_count() only read the cgroup v1 quota files, so under a cgroup v2 CPU limit it returned the node CPU count. Read cpu.max for the process cgroup (falling back to the cgroup mount root), treat max as unlimited, and keep the v1 fallback. Signed-off-by: Charan Rathore <[email protected]>

charan-rathore · 2026-10-10T09:09:09Z

Rebased onto master (7a8b7da) and pushed. The add/add on test_cpu_count.py is joined: your two tests plus the eight v1/v2 tests, 10 in the file.

Local runs from python/kserve: pytest test_cpu_count.py - 10 passed; pytest test_model_server.py - 5 passed.
Applied your suggestion: test_cpu_count_respects_affinity now runs under the fake_files fixture, so a real cpu.max on the test box can no longer leak into it. My box has no cgroup controllers at all, so I could not recreate the CPUQuota=100% failure myself - that conversion rests on your repro. The joined suite is green here.
Signed off - the pre-rebase commit was missing the trailer, which was the DCO failure.
On the sub-1-CPU note: agreed, int(quota / period) is 0 there and cpu_count() skips the limit, matching the v1 branch. Left as is.

@Jooho ready for review.

chrikrah

@charan-rathore approving at a52ba5f. The join, the sign-off and the fake_files move are in. Your box has no cgroup controllers, so here is the run that needs one: the joined file with test_cpu_count_respects_affinity back on monkeypatch fails at CPUQuota=100%, and your version passes.

# head a52ba5f, master 7a8b7da, from python/kserve; CPython 3.12, pytest 9.1.1, cgroup2fs, 32 CPUs
$ systemd-run --user --scope -q -p CPUQuota=100% python -m pytest test/test_cpu_count.py -q
10 passed                     # head
1 failed, 9 passed            # head, affinity test back on monkeypatch
FAILED test/test_cpu_count.py::test_cpu_count_respects_affinity - assert 1 == 2
$ systemd-run --user --scope -q -p CPUQuota=200% python -m pytest test/test_cpu_count.py -q
10 passed                     # head
$ systemd-run --user --scope -q -p CPUQuota=200% python -c 'from kserve.utils.utils import cpu_count; print(cpu_count())'
2                             # head
32                            # master 7a8b7da
$ python -m pytest test/test_cpu_count.py -q
10 passed                     # head
2 failed, 8 passed            # utils.py from 7a8b7da: the two v2 tests
$ python -m pytest test/test_model_server.py test/test_cpu_count.py -q
15 passed                     # head
# from the repo root, ruff 0.14.13 (the pre-commit pin)
$ ruff format --config ruff.toml --check python/kserve/kserve/utils/utils.py python/kserve/test/test_cpu_count.py
2 files already formatted
$ ruff check --config ruff.toml python/kserve/kserve/utils/utils.py python/kserve/test/test_cpu_count.py
All checks passed!
# not run: the e2e suites.

The single commit carries Signed-off-by and the DCO check is green.

@spolti, the Actions runs on a52ba5f are waiting for approval. Could you approve them and take a look? It builds on your #5910 merge in the same file.

Jooho

Thanks for your contribution!

I noticed you already mentioned the parent cgroup limitation in the PR description. Since Kubernetes now supports Pod-level CPU limits, I think this is worth addressing in the future.

I don't think it should block this PR, as the existing cgroup v1 implementation has the same limitation. Would it make sense to create a follow-up issue to track this?

/lgtm

charan-rathore · 2026-10-15T02:33:06Z

@Jooho Thanks for the review! Opened #6393 to track the parent-cgroup limitation, including the Pod-level CPU limit case you mentioned.

Jooho · 2026-10-16T01:22:01Z

Thanks for the update and for creating the follow-up issue! Everything looks good to me.

/approve

github-actions Bot requested review from bartoszmajsak and israel-hdez October 4, 2026 15:32

chrikrah reviewed Oct 10, 2026

View reviewed changes

charan-rathore force-pushed the fix-cpu-count-cgroup-v2-6344 branch from 31f8428 to a52ba5f Compare October 10, 2026 09:09

chrikrah reviewed Oct 11, 2026

View reviewed changes

Jooho reviewed Oct 15, 2026

View reviewed changes

charan-rathore mentioned this pull request Oct 15, 2026

cpu_count() misses CPU limits set on parent cgroups #6393

Open

github-actions Bot approved these changes Oct 16, 2026

View reviewed changes

Jooho merged commit a50eda5 into kserve:master Oct 16, 2026

jlost mentioned this pull request Oct 16, 2026

chore: sync upstream/master to odh/master opendatahub-io/kserve#2110

Closed

maskarb mentioned this pull request Oct 19, 2026

chore: sync upstream/master to odh/master opendatahub-io/kserve#2115

Merged

jlost mentioned this pull request Oct 21, 2026

chore: sync master to release-v0.17 opendatahub-io/kserve#2124

Merged

rhods-devops-app Bot mentioned this pull request Oct 22, 2026

Automated Sync from main to stable red-hat-data-services/kserve#4710

Merged

Fazlul105 mentioned this pull request Oct 23, 2026

fix(utils): honor parent cgroup CPU limits in cpu_count() #6457

Open

2 of 3 tasks

Navigation Menu

Repository navigation

fix(utils): honor the cgroup v2 CPU limit in cpu_count() - #6347

fix(utils): honor the cgroup v2 CPU limit in cpu_count()#6347
Jooho merged 1 commit into
kserve:masterfrom
charan-rathore:fix-cpu-count-cgroup-v2-6344

charan-rathore commented Oct 4, 2026

Uh oh!

chrikrah left a comment

Uh oh!

charan-rathore commented Oct 10, 2026

Uh oh!

chrikrah left a comment

Uh oh!

Jooho left a comment

Uh oh!

charan-rathore commented Oct 15, 2026

Uh oh!

Jooho commented Oct 16, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

charan-rathore commented Oct 4, 2026

Uh oh!

chrikrah left a comment

Choose a reason for hiding this comment

Uh oh!

charan-rathore commented Oct 10, 2026

Uh oh!

chrikrah left a comment

Choose a reason for hiding this comment

Uh oh!

Jooho left a comment

Choose a reason for hiding this comment

Uh oh!

charan-rathore commented Oct 15, 2026

Uh oh!

Jooho commented Oct 16, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants