Skip to content

Instantly share code, notes, and snippets.

@colesbury
Last active June 25, 2026 17:16
Show Gist options
  • Select an option

  • Save colesbury/1e5c1eb4ccc1e626696b568d18d8bbc4 to your computer and use it in GitHub Desktop.

Select an option

Save colesbury/1e5c1eb4ccc1e626696b568d18d8bbc4 to your computer and use it in GitHub Desktop.
Done. Here's the full comparison of PR #150160 (4fbec935d7a, per-type method cache) vs its base (11f032f904c, merge-base with main), both built with CC=clang-20 ./configure -C
--enable-optimizations --with-lto=thin.
Results
fastmark score (3 runs, higher = better):
┌──────┬─────────┬──────────────────┐
│ Run │ base │ PR │
├──────┼─────────┼──────────────────┤
│ 1 │ 18903.0 │ 18992.3 │
├──────┼─────────┼──────────────────┤
│ 2 │ 18751.3 │ 18740.0 │
├──────┼─────────┼──────────────────┤
│ 3 │ 18787.3 │ 18776.3 │
├──────┼─────────┼──────────────────┤
│ mean │ 18813.9 │ 18836.2 (+0.12%) │
└──────┴─────────┴──────────────────┘
perf stat (3 runs each, full suite, pinned to core 4 / P-core, PYTHONHASHSEED=0):
┌──────────────┬──────────┬──────────┬────────┐
│ Metric │ base │ PR │ Δ │
├──────────────┼──────────┼──────────┼────────┤
│ Instructions │ 192.59 B │ 193.45 B │ +0.45% │
├──────────────┼──────────┼──────────┼────────┤
│ Cycles │ 60.97 B │ 60.65 B │ −0.54% │
├──────────────┼──────────┼──────────┼────────┤
│ IPC │ 3.159 │ 3.190 │ +1.0% │
└──────────────┴──────────┴──────────┴────────┘
Verdict
Performance-neutral on single-threaded workloads. The method cache adds ~0.45% more instructions (the cache lookup/insert path), but they retire at slightly higher IPC, so cycle count is
actually marginally lower. Score difference (+0.12%) is well within run-to-run noise (the score range itself spans ~0.8%). No regression.
The type_cache_* counters are the MRO stats — they count the cache that memoizes find_name_in_mro() results. Total lookup counts are near-identical across both builds (27.95M non-dunder, 21.42M
dunder), so the workloads are directly comparable.
┌───────────────────┬─────────────────────┬─────────────────────┬──────────┐
│ │ base (global cache) │ PR (per-type cache) │ Δ │
├───────────────────┼─────────────────────┼─────────────────────┼──────────┤
│ Non-dunder hits │ 27,461,521 │ 27,573,533 │ │
├───────────────────┼─────────────────────┼─────────────────────┼──────────┤
│ Non-dunder misses │ 491,297 │ 379,582 │ −22.7% │
├───────────────────┼─────────────────────┼─────────────────────┼──────────┤
│ hit rate │ 98.242% │ 98.642% │ +0.40 pp │
├───────────────────┼─────────────────────┼─────────────────────┼──────────┤
│ Dunder hits │ 21,365,356 │ 21,413,715 │ │
├───────────────────┼─────────────────────┼─────────────────────┼──────────┤
│ Dunder misses │ 55,748 │ 6,267 │ −88.8% │
├───────────────────┼─────────────────────┼─────────────────────┼──────────┤
│ dunder hit rate │ 99.740% │ 99.971% │ +0.23 pp │
├───────────────────┼─────────────────────┼─────────────────────┼──────────┤
│ Collisions │ 541,175 │ 0 (untracked) │ — │
└───────────────────┴─────────────────────┴─────────────────────┴──────────┘
Interpretation
The per-type cache does exactly what it's designed to do for MRO lookups:
- Dunder misses drop ~89%. Dunders (__init__, __iter__, __hash__, …) are looked up against every type. In the base build they all share one fixed 4096-entry global cache, so hot types
constantly evict each other. Giving each type its own table eliminates that cross-type contention — the dramatic dunder improvement is the headline result.
- Non-dunder misses drop ~23%, same mechanism, smaller magnitude.
- Fewer misses = fewer slow-path find_name_in_mro() walks, and in the free-threaded build each of those slow paths is where the type lock gets taken — so this directly reduces lock traffic,
which is the whole motivation for the PR.
Two caveats on the numbers
1. Collisions = 0 in the PR is not real — it's untracked. Base increments type_cache_collisions (typeobject.c:6210); the PR's typecache.c never increments it anywhere. So that counter reads 0
by omission, not because the per-type cache is collision-free. If you want apples-to-apples collision data, the PR needs an OBJECT_STAT_INC(type_cache_collisions) added in cache_insert()'s
probe loop. Worth fixing for observability.
2. These pystats builds are unoptimized, so the Score (base 11510 / PR 11304) isn't meaningful for timing — only the optimized PGO run from before is. The counts above are what matter here, and
they're robust (deterministic event counts, identical lookup totals).
Net: the earlier PGO run showed the change is timing-neutral single-threaded; these stats show why it's still a win — substantially better MRO cache hit rates, especially for dunders, which
translates to reduced lock contention under free-threading.
Here's the memory comparison — PR #150160 (per-type method cache) vs base, optimized builds, MaxRSS via /usr/bin/time -v and getrusage, all deterministic (identical across repeated runs):
Memory usage
┌───────────────────────────────────────────┬────────────┬────────────┬──────────────────┐
│ Scenario │ base │ PR │ Δ │
├───────────────────────────────────────────┼────────────┼────────────┼──────────────────┤
│ Startup (python -c pass) │ 11,264 KB │ 11,440 KB │ +176 KB (+1.6%) │
├───────────────────────────────────────────┼────────────┼────────────┼──────────────────┤
│ After importing 31 modules (1132 types) │ 32,384 KB │ 32,384 KB │ ~0 │
├───────────────────────────────────────────┼────────────┼────────────┼──────────────────┤
│ Same + every type's cache force-populated │ 32,368 KB │ 32,560 KB │ +192 KB (+0.6%) │
├───────────────────────────────────────────┼────────────┼────────────┼──────────────────┤
│ Full fastmark suite, peak RSS │ 155,036 KB │ 155,244 KB │ +208 KB (+0.13%) │
└───────────────────────────────────────────┴────────────┴────────────┴──────────────────┘
Analysis
The overhead is small, bounded, and roughly constant (~175–210 KB) — it does not scale with total program size. As a fraction it shrinks from +1.6% at the tiny startup footprint to +0.13% at a
realistic 151 MB footprint.
Why it stays bounded, from the struct layout (pycore_typecache.h):
- Each type gets an 8-byte _tp_cache pointer (part of the type object — negligible).
- A struct type_cache is allocated lazily — only when a type actually has a lookup inserted (_PyTypeCache_Insert). Until then every type shares one static empty_cache and costs zero extra
bytes. That's why plain imports (row 2) show no delta: most imported types are never exercised.
- A populated cache is 144 bytes minimum (16-byte header + 8 entries × 16 bytes), doubling on demand. So cost ≈ 144+ bytes × number of actively-used types, which is what rows 3–4 confirm (~170
bytes/type).
- This partially offsets against base, which always carries one fixed global cache (~4096 entries) regardless. The numbers above are the net difference.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment