Last active
June 25, 2026 17:16
-
-
Save colesbury/1e5c1eb4ccc1e626696b568d18d8bbc4 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Done. Here's the full comparison of PR #150160 (4fbec935d7a, per-type method cache) vs its base (11f032f904c, merge-base with main), both built with CC=clang-20 ./configure -C | |
| --enable-optimizations --with-lto=thin. | |
| Results | |
| fastmark score (3 runs, higher = better): | |
| ┌──────┬─────────┬──────────────────┐ | |
| │ Run │ base │ PR │ | |
| ├──────┼─────────┼──────────────────┤ | |
| │ 1 │ 18903.0 │ 18992.3 │ | |
| ├──────┼─────────┼──────────────────┤ | |
| │ 2 │ 18751.3 │ 18740.0 │ | |
| ├──────┼─────────┼──────────────────┤ | |
| │ 3 │ 18787.3 │ 18776.3 │ | |
| ├──────┼─────────┼──────────────────┤ | |
| │ mean │ 18813.9 │ 18836.2 (+0.12%) │ | |
| └──────┴─────────┴──────────────────┘ | |
| perf stat (3 runs each, full suite, pinned to core 4 / P-core, PYTHONHASHSEED=0): | |
| ┌──────────────┬──────────┬──────────┬────────┐ | |
| │ Metric │ base │ PR │ Δ │ | |
| ├──────────────┼──────────┼──────────┼────────┤ | |
| │ Instructions │ 192.59 B │ 193.45 B │ +0.45% │ | |
| ├──────────────┼──────────┼──────────┼────────┤ | |
| │ Cycles │ 60.97 B │ 60.65 B │ −0.54% │ | |
| ├──────────────┼──────────┼──────────┼────────┤ | |
| │ IPC │ 3.159 │ 3.190 │ +1.0% │ | |
| └──────────────┴──────────┴──────────┴────────┘ | |
| Verdict | |
| Performance-neutral on single-threaded workloads. The method cache adds ~0.45% more instructions (the cache lookup/insert path), but they retire at slightly higher IPC, so cycle count is | |
| actually marginally lower. Score difference (+0.12%) is well within run-to-run noise (the score range itself spans ~0.8%). No regression. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| The type_cache_* counters are the MRO stats — they count the cache that memoizes find_name_in_mro() results. Total lookup counts are near-identical across both builds (27.95M non-dunder, 21.42M | |
| dunder), so the workloads are directly comparable. | |
| ┌───────────────────┬─────────────────────┬─────────────────────┬──────────┐ | |
| │ │ base (global cache) │ PR (per-type cache) │ Δ │ | |
| ├───────────────────┼─────────────────────┼─────────────────────┼──────────┤ | |
| │ Non-dunder hits │ 27,461,521 │ 27,573,533 │ │ | |
| ├───────────────────┼─────────────────────┼─────────────────────┼──────────┤ | |
| │ Non-dunder misses │ 491,297 │ 379,582 │ −22.7% │ | |
| ├───────────────────┼─────────────────────┼─────────────────────┼──────────┤ | |
| │ hit rate │ 98.242% │ 98.642% │ +0.40 pp │ | |
| ├───────────────────┼─────────────────────┼─────────────────────┼──────────┤ | |
| │ Dunder hits │ 21,365,356 │ 21,413,715 │ │ | |
| ├───────────────────┼─────────────────────┼─────────────────────┼──────────┤ | |
| │ Dunder misses │ 55,748 │ 6,267 │ −88.8% │ | |
| ├───────────────────┼─────────────────────┼─────────────────────┼──────────┤ | |
| │ dunder hit rate │ 99.740% │ 99.971% │ +0.23 pp │ | |
| ├───────────────────┼─────────────────────┼─────────────────────┼──────────┤ | |
| │ Collisions │ 541,175 │ 0 (untracked) │ — │ | |
| └───────────────────┴─────────────────────┴─────────────────────┴──────────┘ | |
| Interpretation | |
| The per-type cache does exactly what it's designed to do for MRO lookups: | |
| - Dunder misses drop ~89%. Dunders (__init__, __iter__, __hash__, …) are looked up against every type. In the base build they all share one fixed 4096-entry global cache, so hot types | |
| constantly evict each other. Giving each type its own table eliminates that cross-type contention — the dramatic dunder improvement is the headline result. | |
| - Non-dunder misses drop ~23%, same mechanism, smaller magnitude. | |
| - Fewer misses = fewer slow-path find_name_in_mro() walks, and in the free-threaded build each of those slow paths is where the type lock gets taken — so this directly reduces lock traffic, | |
| which is the whole motivation for the PR. | |
| Two caveats on the numbers | |
| 1. Collisions = 0 in the PR is not real — it's untracked. Base increments type_cache_collisions (typeobject.c:6210); the PR's typecache.c never increments it anywhere. So that counter reads 0 | |
| by omission, not because the per-type cache is collision-free. If you want apples-to-apples collision data, the PR needs an OBJECT_STAT_INC(type_cache_collisions) added in cache_insert()'s | |
| probe loop. Worth fixing for observability. | |
| 2. These pystats builds are unoptimized, so the Score (base 11510 / PR 11304) isn't meaningful for timing — only the optimized PGO run from before is. The counts above are what matter here, and | |
| they're robust (deterministic event counts, identical lookup totals). | |
| Net: the earlier PGO run showed the change is timing-neutral single-threaded; these stats show why it's still a win — substantially better MRO cache hit rates, especially for dunders, which | |
| translates to reduced lock contention under free-threading. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Here's the memory comparison — PR #150160 (per-type method cache) vs base, optimized builds, MaxRSS via /usr/bin/time -v and getrusage, all deterministic (identical across repeated runs): | |
| Memory usage | |
| ┌───────────────────────────────────────────┬────────────┬────────────┬──────────────────┐ | |
| │ Scenario │ base │ PR │ Δ │ | |
| ├───────────────────────────────────────────┼────────────┼────────────┼──────────────────┤ | |
| │ Startup (python -c pass) │ 11,264 KB │ 11,440 KB │ +176 KB (+1.6%) │ | |
| ├───────────────────────────────────────────┼────────────┼────────────┼──────────────────┤ | |
| │ After importing 31 modules (1132 types) │ 32,384 KB │ 32,384 KB │ ~0 │ | |
| ├───────────────────────────────────────────┼────────────┼────────────┼──────────────────┤ | |
| │ Same + every type's cache force-populated │ 32,368 KB │ 32,560 KB │ +192 KB (+0.6%) │ | |
| ├───────────────────────────────────────────┼────────────┼────────────┼──────────────────┤ | |
| │ Full fastmark suite, peak RSS │ 155,036 KB │ 155,244 KB │ +208 KB (+0.13%) │ | |
| └───────────────────────────────────────────┴────────────┴────────────┴──────────────────┘ | |
| Analysis | |
| The overhead is small, bounded, and roughly constant (~175–210 KB) — it does not scale with total program size. As a fraction it shrinks from +1.6% at the tiny startup footprint to +0.13% at a | |
| realistic 151 MB footprint. | |
| Why it stays bounded, from the struct layout (pycore_typecache.h): | |
| - Each type gets an 8-byte _tp_cache pointer (part of the type object — negligible). | |
| - A struct type_cache is allocated lazily — only when a type actually has a lookup inserted (_PyTypeCache_Insert). Until then every type shares one static empty_cache and costs zero extra | |
| bytes. That's why plain imports (row 2) show no delta: most imported types are never exercised. | |
| - A populated cache is 144 bytes minimum (16-byte header + 8 entries × 16 bytes), doubling on demand. So cost ≈ 144+ bytes × number of actively-used types, which is what rows 3–4 confirm (~170 | |
| bytes/type). | |
| - This partially offsets against base, which always carries one fixed global cache (~4096 entries) regardless. The numbers above are the net difference. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment