Repository navigation
Commit 63fb7b3
committed
perf(mem): recommit purged memory on macOS so phys_footprint drops
8126481 built the production allocator as mimalloc's release build
(-DMI_DEBUG=0). That bought a large speed-up, but it raised what macOS
charges an index worker, its phys_footprint:
| peak phys_footprint | v0.11.0 | main |
|---|---:|---:|
| TypeScript | 1.25 GB | 1.94 GB |
| dotnet/runtime | 4.5 GB | 12.7 GB |
The 2026-10-09 bench missed most of this, because it sampled RSS.
The difference is one compile-time branch in mimalloc's unix decommit.
With MI_DEBUG or MI_SECURE>2, a purged range is mprotect(PROT_NONE)'d
and flagged needs_recommit. A release build leaves it "committed".
When the arena hands such a range out again, it is re-accounted as a
whole with MADV_FREE_REUSE (arena.c, _mi_os_reuse). macOS then charges
every page of it to the footprint, touched or not.
On macOS, take the recommit path in every build. Linux and Windows keep
upstream's behaviour; there MADV_DONTNEED already drops RSS at once.
Measured on Apple arm64 (macOS 26.6), main ec4fbd3 vs this change, cold
full indexes interleaved with main (C#: 2 runs each, in the same session
as main):
| corpus | footprint before | footprint after | change | wall time |
|---|---:|---:|---:|---|
| TypeScript | 1.98 GB | 1.09 GB | -45 % | unchanged |
| kubernetes | 2.92 GB | 1.88 GB | -35 % | unchanged |
| dotnet/runtime | 12.6 GB | 4.9 GB | -61 % | unchanged |
| linux kernel | 16.0 GB | 13.5 GB | -16 % | 574 -> 559 s |
The graph is byte-identical: TypeScript, kubernetes and django give
the same nodes, edges and CALLS-set hash.
dotnet/runtime is now close to v0.11.0's 4.5 GB.
Measured but not taken:
- the whole debug build, which costs ~5 % wall time on C#;
- MI_STAT=2, MI_PADDING=1 or MI_ENCODE_FREELIST=1 alone, none of which
moves the footprint;
- MIMALLOC_PURGE_DELAY=0, which is already set.
Vendored file changed in place (precedent 1f674c8); its hash in
scripts/vendored-checksums.txt is updated and security-vendored.sh passes.
Signed-off-by: Martin Vogel <[email protected]>1 parent ec4fbd3 commit 63fb7b3
2 files changed
Lines changed: 13 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1024 | 1024 | | |
1025 | 1025 | | |
1026 | 1026 | | |
1027 | | - | |
| 1027 | + | |
1028 | 1028 | | |
1029 | 1029 | | |
1030 | 1030 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
500 | 500 | | |
501 | 501 | | |
502 | 502 | | |
503 | | - | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
504 | 515 | | |
505 | 516 | | |
506 | 517 | | |
| |||
0 commit comments