Allocate with mimalloc in einfold's Python extension, as datafusion-python does - #83

🪭 Claude (einfold)

For #79's first ask: measure peak memory next to time, einfold off and
on. `python/benchmarks/memory.py` runs each case in a process of its
own, so each peak is that case's alone. The peak is Linux's VmHWM, the
process's peak resident memory, reset just before the query, so it
counts what the query adds above its loaded tables. Each case runs three
times, and the fastest time and largest peak are reported.

The cases:
- #79's contraction `x(s, p, k) · w(k, e)`, at s = 16, 64 and 256;
- demo 1's causal attention, at 1,024 and 4,096 tokens;
- with `XARRAY_SQL` pointing at an xarray-sql checkout, the loss and the
gradients of its GPT demo (`benchmarks/ml/02_nanogpt.py`, in
xqlsystems/xarray-sql#269), at batch 8, 64 channels, context 32.

From xarray-sql's environment, on this stack (#82, #83), on an i7-8700:

| case | einfold off | einfold on |
|---|---:|---:|
| contraction-16 | 104 ms, 106 MB | 8 ms, 71 MB |
| contraction-64 | 302 ms, 171 MB | 14 ms, 103 MB |
| contraction-256 | 1,359 ms, 400 MB | 42 ms, 175 MB |
| attention-1024 | 1,912 ms, 630 MB | 14 ms, 99 MB |
| attention-4096 | 48,905 ms, 6,059 MB | 39 ms, 117 MB |
| gpt-loss | 7,048 ms, 2,320 MB | 7,272 ms, 3,079 MB |
| gpt-gradients | 2,510 ms, 1,184 MB | 1,240 ms, 587 MB |

einfold takes less memory than DataFusion in every case but the GPT's
loss. That's the case #79's second ask is about: `EinFoldExec` reads its
inputs whole, where the host's joins stream their larger side.

It's a script to run by hand, not a test: it needs datafusion-python,
and the GPT cases need xarray-sql and ddxdb (ddx's Python package; ddx,
an XQL Systems project, differentiates SQL queries).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01M96xSez6PZwTmfaGh2eJgS

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>

…memory is what it was before mimalloc 2 (#120)

🪭 Claude (einfold)

Fixes #119: with einfold 0.1.0rc1, fused attention at 32,768 tokens
peaked about 0.1 GB higher than with a build of 5ee2256, at the same
speed.

**The cause is mimalloc 2.** #83 moved einfold for Python from mimalloc
3 to 2, because mimalloc 3 crashes beside PyArrow 24's copy. On #119's
reproduction, on an i7-8700:

| build | peak RSS growth |
|---|---:|
| 0.1.0rc1 from PyPI | +0.33 GB |
| `main`, built locally (mimalloc 2) | +0.33 GB |
| `main` with mimalloc 3 | +0.25 GB |
| this pull request | +0.22 GB |

So the PyPI wheel's build isn't the cause, and neither is most of the
code changed since 5ee2256 (+0.23 GB there). Going back to mimalloc 3
isn't an option: the 3.3.2 that the crate bundles also crashes from
several threads on Linux (microsoft/mimalloc#1287), and the fix isn't
released yet. Tuning mimalloc 2's options through its environment
variables recovered at most 0.04 GB, and not consistently.

**The fix.** mimalloc 2 keeps large freed blocks for reuse, and a query
allocates a few large ones: the dense copies of its inputs and its
output's columns. Blocks of 8 MiB and more now go to the system's
allocator, which gives them back to the operating system when they're
freed. Smaller blocks stay with mimalloc, which is what fixed #83's
problem, a session that grew with each query. On Linux, einfold pins
glibc's mmap threshold at the same size, since glibc would otherwise
raise it after the first frees and keep large blocks in its heap.

## Decisions to review

- **The 8 MiB cutoff.** At 1 MiB, blocks that a session reuses
constantly went to glibc too, and xarray-sql's GPT loss, run 10 times in
one session (#83's measurement), grew to 2.35 GB, against mimalloc 2's
2.22. At 8 MiB, over 20 runs, it holds at 2.25–2.26 GB, while mimalloc 2
alone drifts to 2.34.
- **Speed is unchanged:** attention at 32,768 tokens takes 1.55–1.62 s
per query either way, over five runs of each.
- **Changing einfold's global allocator** affects every allocation in
the extension, so the release workflow, which this pull request
triggers, builds and tests the wheel on every platform.

## How it is tested

The 24 Python tests pass, and the measurements are above. Once mimalloc
releases its fix for 3.3, mimalloc 3 is worth measuring again.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01M96xSez6PZwTmfaGh2eJgS

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>