Skip to content

timely-util: page serialized edge bodies into the buffer pool - #38334

Draft
antiguru wants to merge 1 commit into
MaterializeInc:mainfrom
antiguru:mh/columnar-edge-paging
Draft

antiguru wants to merge 1 commit into
MaterializeInc:mainfrom
antiguru:mh/columnar-edge-paging

Conversation

@antiguru

@antiguru antiguru commented Aug 18, 2026 •

Copy link
Copy Markdown
Member

Serialized column bodies on dataflow edges are owned by whoever holds the container, so their bytes sit outside the buffer pool's ledger and pressure decisions cannot see them. Staging measurement put their lifetime at seconds at the median and minutes in the tail under memory pressure, and their peak in-flight bytes above the whole pool budget. This moves them into the pool, so they are budgeted and evictable between minting and a consumer reaching them.

Column::Align holds an AlignBuffer, either heap words or a pool chunk. With enable_column_edge_paging on and a pool installed, a shipped body is serialized straight into a pool slot. The pool hands out no references, so each consumer copies the body out on first borrow. Fan-out clones share the chunk, which stays in the pool until the last consumer takes it. The word and record counts stay resident, so Timely's record_count at push and pull and length_in_bytes do not materialize it.

enable_column_align_buffer_tracking records per-origin lifetimes and in-flight bytes as mz_column_align_buffer_*, counting heap bytes only since the pool reports paged ones, and Origin::Unpage tracks consumers' copies, so what paging costs on a given workload is visible in its metrics. Both flags are off in production and varied in the test configuration. The paging flag also installs the pool, on the same terms as the chunk-spill gates.

The one staging comparison so far showed no performance movement either way. That is a single workload and does not generalize.

Posted by Claude Code.

🤖 Generated with Claude Code

@antiguru
antiguru force-pushed the mh/columnar-edge-paging branch 2 times, most recently from 5b1875e to 73c9d98 Compare August 19, 2026 13:34
@antiguru
antiguru force-pushed the mh/columnar-edge-paging branch from 73c9d98 to c61670d Compare September 23, 2026 18:24
@antiguru antiguru changed the title DO NOT MERGE: page serialized edge bodies into the buffer pool timely-util: page serialized edge bodies into the buffer pool Sep 23, 2026
antiguru added a commit that referenced this pull request Oct 1, 2026
`Column` served two roles. On a dataflow edge it is built by pushing,
crosses an exchange as channel bytes, and is read on arrival, which is
what its `Typed`, `Bytes`, and `Align` variants are for. Behind an edge,
in a merge batcher's chains, a spill-backed `ColumnChunk`, and a batch
builder's seal, the same type held sorted, consolidated bodies that were
only ever typed or, when read back from a store, serialized words. That
code never constructed `Bytes`, and it asserted the variant it expected
at seven sites with `unreachable!`.

`ColumnBody<C>` is the type at rest: `Typed(C::Container)` while a body
is written, `Words(Vec<u64>)` once a store handed it back, and nothing
else. Writing to a serialized body materializes it by a bulk per-leaf
copy, so the merge, extract, advance, and settle bodies address the
typed containers without matching on a variant. `ColumnChunk`,
`ColumnChunker`, `ColumnMerger`, `UnchunkBuilder`,
`RowRowColPagedBuilder`, and the correction buffer's chunks and builder
now hold bodies. The edge container keeps its name and every operator
that names it is untouched.

The boundaries are moves. A chunker takes whatever the edge delivered
and emits bodies, and a body goes back onto an edge through `From`,
typed staying typed and words becoming `Align`. The column pager's
batcher and `ValRowColPagedBuilder` keep `Column` chains until the pager
is retired, so the merge and extract bodies are generic over a
`MergeChunk` trait both types implement, and `PagedChunker` wraps the
chunker for that batcher.

Fifth in the stack, on top of #38897. The chunked consolidation from
#38900 hands its bodies back onto the edge through `From`. #38334
touches the same files and will need rebasing onto this, since edge
paging lands on `Column` alone, which is the point of the split.

Part of [CPU-279](https://linear.app/materializeinc/issue/CPU-279).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
@antiguru
antiguru force-pushed the mh/columnar-edge-paging branch 6 times, most recently from 9c964d9 to 4de7839 Compare October 8, 2026 20:16
A serialized column body, the payload of `Column::Align`, is owned by
whoever holds the container. Bodies on a dataflow edge therefore belong to
no budget: they sit outside the buffer pool's ledger, so the pool's
pressure decisions cannot see them and the kernel is left to reclaim them,
faulting a Timely worker to do it. Staging measurement put their lifetime
at seconds at the median and minutes in the tail under memory pressure,
and their peak in-flight bytes above the whole pool budget.

`Column::Align` now holds an `AlignBuffer`, which is either heap words or
a pool chunk. With `enable_column_edge_paging` on and a pool installed,
serialization writes a shipped body straight into a pool slot, so the
bytes are budgeted and evictable for the interval between minting the
body and a consumer reaching it. Bodies under 64 KiB, the pool's smallest
size class, stay on the heap. `column_edge_paging_lz4` selects whether
paged bodies are stored under lz4 or identity, separately from the chunk
compression floor so the two can be measured one at a time. The
`edge_codec` bench measures the codec cost and lz4 ratio of shipped
bodies. Whether compression pays under pressure, where it trades that CPU
against swap traffic and compressed-tier residency, has not been measured,
so production stays on identity.

The pool hands out no references, so a consumer's first borrow copies the
body into a buffer of its own. Clones share the chunk: Timely clones a
body once per extra consumer at push time, and copying out there would
leave a heap body outside the budget for as long as the slowest consumer
takes. Every holder but the last reads the chunk and leaves it in the
pool, and the last one takes it, freeing it with the copy.

The state lives behind `AlignBuffer`'s `Deref` and `as_words` rather than
in a fourth `Column` variant, which would touch roughly ninety exhaustive
matches. Two facts stay resident so reading them does not materialize a
paged body: the word count, because `length_in_bytes` is on the hot path,
and the record count, because Timely calls `Accountable::record_count` at
both push and pull and it would otherwise decode the body.

A consumer's copy is allocated and freed on the consumer's thread, so
each worker keeps one retired buffer to refill on the next copy-out. It is one per thread and capped at a ship-sized body,
not one per builder, which would park an allocation in every idle
operator, the reason Timely's own container recycling is disabled.

Each buffer records the `Origin` that minted it, and
`enable_column_align_buffer_tracking` records per-origin lifetimes and
in-flight bytes as `mz_column_align_buffer_*`. In-flight bytes are heap
bytes only, since the pool already reports the paged ones. `Origin::Unpage`
tracks consumers' copies, so the round-trip cost of paging is measured
rather than inferred. Both flags are off in production and varied in the test
configuration. The paging flag also installs the pool, on the same terms
as the chunk-spill gates.

The pool singleton and both gates are process-global with no teardown,
so their tests live in their own binaries.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@antiguru
antiguru force-pushed the mh/columnar-edge-paging branch from 4de7839 to df0f3b2 Compare October 8, 2026 20:33

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant