Skip to content

Write bounded drawing arenas in place and share their slots between checkpoints - #759

Closed
luisleo526 wants to merge 7 commits into
mainfrom
drawing-bound-in-place
Closed

luisleo526 wants to merge 7 commits into
mainfrom
drawing-bound-in-place

Conversation

@luisleo526

Copy link
Copy Markdown
Collaborator

A bounded drawing arena (the line, box and label arenas the generated code builds from a script's declared max_lines_count, max_boxes_count and max_labels_count) replaced its whole state with a deep copy on every create, every delete and every setter, whether or not a script-state checkpoint still shared it. The copy covered every slot, the text of every label, the directory and the order. A script that moves a few of its live drawings on every bar therefore paid time proportional to the bound on every call, and the first write after each checkpoint paid it again. The unbound arena already avoids this: its records live in blocks of 32 that copies share, and a write clones only the block it reaches.

The new test test_drawing_bound_checkpoint_cost is the bounded twin of test_coof_drawing_checkpoint_cost. It fills an arena to its bound, then runs the scheduler's checkpoint sequence (snapshot, restore, execute, commit) around an execution that deletes the oldest label, creates one, moves three and changes one text, with texts that are never inline so that a record copy allocates:

per execution, after the restore before after
bound 100: records not shared with the checkpoint (mean / max) 100 / 100 63.5 / 96
bound 100: heap blocks allocated (mean / max) 634 / 634 74.6 / 108
bound 1,000: records not shared with the checkpoint (mean / max) 1,000 / 1,000 65.7 / 96
bound 1,000: heap blocks allocated (mean / max) 6,076 / 6,076 83.8 / 115

What changes

  • The slots of a bounded arena live in a DrawingRecordStore, the block store the unbound arena already uses. A copy of the arena's state shares the blocks and copies only the directory and the order (an integer per occupant); a write copies the one block it reaches, and only while another copy still holds it.
  • Creation, deletion and every setter give the arena a state of its own when a checkpoint shares it, then change it in place. Each does every step that can allocate or throw (the state's copy, the copy of the slot's block, the directory's room, the order's growth) before its first change and only steps that cannot throw after it, so a refused allocation leaves the arena and its checkpoint as they were (test_drawing_slot_atomicity refuses an allocation at every site of each operation).
  • The search for the lowest free slot starts at a recorded index below which every slot is occupied, so the order in which slots are reused is unchanged and a creation no longer scans from slot 0.
  • Unchanged: the bound, the typed overflow outcome and its diagnostic, the stale-handle no-ops, the id issue, and every value a script reads.
  • The source allocation-bound gate's reviewed site inventory and manifest move with the change: the two groups for the removed whole-state copies are deleted, the OPEN ceiling drops by two, and the groups for the slots, the directory and the state's allocation gain the new site identities. The inventory is the gate's own proposal for this tree.

Verification

Every run is on Linux aarch64, GCC 13.3, against this branch built from its two commits.

  • Tests first. The test commit alone, on main: the cost row fails (8 passed, 3 failed in the new binary; 1 of the 13 drawing rows red in ctest), for the reason in the table above; its model row (the arena equals a plain ordered-map model of the same operations after every execution) and its unchanged-checkpoint row pass on main by design. With the change: 11 passed, 0 failed, and 13 of 13 drawing rows pass, the twelve older ones unmodified (test_drawing, test_drawing_bound_status, test_drawing_bounded_builtins, test_drawing_bounded_creation_failure, test_drawing_checkpoint, test_drawing_id_domain, test_drawing_slot_atomicity, test_drawing_slot_bound, test_drawing_slot_checkpoint, test_drawing_slot_defaults, test_drawing_slot_reuse, test_coof_drawing_checkpoint_cost).
  • AddressSanitizer and UndefinedBehaviorSanitizer (Debug build): the same 13 drawing rows pass, the new test included (it runs a third of its executions there, and its numbers match the release build's), and so does test_drawing_slot_atomicity, which refuses an allocation at every site while the sanitizers watch for leaks and use after free.
  • The source allocation-bound gate passes with the new inventory, which is byte for byte what the gate itself proposes for this tree; its mutation controls (scripts/test_cr150_source_gate.py, 91 tests) pass with the real-tree count pin moved by two; the dangling-comment-name guard, the native source guard, the public-wording guard and the doc-lint test pass.

Not run on this head: x86, the whole suite, and any run of generated strategies (the code generator emits unbound arenas on main, so no corpus strategy reaches this path until it emits the bound).

Limits

  • The first copy of a freshly created state builds its blocks from the initial vector (O(bound), once per arena); every later copy shares.
  • The directory and the order are still copied once per script execution that writes the arena: an integer per occupant, not a record.
  • A setter's edit is applied where the record is, so it must not throw. Every builtin setter is a scalar store or a string move assignment.

🤖 Generated with Claude Code

luisleo526 and others added 2 commits November 6, 2026 14:10
A bounded drawing arena replaces its whole state with a deep copy on every
create, delete and setter, whether or not a checkpoint shares it, so a script
that moves a few of its live drawings per bar pays O(bound) per call and the
first write after each checkpoint pays it again. The unbound arena already
shares blocks of 32 records between copies and clones only the blocks a write
reaches; this is the bounded twin of test_coof_drawing_checkpoint_cost.

The test fills an arena to its bound (100 and 1,000), then runs the scheduler's
checkpoint sequence (snapshot, restore, execute, commit) around an execution
that deletes the oldest label, creates one, moves three and changes one text,
all with texts that are never inline so a record copy allocates. Per execution
it checks that the arena shares all but a few blocks of records with its
checkpoint (the same record object through both) and that the heap blocks
allocated do not depend on the bound, that the arena equals a plain model of
the same operations after every execution, and that checkpointing an unwritten
arena allocates the same at both bounds.

The cost row is the row the change that follows must turn from red to green;
the model row and the unchanged-checkpoint row are the guards that change has
to keep green.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
A bounded arena replaced its whole state with a deep copy on every create,
delete and setter: every slot, the text of every label, the directory and the
order, whether or not a script-state checkpoint still shared it. A script that
moves a few of its live drawings on every bar therefore paid time proportional
to the bound on every call, and the first write after each checkpoint paid it
again.

The slots now live in a DrawingRecordStore, the block store the unbound arena
already uses: blocks of 32 that a copy of the state shares, one path copied per
write that reaches a shared block. A copy of the state shares the blocks and
copies only the directory and the order (an integer per occupant), the
directory with room for every slot so a creation never reallocates it.
Creation, deletion and every setter give the arena a state of its own when a
checkpoint shares it, then change it in place. Each does every step that can
allocate or throw (the state's copy, the copy of the slot's block, the
directory's room, the order's growth) before its first change and only steps
that cannot throw after it, so a refused allocation leaves the arena and its
checkpoint as they were. The search for the lowest free slot starts at a
recorded index below which every slot is occupied, so slot reuse order is
unchanged and a creation no longer scans every slot.

The bound, the typed overflow outcome, the stale-handle behaviour and every
value a script reads are unchanged.

The source allocation-bound gate (ci/cr150-source-bounds.yaml) pins every
growable-container site of include/pineforge/**, so its reviewed site
inventory moves with this change: the two groups for the removed whole-state
copies are deleted and the OPEN ceiling drops by two; the groups for the slots
(a block store filled by push_back), the directory and the state's allocation
gain the new site identities. The inventory is the gate's own proposal for this
tree; its differences from the previous one are exactly those sites. The
gate's real-tree test pins the number of OPEN groups, so that pin moves by the
same two.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.

Not checked: tests were not run; no formal proof of the changed code.


Graphify review — findings

Switches bounded drawing arenas (the line, box and label arenas built from a script's max_*_count) to write in place, with slots held in shared blocks of 32 that checkpoints reuse, the same way unbound arenas already store records. Creates, deletes and setters no longer deep-copy the whole state, so they and the first write after each script-state checkpoint copy only the blocks of 32 records they touch, instead of paying time proportional to the bound. The bound, overflow outcome, slot reuse order and stale-handle behaviour stay the same; the CR150 source-bounds ceiling drops by two (unreviewed: 1401) and the site list is updated to match, with new tests added for checkpoint cost.

Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 484 functions depend on the 425 functions this change touches.

Health — this change adds coupling hotspots:

  • new: run() — 1 callers, 7 callees
  • new: body() — 1 callers, 6 callees

Verification — 484 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 481 function(s) in the blast radius were not formally verified this run

· 2 grounded finding(s) anchored inline below.

}

// ---- one script execution, on the arena and on its model ---------------------
void body(DrawingArena<LabelRec>& arena, int bar) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — body()

fans out to 6 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

bool final_state_ok = false;
};

Cost run(int bound, int executions) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — run()

fans out to 7 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

luisleo526 and others added 5 commits November 6, 2026 19:14
Main moved to the hosted verification jobs and their documentation since this
branch was cut. No file of this change is touched by those commits and no
conflict arises.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… lines

A bounded drawing arena's edits and checkpoints do no per-record work in the
bound: they copy only the blocks of 32 records they write. They do not cost
O(changes): the first write after a checkpoint still copies the slot directory
and the creation order (one 8-byte integer per occupant), and deleting the
oldest drawing moves the directory's entries. The test header and the
CHANGELOG now say so, and the test's function is renamed to match.

The source allocation-bound manifest keeps its line fields as diagnostics. The
entries of drawing.hpp that the change moved (U1665, U1669, U1670, U1671,
U1672) carry the first line the gate finds for them again. No identity changes.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The CHANGELOG entry and the cost test header of the bounded drawing change said
that the first write after a checkpoint copies "one 8-byte integer per
occupant". The slot directory holds two 32-bit integers per occupant (the id and
the slot) and the creation order one 32-bit integer, so both now say "a
directory entry and an order index per occupant". Words only: no code, no
assertion and no manifest line changes.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Main moved by one commit since the previous merge: the hosted sanitizer builds
get a dedicated data disk (two workflow files). No file of this change is
touched by it and no conflict arises.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…n moved its lines

The bounded drawing checkpoint test adds one source, test_drawing_bound_checkpoint_cost, to the TEST_SOURCES list near the top of tests/CMakeLists.txt, which moves every later line of that file by one. The evidence row for test_pooc_close_fill_view_tapes cites five of those lines: the line the sweep re-reads (3459, now 3460) and four in its reason text (:3443, :3455, :3479 and :3485-3501, now :3444, :3456, :3480 and :3486-3502). test_private_fixture_reads failed with "the token no longer sits on tests/CMakeLists.txt:3459", and test_private_fixture_reads_controls failed with it.

Only those five numbers change; the row's text is otherwise as it was. Each new line was read from the file: 3444 is the EXISTS test of a tape, 3456 the increment of _pf_pcv_emitted, 3460 the condition that is the row's token, 3480 the registration of the test, 3486 the registration of the skipped placeholder.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526 added a commit that referenced this pull request Nov 7, 2026
Combine six engine pull requests into one change, gated once.

Source pull requests and the heads merged here:
- #774 Add live drawing capacity to the runtime limit vocabulary (99b0bfe)
- #764 Preserve direction in trade size accessors (834333f)
- #765 ci: reduce sanitizer debug info footprint (be68cab)
- #773 Include ABI layout probe in PR smoke (4f31c10)
- #772 ci: promote the engine baseline on parity alone (349418e)
- #759 Write bounded drawing arenas in place and share their slots between checkpoints (cecedc0)

Gate on head b4c316b (tree 1a4b2ea): required CI green; one parity sweep over 7,989 probes with no lost coverage, no band movement and no hard regression; one review of the combined diff. In the post-merge native run on the same tree, every native shard and the thread-sanitizer leg pass; the address-sanitizer legs fail on runner disk space (No space left on device), the same jobs that fail on main.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@luisleo526

Copy link
Copy Markdown
Collaborator Author

Merged as part of #776 (combined change, commit 63f3978), which carries this pull request's head unchanged and passed the full gate once. Closing this one; the branch is kept.

@luisleo526 luisleo526 closed this Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant