Skip to content

Exact native live-object limits and lighter standing-order scratch reads - #790

Closed
luisleo526 wants to merge 19 commits into
mainfrom
train/engine-union-20261109
Closed

luisleo526 wants to merge 19 commits into
mainfrom
train/engine-union-20261109

Conversation

@luisleo526

Copy link
Copy Markdown
Collaborator

This pull request combines two reviewed engine changes on top of the current main (e1d29cb9), each merged in as its own plain merge commit, so the release can take them in one verification and one parity run. One small commit on top replaces a working-role word with a neutral label in six notes of the source-bounds manifest (the gate does not read that field).

Live-object limits

  • The native live-object limits are now exact and unbounded by default; a configured limit is enforced at the exact count.
  • Typed I/O failures from native Store status errors are carried through instead of being flattened, and the native Store I/O outcomes are pinned by tests.
  • New regression rows cover the live-object policy, including its population rows in CI and the boundary controls.
  • The allocation-site inventory is regenerated for the exact limits, the checked live-object field assignments are registered, and the source-gate test accounts for the checked field.
  • The policy tests use neutral wording.

Standing-order scratch reads

  • Standing-order reads reuse the bounded scratch search routes instead of building new ones.
  • Lookups of non-local IDs no longer do unnecessary local-route I/O.

Verification

  • Each change was reviewed on its own head before it was merged in here.
  • The combined tree passes the repository's static source guards.
  • The full verification and parity runs are made on this exact head before the merge.

🤖 Generated with Claude Code

luisleo526 and others added 17 commits November 3, 2026 20:00
Retain and invalidate bounded search routes with real key comparisons.
Add ordered, interleaved, interior and handle-growth controls, full witnesses
and backing/comparator failure checks. Preserve the standing-order guards.

Part of #621

All three targeted native executables pass on the candidate; the unchanged
base fails all three. Independent review, full corpus/report verification,
current-main composition and release gates remain pending.
Use bounded prefix hints to reject inapplicable routes before backing reads;
selected routes still compare full stored keys and propagate I/O errors.
Add measured root-reference checks, varied cold cases and EIO controls.

Part of #621

The paired native cold regression fails on the preceding candidate and all
three candidate targets pass with the new source. Long indistinguishable
prefixes use ordinary root descent; no universal performance or acceptance
claim is made. Independent review and required final-tree gates remain open.
Import the repository generator output without edits after the live-object
policy and typed I/O changes. The generated bytes match the recorded output
from the parent source tree exactly.
Add the repository-generated assignment entry for the checked live-object
field. Keep it unreviewed and OPEN, and raise that explicit inventory ceiling
by one to account for the new key rather than claim an unsupported bound.
Place the reason for the single added OPEN key beside its ceiling.
Include its one OPEN assignment key in the real-tree count. Keep the actual
gate invocation, zero-findings check and mutation controls unchanged.
Retain the trade-text inventory and live-object scalar as distinct open keys.
Combine both slow-label additions and update their source-derived accounting.
Keep both histories of the corrected expectations.
Describe the disk capacity requirement without naming an execution host.
Keep the physical refusal diagnostic focused on the missing typed cause.
Six rows of ci/cr150-source-bounds.yaml (U1486, U1529, U1534, U1535,
U1541, U1543) named a working role in their note text. Replace that one
word with "audit". No other character of the manifest changes and its
line count stays 1750.

The note field is only an allowed optional key of an entry: the gate
never reads its content.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

@graphify-labs graphify-labs Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.

Not checked: tests were not run; formal verification proved 0 of 2 changed function(s) (2 not verified).

Formal verification. PR-changed functions: 0/2 verified (0 proven, 0 may-equivalent, 0 distinguished) · 2 not verified (2 unsupported).

Not verified on this run: scratch\_object\_limit\_message (unsupported), create (unsupported).


Graphify review — findings

Changes the internal native Store live-object limit so it defaults to 0 (unbounded) instead of the former 1,048,576 cap. Every positive uint64, including UINT64_MAX, is treated as a finite limit and signed negative input is refused before conversion. Finite cohort refusals still report scratch_capacity_exceeded with resource=cohort but drop the "approved maximum" text, and physical backing failures stay separate. Only native embedders and tests set this parameter, and strategies must be recompiled against the new headers to get the unbounded default. Shape layout is unchanged, so an old plugin's explicit 1,048,576 stays finite on a new engine, while a new plugin's 0 selects the former default on an old engine. Updates the CR150 source-bounds ledger to match: the unreviewed ceiling goes from 1409 to 1410 for the extra scalar key, and the notes for the scratch_object_limit_message and Store reset allocation entries are rewritten for the new message and the unbounded default. The diff is truncated, so the CI preflight/verify script tests and C++ test changes in the symbol list are not shown here.

Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 3292 functions depend on the 2908 functions this change touches.

Health — this change adds coupling hotspots:

  • worse: put_ok() — 22 callers, 4 callees
  • worse: check_rollback_only() — 9 callers, 8 callees
  • worse: expected_ordered_digest() — 11 callers, 3 callees
  • worse: erase_ok() — 9 callers, 3 callees
  • new: verify_retained() — 3 callers, 8 callees
  • worse: expected_state_digest() — 8 callers, 3 callees
  • new: handle_route_read_growth() — 1 callers, 22 callees
  • new: check_finite_boundary() — 2 callers, 11 callees
  • …and 8 more — each is listed as a finding

Verification — 3292 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 3232 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify scratch\_object\_limit\_message.

The verifier did not have enough to check scratch\_object\_limit\_message, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported: signature arity changed (3 -> 2 positional params)

Could not verify: Could not verify create.

The verifier did not have enough to check create, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: unsupported: member function — receiver/self (`this`) not synthesizable from the simple-type literal pools — abstaining rather than guessing a receiver

· 11 grounded finding(s) anchored inline below; 5 more finding(s) on lines outside this diff (see the check run).

return UINT64_C(0xd38a91e65b7402cf) ^ ((index + 1) * UINT64_C(0x9e3779b97f4a7c15));
}

void write_marker(Store& store, Id id, std::uint64_t index, std::uint64_t requested) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — write_marker()

high coupling complexity (Ca·Ce = 12).

Grounded coupling-delta finding (deterministic), not an LLM guess.

require(store.write(id, 0, &value, sizeof(value)) == sizeof(value), "short marker write");
}

void verify_retained(Observation& observation) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — verify_retained()

fans out to 8 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

observation.readback_complete = true;
}

void populate(Observation& observation, std::vector<Id>& ids) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — populate()

fans out to 11 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

observation.report("population_verified");
}

void expect_refusal(Observation& observation, std::vector<Id>& ids) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — expect_refusal()

fans out to 7 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

require(ids.size() == ids_before, "refusal changed retained IDs");
}

Id check_finite_boundary(Observation& observation, std::vector<Id>& ids) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — check_finite_boundary()

fans out to 11 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

verify(*s, model, 512, numeric_probes(), 1, true, "after forced eviction");
}

bool ordered_search_growth() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — ordered_search_growth()

fans out to 18 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

return flat;
}

bool interleaved_range_growth() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — interleaved_range_growth()

fans out to 20 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

return flat;
}

bool interleaved_interior_growth() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — interleaved_interior_growth()

fans out to 19 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

return flat;
}

bool handle_route_read_growth() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — handle_route_read_growth()

fans out to 22 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

return bounded;
}

void ordered_search_error_controls() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — ordered_search_error_controls()

fans out to 19 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

luisleo526 and others added 2 commits November 9, 2026 09:36
The Linux x86_64 GNU libstdc++ build of the integrated tree reports
sizeof(PineExecutionAdapter) = 90352 and sizeof(PineStrategyHost) =
109256 in Release and in Debug, 832 bytes above the pins 89520 and
108424. Move the compile-time and the run-time size pins to the
measured values. No other pin changes in this commit.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The disk identifier table keeps cached key hints now (sizeof
DiskIdTable<std::uint32_t> 584 -> 1000 bytes), and the adapter holds
such tables by value ahead of the pinned members. Every pinned member
declared after them therefore moved by the same 832 bytes the adapter
grew: pine_view_ 88240 -> 89072, retired_row_scratch_ 88248 -> 89080
and staged_scopes_ 88920 -> 89752. The offset of
PineStrategyHost::adapter_ (6440), sizeof(BacktestEngine) (6408) and
every RetiredRowScratch offset are unchanged.

Measured with the probe's own access mechanism on the integrated tree
and on its base. The base reproduces every existing pin value, and the
new sizes equal the Linux x86_64 compiler diagnostics (90352, 109256).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
@luisleo526

Copy link
Copy Markdown
Collaborator Author

Superseded by #791, which carries the same changes as one commit on top of main (plus a test-only update), so the documentation-revert check can read one message. The branch of this pull request is kept.

@luisleo526 luisleo526 closed this Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant