Skip to content

Commit 39fc78c

Browse files
luisleo526claude
andauthored
Train: report-channel test bounds, stdin idle default, na string rule, capacity stop codes (#691)
Combines four open pull requests into one head on current main, so they merge and are verified once. - #690 tests: assert behaviour instead of wall-clock bounds in the report-channel, deadline and compile-limit tests; the drain grace is observed as an event and the stall control's guard is an iteration count. - #686 runner: the stdin feed's idle and message bounds default to none, with `tests/native_live_stdin_idle_e2e.py`. - #688 na: read string-convertible types through the string rule (`include/pineforge/na.hpp`, `tests/test_na_generic_containers.cpp`). - #684 report distinct-id and live-order capacity stops under separate failure codes. Main (d6f03f2) plus the four heads, merged in that order: no conflicts, nothing edited by hand. Supersedes #690, #686, #688 and #684 (their branches are kept). 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## The four pull requests in this train (bodies verbatim) ### #690 tests: assert behaviour instead of wall-clock bounds in the report-channel, deadline and entrypoint rows Main is red on the post-merge Debug and arm jobs since the report-channel change: several tests asserted wall-clock upper bounds (for example "elapsed < 1.8 s"), which slower runners miss. This change removes every elapsed-time bound in these tests, in both directions, and asserts behaviour instead: the terminal record's exit class, failure code, the reason (deadline or stall) and which grace expired. - `tests/report_channel/test_contract.py` - `tests/report_channel_deadlines/test_deadlines.py` - `tests/entrypoint_limits/test_compile_limits.py` The rows were checked red-first by breaking the end path: they fail when the supervisor does not end the run. Test-only; no product code changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ### #686 runner: the stdin feed's idle and message bounds default to no limit ## What changes The native runner's stdin feed (`pineforge-live run --feed -`) ended a run on its own after 15 seconds without data or 15 seconds to complete a line, and it counted its own processing time as "idle". Under the sanitizer builds one read of a burst took longer than the bound, so `native_live_ratio_interval_e2e` died with `native stdin idle or message timeout; 0 actions not sent` although data had just arrived. 1. **The clocks measure waiting, not processing.** After the pass that consumes the complete lines of a read, the idle clock restarts; the message-assembly clock restarts only if that pass consumed a message and a partial line remains. A continuation read that only appends bytes to an unfinished line does not refresh the assembly clock, so a sender that dribbles bytes cannot keep an unfinished line alive past a set message timeout (`runner/main.cpp`, stdin loop). 2. **Default change (user-visible): the stdin feed has no deadline unless you set one.** With neither `--feed-idle-timeout` nor `--feed-message-timeout` given, both default to 0 (no limit), each independently, so a quiet market is not an error. A set value still ends intake. For stdin the flags accept 0 (no limit) and any positive number of seconds (the 300 second upper bound is gone). Network feeds (`--feed-url`, HTTP and WebSocket) and local files keep the default of 15 and the 1 to 300 range, and refuse 0 with a message naming the flag. `--status-interval` is unchanged. A caller that relied on the 15 second default to end a run must close stdin or pass an explicit value (`CHANGELOG.md`, `runner/README.md`, `--help`). 3. **Tests.** `tests/native_live_stdin_idle_e2e.py` (new, registered in `runner/CMakeLists.txt`) covers: a burst whose reports exceed the pipe buffer while the test leaves stdout unread for two seconds (the runner blocks inside the consume pass for longer than the one second bound on any machine; red on main); a partial line and a valid line dribbled over several seconds with a one second message bound (must end intake) and with 0 (must commit); a two second silent gap with idle 0 (survives) and idle 1 (ends); the existing silence row; and a row with no timeout flags and a 17 second silence (must survive). The dribble rows start their clock only after the runner has committed a first bar, so they measure the bound and not the start-up speed of a sanitizer build. The test's own host-side waits are one parameter, `PINEFORGE_STDIN_TEST_HANG_GUARD_SECONDS`: unset, empty or `0` means no bound, a positive number of seconds is that bound, anything else stops the test naming it. `tests/native_live_service_e2e.py` replaces the old stdin refusal rows for 0 and 301 by rows that start and finish cleanly and adds exact-message refusal rows for HTTP and WebSocket feeds. 4. Two documentation anchors that cite `runner/main.cpp` line numbers follow the added lines (`docs/design/native-feature-parity.md`, `docs/live-ledger-budget.md`). No kernel, strategy or report byte changes; only the runner's argument handling and stdin intake loop. ## Consumer sweep (who ran the stdin runner and relied on the 15 second default) Read from the clones as they stand; "none found" is not "proved none". - `pineforge-fund`, `pfund/feed/runner_proc.py:221-223`: passes `--feed -`, `--feed-idle-timeout 300` and `--feed-message-timeout 15` explicitly; unchanged by this PR. The feed process owns the runner's stdin. - `pineforge-data`, `native/feed/README.md` (lines 66-67, 441-442, 547): documentation examples pipe the feeder into `pineforge-live run ... --feed -` with no timeout flags. Today a 15 second silence ended that pipeline; after this change it ends when the feeder exits and closes the pipe, or the process is stopped. Documentation only; no code in that repository launches the runner. - This repository: no non-test caller of the stdin runner. Tests that start it on stdin pass explicit values (`tests/test_native_live_report_cli.cpp` and `tests/native_live_checkpoint_e2e.py`: 60 and 60; `tests/native_live_service_e2e.py`: 1/3 and 3/1), finish a finite input and close stdin (`tests/generate_native_live_opaque_report_golden.py`), or stop the child. - No mention of `pineforge-live`, `--feed -` or the two flags in the application, compute, optimizer, MCP, lab, market, license, data-api, data-worker, dashboard and workflow repositories. - Not readable from here, not guessed: deployed settings that live outside the repositories (job environments and entrypoints, container entrypoint variables), repositories without a local clone, and runtime configuration on the trading box. ## Evidence Head tested: the tip of this branch (tree `bac743d2d258`). Its parent commit differs from it by one test file (the six host-side wait lines of `tests/native_live_stdin_idle_e2e.py`); the runner, CMake, README, CHANGELOG and every other test are byte-identical to that parent. - Cloud verify (four steps: preflight, release, debug, sanitizers) on the parent commit: all four exited 0. Cloud preflight on the tip: exit 0 (60 stages, 0 advisory). - Native-live on the tip: live-sanitizers 120 of 120 passed (`native_live_stdin_idle_e2e` 516.59 s, `native_live_ratio_interval_e2e` 179.05 s, `native_live_service_e2e` 138.94 s). Release native in four shards: shards 1, 2 and 4 passed (410, 371, 414 tests); shard 3 had exactly one failure, the known `test_native_live_report_memory` at its 160 s deadline, which passes alone three times (53.13, 53.43, 52.80 s) and does not touch the stdin runner. `native_live_stdin_idle_e2e` passed in shard 4 in 99.27 s, including the row with no timeout flags and a 17 s silence. live-tsan on the tip: exit 0, and `native_live_stdin_idle_e2e` alone under tsan three times: 213.62, 212.45, 213.50 s, each 1 of 1 passed. live-tsan on the parent commit: 120 of 120 (`native_live_stdin_idle_e2e` 291.84 s, `native_live_ratio_interval_e2e` 157.22 s, `native_live_service_e2e` 132.76 s). - Red first: an earlier version of the new test, run against main's runner, failed at its first (burst) row with `native stdin idle or message timeout; 0 actions not sent`, twice, in 16.9 s; that version's dribble rows were then found to depend on start-up speed under sanitizers (they failed in live-tsan and live-sanitizers) and were rebuilt to start their clock after the runner has committed a first bar. The final test text was not re-run against main. - Reviews (static reads): the first two commits, the review fixes, the default change and the test-host parameter each had an independent read; the last read (tip against its parent) found no P0, P1 or P2. - Known class named, not hidden: shard 3's `test_native_live_report_memory` 160 s deadline red is the subject of a separate change that makes that test's budget a parameter. ## Seeing a stalled feed With no idle or message limit set, a consumer detects a silent feed from the status file: `metrics.source_lag_ms` (runner/README.md:326) is refreshed every `--status-interval` even when no data arrives, so it grows while the feed is stalled. A dedicated `metrics.feed_idle_ms` (time since the last byte read) and the same no-limit default for the network feeds follow in a separate pull request. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ### #688 na: read string-convertible types through the string rule, with rows for the double-conversion clause and the limit stated on the overload ## What `pineforge::is_na` has a non-template overload for `std::string` (an empty string reads as na) and a generic template for everything else that is not arithmetic or convertible to `double`. A type that converts implicitly to `const std::string&` but is not a `std::string` (the shape of the engine's `SessionKey`) fell into the generic template and was always answered "present". This change reads such a type through the string overload instead: - `include/pineforge/na.hpp`: the generic template gains the constraint `!std::is_convertible_v<const T&, const std::string&>`, so a string-convertible type is read through the conversion to `const std::string&` and the `std::string` rule ("is it empty") applies. `std::string_view` and engine string handles such as `ScriptString` are not `std::string`, never convert implicitly to it, and stay never-na; the comment on the overload now says so. - The limit is stated on the overload itself: a plain value container (`std::vector<T, A>`, `std::map`, `std::unordered_map`) cannot hold an na reference, so `is_na` of one is always false, even for an empty container, and `std::vector` cannot tell an na array from an empty one. The comment on the unbound `ScriptArray` state also says what unbound means (no store behind it: default constructed or moved from). ## Tests `tests/test_na_generic_containers.cpp` gains rows for: - a type with only a non-const `operator double()`, called on a non-const lvalue (it must keep answering `isnan(double(x))`, which the existing `T&` clause of the template guarantees); - a type that converts to `const std::string&`: an empty one is na, a non-empty one is not; - `std::string_view{}`: never na. ## Files - `include/pineforge/na.hpp` (header only: no struct, layout or ABI change) - `tests/test_na_generic_containers.cpp` Only calls of `is_na` on a type that converts to `const std::string&` without being a `std::string` change their answer. A search of `src/` and `include/` finds no call that passes the engine's `SessionKey` (the one such type in the engine, `include/pineforge/native_order_identity.hpp`) to `is_na`, so no engine behaviour, hash, state or report byte changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ### #684 Report distinct-id and live-order capacity stops under separate resource labels Both stops ended as scratch_capacity_exceeded with resource cohort, so a consumer could not tell the 65,536-id historical tier from the 4,096-id live cap without parsing the message. The historical count and byte refusals now report cohort_ids_total and the live-order guard reports cohort_orders_live. The slab region, the single-id length refusal and the generic store limit keep cohort. No limit, message or refusal condition changes. The two labels join the resource vocabulary in the failure catalog, its registry and the generated documentation. The memory-management failure table row for scratch_capacity_exceeded is rewritten: the sentence "A slab region ran out, more than 4,096 cohorts are live in the bounded profile, or an id is longer than 64 bytes in the bounded profile" is replaced by one entry per resource label, and the sentence "Admit larger maxima or shorter ids before another attempt" is reworded in that row to also name closing or cancelling live orders sooner. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Commits (verbatim, as git log prints them) a296f35 Merge #684 into the train: report distinct-id and live-order capacity stops separately Co-Authored-By: Claude Opus 5.5 <[email protected]> f796b5e Merge #688 into the train: na reads string-convertible types through the string rule Co-Authored-By: Claude Opus 5.5 <[email protected]> c99b140 Merge #686 into the train: runner stdin feed idle and message bounds default to none Co-Authored-By: Claude Opus 5.5 <[email protected]> 1f3c5db Merge #690 into the train: tests assert behaviour instead of wall-clock bounds Co-Authored-By: Claude Opus 5.5 <[email protected]> 1f54ff8 tests: the drain marker is released by an event, not by a delay The descendant now writes its marker once a file exists that the host creates only after it has seen the producer gone while the descendant is alive. No fixed wait remains in the fixture: a supervisor that stopped draining at once never reads the marker, one that drains for its grace does, decided by the order of events. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> d1ee1b1 tests: the drain marker waits a scenario delay after the producer's exit Written at once, the marker raced into the supervisor's last read even when the finite grace returned immediately, so a grace of zero was not caught. The descendant now writes it 250 ms after it sees the producer gone, well inside the default grace: a grace that returns at once loses it, one that drains for its grace reads it. The assertion stays an event (the line is in the captured diagnostic), not a duration. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> c24fc8c tests: the drain rows accept the marker line in the no-corruption check The captured diagnostic now holds two kinds of complete lines (the descendant's progress line and the marker it writes only after the producer exited); the exact-pattern comparison is replaced by a per-line check that keeps the same guarantee: every complete line is one of the two and a cut last line is a prefix of one. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> b966b43 Merge main into the cohort resource label branch Main now carries the parity-path and arena-bound script changes and the profile environment anchor refresh. The merge is clean: the two changes share only two documentation pages, and both sides' text is kept as it is. No source, test or build file of the label change is touched by the merge. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 5542f18 Merge main into na-string-convertible Brings the current engine main (the parity adapter for a jailed host and the public wording in the design docs) into the branch that lets string-convertible types read through the na string overload. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 6ce6bc4 tests: the drain grace is observed as an event, the control's guard is a count The post-exit drain rows now assert that output the descendant wrote only after the producer had exited reached the host through the supervisor, so a grace that returns at once fails them without any duration. The report channel mutation control's guard is a count of the host's own loop iterations since it saw the stall (80 in the CTest row), not seconds; unset means no guard and the row skips locally. CTest comment keeps its line count, so the pinned CMake lines do not move. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 0478add Merge main into the stdin idle default branch Brings the branch level with main after the parity path changes (#687). No conflicts. Only docs/design/native-feature-parity.md changed on both sides; the two edits are in different places and merged cleanly. Every other file the merge brought in is untouched by this branch. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 7d0994a Merge main (the jailed parity adapter) into the no-wall-bounds branch No conflicts. The only file both sides changed is tests/CMakeLists.txt: main's hunk starts at line 7503, after every line this branch pins, so the private fixture evidence pin at :3416 still holds. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> e15d821 tests: re-pin the private fixture evidence row to the shifted CMake line The report-channel rows added twelve lines to tests/CMakeLists.txt ahead of the pooc close-fill view block, so the evidence token that check_private_fixture_reads pins at :3404 now sits at :3416, and the line cites in its reason text moved by the same twelve. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> a8aaada tests: the mutation control row runs every time with an explicit opt-in deadline test_report_channel_mutation_control sets PINEFORGE_REPORT_TEST_MUTATION_CONTROL_AFTER_STALL_SECONDS=4 itself. That is not a limit on a real run: it is the test host's opt-in deadline parameter given an explicit value (the product default stays no deadline), and the row asserts the terminal record (HostWatchdogExpired), not elapsed time. The row never skips, so ci_verify's skip policy is untouched. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 81e8da4 tests: the report-channel mutation control gets its own CTest row A unittest-internal skip leaves the CTest row green, so the control's skip was visible only inside one log. The control now runs as test_report_channel_mutation_control, which CTest reports as Skipped when PINEFORGE_REPORT_TEST_MUTATION_CONTROL_AFTER_STALL_SECONDS is unset (the mutant never ends without a deadline); the contract row deselects it. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 1990737 tests: no elapsed assertion in either direction; the mutation control's deadline is a parameter The deadline suite drops every lower bound on elapsed time and on the fixtures' own tick gaps (and the re-arm mutation half that depended on one), and the timing data nothing reads any more. The terminal record, the moved bytes and the syscall observations stay the assertions. The report-channel mutation control takes its deadline from PINEFORGE_REPORT_TEST_MUTATION_CONTROL_AFTER_STALL_SECONDS; unset means no deadline, and the row then reports itself skipped because its mutant never ends by design. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> f93075b tests: pin the doctored-grace controls to the printed limits record, drop a dead assertion The two doctored-default controls now match the limits dict the assertion prints ('effective': 3000) instead of any 3000 in the failure context. The assertion after the ready loop in the entrypoint limits row can no longer fail and is removed. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> ab42036 tests: the last fixed wall limits in the same directories go The class sweep found wall limits the earlier commits did not touch: the producer fixture gave up after 15 s waiting for the host to observe a stopped validator; the entrypoint blocked-stderr row bounded its ready loop by the compile wall; test_off_identity.py carried a 600 s watchdog (and its fd-sink and wait deadlines, an exception wrapper and a 5 s wait after kill) and a 30 s git timeout; the host had a 30 s git timeout, a 10 s timeout on the reference run, a 1 s join of its exit-observer thread and a 2 s wait after kill. They now wait without a limit, as the rest of the three directories do (a real hang is caught only by the platform's own maximum job time). Not touched, on purpose: the scenario durations of the fixtures (a descendant that lives N ms, a CPU spinner for N ms, a diagnostic reader that starts after N ms) and the host's optional guard whose default is none: they are inputs of the scenarios, not limits on a run. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 0ad3b43 tests: the doctored-default controls pin their failure; the unused bound_ms parameters go The two doctored-default controls (post-exit drain and stop grace) now expect an AssertionError whose text names the doctored value (3000), so only the limits-record mismatch passes them, not any unrelated assertion or setup failure. The deadline helpers single, write_case, read_case and relay_case no longer take a bound_ms parameter (it had no use left once the external waits were removed), and every call site drops the argument; an unused bound parameter would read as a bound that still exists. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> cca6659 tests: the finalizer observer builds on arm64 and says why when it does not; dead waits and a false comment go The C observer that the report-channel rows load into the validator used the raw pause syscall (SYS_pause), which aarch64 does not have, so `cc -Werror` failed on the arm64 runners, and the build helper swallowed cc's stderr, so the failure showed only as a CalledProcessError. The observer now uses ppoll with no descriptors and no timeout where SYS_pause is absent (it blocks until a signal, as pause does; x86_64 keeps SYS_pause), and the helper raises an AssertionError that carries the cc command and its stderr. The helper's 30 s build timeout is removed too. Also: the cleanup that reaps a fixture child no longer fails after two seconds (it waits as the earlier reap does); Process.line() and Process.result() in the deadline harness lose their timeout parameter and the "external wall bound" branch (no caller passes one), and the unused start time goes; and the comment that named a "no-new-output guard" now names what the repository has, the platform's own maximum job time (the workflows set no timeout-minutes). Co-Authored-By: Claude Sonnet 5.5 <[email protected]> c4a4af6 tests: the two doctored-grace controls now reject the doctored default through the limits record test_post_exit_drain_default_assertions_reject_a_doctored_grace and test_stop_grace_default_assertions_reject_a_doctored_grace expected the removed "returned too late" timing messages. A supervisor whose default grace is doctored from 1000 ms to 3000 ms is still rejected, by the row's own behaviour assertion: the terminal record's limits object shows the effective default (1000 expected, 3000 seen). They now expect an AssertionError from the doctored run, and no longer name a timing message. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 56a8ac6 tests: the report-channel, deadline and entrypoint rows assert behaviour, not wall time No test bounds elapsed time any more. Two rows failed on GitHub's Debug jobs for exactly that reason (test_finite_stall_expires_through_supervisor_diagnostic_pipe: 1.849 not less than 1.8; test_paused_validator_is_ended_by_supervisor_wall: 3.187 not less than 3.15). Removed, in both directions, from tests/report_channel/test_contract.py (cancel delay and exit-after-cancel, finite stall, wall exit, diagnostic stall, validator wall, elapsed_seconds limits, drain grace, producer and validator stop grace, process-exit polling allowance, the hold delay, the delayed-diagnostic floor and the one-second fixture-child reap wait), from tests/entrypoint_limits/test_compile_limits.py (the elapsed windows around the compile wall, the outside timeouts of 4, 5, 7 and 30 s and the 180 s hang-guard constant) and, for the upper bounds, from tests/report_channel_deadlines/test_deadlines.py (assert_timing's upper window and every external wait of the harness: result(N), line(N), wait(timeout=2), the pause wait, the EAGAIN loop bound and the fixture build timeout). The harness now waits until the fixture ends; a real hang is caught by the CI runner's own no-new-output guard. Kept: every behaviour assertion (terminal record, exit class, failure code, reason deadline or stall, which grace expired, stage, limits requested/granted/effective, process tree gone, bytes) and pure order comparisons of the host's own timestamps (cancelled_at before exited_at is causality, not a bound). Deleted, not emptied: check_timing and the old assert_timing (the deadline suite keeps assert_not_early, a deadline must not fire before its time, and the re-arm row keeps its one explicit lower assertion). The mutation control keeps its own finite guard counted from the stall: HostWatchdogExpired firing is the feature under test there. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 94e841e Merge main into na-string-convertible Brings the current engine main into the branch that lets string-convertible types read through the na string overload. The merge has no conflict; the branch's two files (include/pineforge/na.hpp and tests/test_na_generic_containers.cpp) are unchanged by it. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 4ea1174 Merge main 59050b4 into the branch One conflict, in CHANGELOG.md: both sides added entries at the head of the Unreleased section. The resolution keeps every line of main's entries unchanged and keeps this change's default-change entry; no other file conflicted. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 41482b3 Merge main into the cohort resource label branch Main now carries the opt-in supervised report channel, the compile-step limits and the deadline helpers for the container runner. The merge is clean: the two changes share only two documentation pages, and both sides' text is kept as it is. No source, test or build file of the label change is touched by the merge. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 33d620f Merge main into na-string-convertible Brings the current engine main (memory caches, equity-curve window, cohort exposure fold, admission batches) into the branch that lets string-convertible types read through the na string overload. The merge has no conflict; the branch's two files (include/pineforge/na.hpp and tests/test_na_generic_containers.cpp) are unchanged by it. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 9426ac8 Keep the two symbol-claimed citations of the memory bound table when naming the id table labels The memory bound table's cohort row cited the cohort record that holds the five summary words, and the entry-rank row cited the entry-rank key table member with the pair of calls that insert into it. The merge of main 7944e10 replaced both with single-line citations that name no symbol, so this commit is a stated revert of 7944e10 for those two citations: it puts main's wording back, the record at its struct and the member at its require-insert and insert pair, next to the link record and end offset citations the label change added. Only these two table rows change. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> e197275 tests: the stdin idle test's host-side wait bounds are one named parameter, default none native_live_stdin_idle_e2e had numeric host-side bounds (select waits of 180 s and communicate bounds of 180, 60 and 15 s). A slow sanitizer build must not be failed by them and they must not be raised to make a slow box pass (no time limit anywhere, tests included). They are now ONE parameter, PINEFORGE_STDIN_TEST_HANG_GUARD_SECONDS: unset, empty or 0 means no bound; a positive decimal number of seconds is that bound; anything else stops the test at start naming the parameter. All six sites (three select waits, three communicate calls) use it. The runner's own deadlines under test stay as they were: the silence row's wait of 15 s, the poll() checks and the named stimulus sleeps. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 3ab8e63 docs: anchors into runner/main.cpp follow the four lines the stdin default added The stdin default block in runner/main.cpp moves the calc_on_every_tick refusal from line 307 to 311 and ledger.checkpoint_recovered_report from 1116 to 1120; docs/design/native-feature-parity.md:124 and docs/live-ledger-budget.md:146 cite the new lines. (runner/README.md:220 cites 1053 for the same call and was already stale on main; it is not touched here.) Co-Authored-By: Claude Sonnet 5.5 <[email protected]> f9a77d1 runner: the stdin feed's idle and message bounds default to no limit; only a set bound ends intake With neither --feed-idle-timeout nor --feed-message-timeout given, the stdin feed now has no deadline (0), so a quiet market no longer ends a run after 15 seconds. A set value behaves as before; the network feeds (HTTP, WebSocket) and local files keep the default of 15 and the 1 to 300 range and still refuse 0. The help text and runner/README.md say so, and CHANGELOG.md states the default change plainly as user-visible. tests/native_live_stdin_idle_e2e.py gains a row with no timeout flags: after one committed bar and a 17 second silent gap (longer than the old 15 second default) the runner is still alive, accepts a second bar and exits 0 with two inputs. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> c1129b4 tests: the stdin dribble rows start their clock only after the runner has proved it is ready The dribble rows of native_live_stdin_idle_e2e started their stimulus clock right after launching the runner. A sanitizer build needs seconds before its loop reads stdin; it then read the whole dribble in one read, its message clock started at that late read, and the check at the end of the 3.2 s dribble found it alive (partial-dribble failed in live-tsan and live-sanitizers while the Release build passed). The helper now writes one valid bar and reads its committed report first, exactly as the gap rows do, and only then dribbles; the dribbled line is the next bar, so a timeout row ends with one committed input and the unlimited row with two. The wait after the dribble is a bound of 60 s instead of 15 s, not an added wait. Nothing else in the rows changes. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> f762657 docs: live-ledger-budget anchors the post-replay checkpoint_recovered_report call at its new line The stdin timeout validation block added to runner/main.cpp in this change moves ledger.checkpoint_recovered_report(recorded) from line 1108 to line 1116. docs/live-ledger-budget.md:146 cited runner/main.cpp:1108; it now cites 1116. Found by the review of the delta, not by the doc-anchors stage. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 7cb92a8 Re-pin the memory bound table's id table citations after the main merge Main's id table restructuring moved the end-offset member and the cohort and entry-rank tables. The two cohort rows now cite the end offset at its new line, the cohort table member and the entry-rank membership check at their new lines. Only the numbers change. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> c91a744 docs: native-feature-parity anchors the runner's calc_on_every_tick refusal at its new line The stdin timeout validation block added to runner/main.cpp in this change moves the calc_on_every_tick refusal from line 299 to line 307. The doc-anchors stage of preflight reported exactly this one bad anchor on the previous head (docs/design/native-feature-parity.md:124, main.cpp:299 -> main.cpp:307). Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 2404b5c Merge commit '69cddc9d102d2ccf5605231cc508b82c08c7081c' into [internal] c4a8cc4 runner: a continuation read must not refresh the stdin message-assembly clock; the stdin idle test is deterministic and covers the dribble cases Review of the two earlier [internal] commits found that the end-of-pass restart refreshed message_started after every read, including a read that only appended bytes to an already partial line. A sender dribbling one byte per interval without a newline therefore kept the assembly clock young for ever (the 1 MiB frame limit was the only stop). The restart now happens only when the pass consumed a message in this read; last_data is still restarted unconditionally, so processing time never counts as idle and a truly silent pipe still times out. tests/native_live_stdin_idle_e2e.py: the burst row no longer depends on how fast a machine is. The test does not read the runner's stdout for two seconds after the burst (a slow stdout consumer, the production cause), so the runner blocks inside the consume pass for longer than the one second bound on any machine, and the test asserts that more than 262144 report bytes were produced before EOF so the construction cannot pass vacuously. New rows: a partial line dribbled one byte per 0.4 s must time out on the message bound; a valid line dribbled over three seconds times out with a message bound of 1 and commits with a bound of 0; a two second silent gap passes with idle 0 and times out with idle 1; the existing silence row is kept. runner/README.md says when a partial line must finish. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 7944e10 Merge main into the cohort resource label branch Main now carries the cohort exposure fold, the id table restructuring and the re-pinned documentation anchors. The one conflict was the memory-bound table in docs/design/arena-bound.md: the two cohort rows keep the labels that name the link record and the end offset, the other rows take main's side, and every citation number in the table is re-pinned in a follow-up commit from the checker's list. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 8232ae8 runner: --feed-idle-timeout and --feed-message-timeout accept 0 (no limit) and any larger value on the stdin feed The two flags were parsed as 1..300 seconds for every feed, so the stdin bounds could not be switched off. They are now parsed as unsigned seconds; for the stdin feed 0 means no limit and there is no upper cap. The network feeds (HTTP, WebSocket) and local files keep 1..300 and refuse anything else after the feed kind is known, with a message that names the flag and says the bound stays for network feeds. The elapsed time is converted to whole seconds before the comparison, so a large accepted bound cannot narrow. --status-interval keeps 1..300 and the default stays 15 seconds. tests/native_live_service_e2e.py replaces the stdin 0 and 301 refusal rows by rows that start and finish cleanly (0, 301 and the largest unsigned value, a padded partial line included) and adds exact-message refusal rows for HTTP and WebSocket feeds; the malformed, status-interval and local-file refusals stay. The help text and runner/README.md describe the new meaning. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 31c6129 runner: the stdin feed's idle and assembly clocks no longer count the engine's own processing time The stdin loop of the native runner stamped last_data when bytes were read and then consumed every complete message of that read before checking the clock again. When one pass took longer than --feed-idle-timeout (15 s by default; the sanitizer builds of native_live_ratio_interval_e2e did), the next loop head threw "native stdin idle or message timeout" although data had just arrived. After the consume pass the loop now restarts last_data, and the message clock for any partial line left over. A feed that really is silent times out exactly as before. tests/native_live_stdin_idle_e2e.py is registered next to native_live_ratio_interval_e2e. One row pipes a burst whose processing exceeds the one second bounds and requires exit 0, every input committed and the final report equal to the independent batch oracle; the other keeps stdin open and silent after one bar and requires the exact timeout diagnostic. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> f058d4e Merge engine main (the output series limit parameter) into the string-convertible na branch Brings the branch up to date with main so the strict merge ruleset holds. The branch still reads string-convertible types through the string overload of na and adds the rows for the reference double-conversion clause and for string-like and string_view inputs; this merge changes none of those files. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 36439bf Name the symbols of the two id table citations in the arena bound note The two table rows cite a line range of the id table header without naming the symbol the range shows, which the anchor check refuses. Name the node layout and the end offset the ranges point at. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 9e94770 Merge main into the capacity label branch The failure catalog changed on main since this branch's base (the output series limit fields), so the generated registry and the catalog diff record are re-derived on top of main's catalog with the two new capacity labels (registry offsets after the resource vocabulary shift by two; the diff record keeps main's release origin and records both changes). The documentation anchors that cite the catalog and the id table are re-pinned. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 891ecdd Merge engine main (scratch-directory wording) into the string-convertible na branch Brings the branch up to date with main so the strict merge ruleset holds. The branch still reads string-convertible types through the string overload of na and adds the rows for the reference double-conversion clause and for string-like and string_view inputs; this merge changes none of those files. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 53fda49 na: read string-convertible types through the string rule; rows for the double-conversion clause; state the container limit on the overload A type that converts implicitly to const std::string& (such as a session key) used to reach the generic template and answer "present" even when its string was empty. The template is now constrained away from such types, so they take the string overload and are read as strings. String views and other string handles that are not std::string are documented as never na. The constraint that keeps a class with a non-const conversion to double on the old is_na(double) path had no row: a class with only a non-const operator double, called on a non-const lvalue, would have answered false instead of the isnan result. Add rows for a NaN and a non-NaN value through a non-const lvalue, plus rows for the string-convertible type and for a string view. The limit that a plain std::vector, std::map or std::unordered_map is never na (an empty one included) is now stated on the generic overload itself, and the unbound ScriptArray state is described as having no store behind it. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> 4f88b15 Report distinct-id and live-order capacity stops under separate resource labels Both stops ended as scratch_capacity_exceeded with resource cohort, so a consumer could not tell the 65,536-id historical tier from the 4,096-id live cap without parsing the message. The historical count and byte refusals now report cohort_ids_total and the live-order guard reports cohort_orders_live. The slab region, the single-id length refusal and the generic store limit keep cohort. No limit, message or refusal condition changes. The two labels join the resource vocabulary in the failure catalog, its registry and the generated documentation. The memory-management failure table row for scratch_capacity_exceeded is rewritten: the sentence "A slab region ran out, more than 4,096 cohorts are live in the bounded profile, or an id is longer than 64 bytes in the bounded profile" is replaced by one entry per resource label, and the sentence "Admit larger maxima or shorter ids before another attempt" is reworded in that row to also name closing or cancelling live orders sooner. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> ## Merge note Squash of the T7 train (#691), which supersedes #690, #686, #688 and #684 (their branches are kept). Gates on head a296f35: pineforge/parity = sweep exp-engine-pr691-a296f352-20261101, 134/134 cases, neutral PASS (7,989 comparable, coverage lost 0, hard regressions 0, leavers 0; verdict 1e19e178); pineforge/verify = full 8 rj-20261101t064927-314e44 on tree 19f53d5; native-live on a296f35 = live-sanitizers and live-tsan exit 0, native shards 1, 2 and 4 exit 0, shard 3 one known timing-class failure (test_native_live_report_memory) that passed solo three times on the same box. GitHub checks on the head: green. Internal tracking ids in two commit messages are replaced by [internal]; nothing else in them is changed. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent d6f03f2 commit 39fc78c

36 files changed

Lines changed: 885 additions & 515 deletions

‎CHANGELOG.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,22 @@ version number follows semantic versioning over the surfaces the
77

88
## Unreleased
99

10+
- **Default change, user-visible:** `pineforge-live run --feed -` (the stdin
11+
feed) no longer ends a run by itself after 15 seconds without data or 15
12+
seconds to complete a line. With neither flag given, `--feed-idle-timeout`
13+
and `--feed-message-timeout` now default to 0, meaning no limit, so a quiet
14+
market is not an error; the run ends when stdin closes or the process is
15+
stopped. A caller that relied on the 15 second default to end a run must
16+
close stdin or pass an explicit value. A set value still ends intake: for
17+
stdin the flags accept 0 (no limit) and any positive number of seconds (the
18+
300 second upper bound is gone); network feeds (`--feed-url`, HTTP and
19+
WebSocket) and local files keep the default of 15 and the 1 to 300 range, and
20+
refuse 0 with a message naming the flag. The stdin idle clock no longer counts
21+
the engine's own processing time: a long consume pass (a sanitizer build, a
22+
slow report consumer) is not idle, and a partial line is measured from its
23+
first byte or from the end of the pass that precedes it, so a sender that
24+
dribbles bytes cannot keep an unfinished line alive past a set message
25+
timeout.
1026
- An opt-in supervised report channel separates report and diagnostic output
1127
and records terminal delivery facts. It is not yet host-attested and needs
1228
an external sink for now, while the internal snapshot handoff is pending.

‎ci/cr150-source-bounds.yaml‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1264,9 +1264,9 @@ entries:
12641264
- {id: "U1213", file: "tutorial/mtf/generated_ltf.cpp", symbol: "itf", ops: ["decl"], container: "std::string", owner: "generated-cpp", phase: "unclassified", state: "unreviewed", line: 158}
12651265
- {id: "U1214", file: "tutorial/mtf/generated_ltf.cpp", symbol: "new:GeneratedStrategy", ops: ["alloc"], container: "unresolved by the index: new GeneratedStrategy", owner: "generated-cpp", phase: "unclassified", state: "unreviewed", line: 145}
12661266
- {id: "U1215", file: "tutorial/mtf/generated_ltf.cpp", symbol: "stf", ops: ["decl"], container: "std::string", owner: "generated-cpp", phase: "unclassified", state: "unreviewed", line: 159}
1267-
- {id: "U1216", file: "include/pineforge/detail/disk_id_table.hpp", symbol: "<temp:string>", ops: ["temp"], container: "unresolved by the index: std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 246}
1268-
- {id: "U1217", file: "include/pineforge/detail/disk_id_table.hpp", symbol: "key", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 70}
1269-
- {id: "U1218", file: "include/pineforge/detail/disk_id_table.hpp", symbol: "read_key", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 245}
1267+
- {id: "U1216", file: "include/pineforge/detail/disk_id_table.hpp", symbol: "<temp:string>", ops: ["temp"], container: "unresolved by the index: std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 286}
1268+
- {id: "U1217", file: "include/pineforge/detail/disk_id_table.hpp", symbol: "key", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 78}
1269+
- {id: "U1218", file: "include/pineforge/detail/disk_id_table.hpp", symbol: "read_key", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 285}
12701270
- {id: "U1219", file: "include/pineforge/detail/disk_resident_id_table.hpp", symbol: "<temp:const_iterator>", ops: ["temp"], container: "unresolved by the index: const_iterator", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 170}
12711271
- {id: "U1220", file: "include/pineforge/detail/disk_resident_id_table.hpp", symbol: "begin", ops: ["decl"], container: "const_iterator", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 133}
12721272
- {id: "U1221", file: "include/pineforge/detail/disk_resident_id_table.hpp", symbol: "end", ops: ["decl"], container: "const_iterator", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 134}
@@ -1533,12 +1533,12 @@ entries:
15331533
- {id: "U1488", file: "include/pineforge/detail/disk_resident_id_table.hpp", symbol: "at_", ops: ["assign", "decl"], container: "const_iterator", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 99}
15341534
- {id: "U1489", file: "include/pineforge/detail/disk_resident_id_table.hpp", symbol: "held_", ops: ["assign", "decl", "grow"], container: "Held", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 273}
15351535
- {id: "U1490", file: "include/pineforge/detail/disk_resident_id_table.hpp", symbol: "resident_by_key_", ops: ["assign", "decl", "grow"], container: "ResidentKeys", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 272}
1536-
- {id: "U1491", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "message", ops: ["append", "decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 62}
1537-
- {id: "U1492", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_byte_limit_message", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 88}
1538-
- {id: "U1493", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_count_limit_message", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 68}
1539-
- {id: "U1494", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_limit_message", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 60}
1540-
- {id: "U1495", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_limit_number", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 43}
1541-
- {id: "U1496", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_resident_cohort_limit_message", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 108}
1536+
- {id: "U1491", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "message", ops: ["append", "decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 66}
1537+
- {id: "U1492", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_byte_limit_message", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 92}
1538+
- {id: "U1493", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_count_limit_message", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 72}
1539+
- {id: "U1494", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_limit_message", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 64}
1540+
- {id: "U1495", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_limit_number", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 47}
1541+
- {id: "U1496", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_resident_cohort_limit_message", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 112}
15421542
- {id: "U1497", file: "include/pineforge/metrics_fold.hpp", symbol: "held_", ops: ["assign"], container: "unresolved by the index: held_ = ...", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 188}
15431543
- {id: "U1498", file: "include/pineforge/ta.hpp", symbol: "held_", ops: ["assign"], container: "unresolved by the index: held_ = ...", owner: "ta-series", phase: "unclassified", state: "unreviewed", line: 1406}
15441544
- {id: "U1499", file: "src/native_order.cpp", symbol: "held_", ops: ["assign"], container: "unresolved by the index: held_ = ...", owner: "native-kernel", phase: "unclassified", state: "unreviewed", line: 2600}
@@ -1573,7 +1573,7 @@ entries:
15731573
- {id: "U1592", file: "src/report_replay.hpp", symbol: "path_", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 58, note: "First declaration in path_, run_id_, directory_ member group; configuration lifetime. Path/run_id each <= 65536 bytes before construction; scratch directory retains existing admission. No row history stored in strings. OPEN."}
15741574
- {id: "U1593", file: "src/report_sink_writer.cpp", symbol: "scratch", ops: ["grow"], container: "state.scratch Line", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 495, note: "Two companion appends: one header, one record; both clear state.scratch first. Existing Line::append checks the fixed 1 MiB max_line array before copying (:28,39-48), preserving one LF byte, and refuses through WriterError. Run ID is separately limited to 64 KiB before escaping. Unsupported custom-container proof stays OPEN."}
15751575
- {id: "U1594", file: "src/source/pine_strategy_host.cpp", symbol: "replay_rows", ops: ["alloc", "grow"], container: "unique_ptr<detail::ScratchSequence<report::ReplayTrade>>", owner: "source-layer", phase: "unclassified", state: "unreviewed", line: 2422, note: "Allocated only for ON recut after the existing eligibility/ordering guards. One fixed 64 KiB scratch window holds replacement annotations in emit order; logical scratch O(slices). Count equals row_count before replacement at :2565-2568; local destroyed at function exit, so no history retained. OFF unique_ptr stays empty. ScratchSequence extent/overflow and typed IO refusal apply. OPEN."}
1576-
- {id: "U1545", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_object_limit_message", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", note: "CR150-CX 10-27 (A5T strings/maps): function return declaration for the cold object-capacity diagnostic; the function-local message combines fixed literals and four uint64 decimal spellings, then is returned and not accumulated; claimed bound: fixed vocabulary plus four at-most-26-byte numbers, verified by reading: yes, include/pineforge/detail/scratch_errors.hpp:43,114-124", line: 114}
1576+
- {id: "U1545", file: "include/pineforge/detail/scratch_errors.hpp", symbol: "scratch_object_limit_message", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", note: "Function return declaration for the cold object-capacity diagnostic; the function-local message combines fixed literals and four uint64 decimal spellings, then is returned and not accumulated; claimed bound: fixed vocabulary plus four at-most-26-byte numbers, verified by reading: yes, include/pineforge/detail/scratch_errors.hpp:47,118-129", line: 118}
15771577
- {id: "U1546", file: "include/pineforge/map.hpp", symbol: "<temp:const_iterator>", ops: ["temp"], container: "unresolved by the index: const_iterator", owner: "engine-core", phase: "unclassified", state: "unreviewed", note: "CR150-CX 10-27 (A5T strings/maps): scanner alias matches on the MapSequence iterator class, its constructors and begin/end temporaries; actual iterators hold one sequence pointer and one size_t index, with no payload storage or growth; claimed bound: fixed-size iterator, verified by reading: yes, include/pineforge/map.hpp:431-474", line: 431}
15781578
- {id: "U1524", file: "include/pineforge/map.hpp", symbol: "<temp:vector>", ops: ["temp"], container: "unresolved by the index: std::vector<Elem>", owner: "engine-core", phase: "unclassified", state: "unreviewed", note: "CR150-CX 10-27 (A5T strings/maps): conversion-operator type spelling for MapSequence's explicit std::vector<Elem> materialization; the local result reserves count_ and pushes each decoded element, then returns to the caller; claimed bound: at most 50,000 elements for map-produced sequences but no decoded string-byte cap, verified by reading: yes, include/pineforge/map.hpp:492-496,560,886-926", line: 492}
15791579
- {id: "U1525", file: "include/pineforge/map.hpp", symbol: "begin", ops: ["decl"], container: "const_iterator", owner: "engine-core", phase: "unclassified", state: "unreviewed", note: "CR150-CX 10-27 (A5T strings/maps): begin return declaration for a function-local MapSequence iterator; construction copies the sequence pointer and zero index, without allocating payload; claimed bound: fixed-size iterator, verified by reading: yes, include/pineforge/map.hpp:460-474", line: 473}

0 commit comments

Comments
 (0)