Repository navigation
run_json: reserve the report channel, so strategy output on fd 1 cannot corrupt a report - #608
Closed
luisleo526 wants to merge 5 commits into
Closed
luisleo526 wants to merge 5 commits into
luisleo526 wants to merge 5 commits into
Conversation
…ot corrupt a report The harness printed the report or a typed failure on fd 1, but the strategy library it loads runs in the same process and could write to fd 1 as well: one native printf corrupted a successful report. - The host saves the caller's stdout as a private report descriptor before it parses arguments or loads any strategy, redirects native fd 1 to stderr, and writes the report and every typed failure through the saved descriptor. Failed-report discard now discards that descriptor. - A publication guard shared by the main handler and the cleanup keeps a fault after a published result or failure from printing a second document; the exit status stays nonzero and the late diagnostic goes to stderr. - FD sink mode keeps the native NDJSON stream on its caller-supplied stream. A sink or phase descriptor, or the arena slot, that collides with the report channel is refused with the existing run_request_invalid failure. - docker/entrypoint.sh sends ordinary transpile, compile and bind subprocess output to diagnostics; the explicit transpile-only mode keeps stdout. - The documentation states the limit: the loaded library inherits the saved descriptor, so this isolates ordinary native writes and is not a security boundary. - tests/test_run_json_stdout.py with tests/test_run_json_stdout_strategy.cpp loads a stub strategy that writes a marker to fd 1 through the real harness: clean runs print exactly the expected report with the marker only on stderr, a failing run prints exactly one failure document, a fault after publication prints no second document, and colliding descriptors are refused. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…and fix two cases of its test A fault after a published result or failure kept the exit status and printed no second document, but the late exception's message was swallowed by the traceback helper and never reached stderr; it is now written as one diagnostic line to the real stderr. In the test, the stub strategy's printing variant changed one reported input, so the two reports differed in content; both runs now use the same inputs and are still compared byte for byte, and the descriptor-collision case builds its fake owner with the type the parser produces. Every check prints its operands on failure. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…brary calls on sys.stdout The report owner redirected native fd 1 and wrote the report through a saved descriptor in every call, so in-process callers that replace sys.stdout (the harness tests call run_json.main directly) saw an empty output and a missing owner. The fd reservation now happens only in the process entry, _host_main; a library call without an installed owner writes to sys.stdout, looked up at call time, exactly as before, so the existing tests pass unchanged. The publication guard is kept per run in the owner instead of one process-wide latch. A new case calls the library entry with a replaced sys.stdout and checks the old behaviour. The documentation citations of docker/run_json.py that the earlier edit moved are re-pointed. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…sion check, and satisfy two CI guards The descriptor-collision guard of the report channel called os.path.sameopenfile on the caller's --report-sink-fd and --run-phase-fd values before the existing range validation, so an out-of-range number died with an OverflowError and exit status 1 instead of the typed refusal with exit status 2. The collision check now runs only on values that passed the range validation; an out-of-range value keeps its old message and status, and a new case covers a huge descriptor number. The stdout-contract test module now carries the sanitizer and coverage guard its neighbours have, and the private-fixture evidence line numbers that the CMake additions moved are updated. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
| abort.close() | ||
| finally: | ||
| abort.close() | ||
| return 1 # A fault after library publication was diagnosed by the run guard. |
| sys.stderr.flush() | ||
| return 1 | ||
| _REPORT_OUTPUT = output | ||
| status = 1 |
| patch.setenv("PINEFORGE_STDOUT_TEST_MODE", mode or "") | ||
| try: | ||
| status = run_json._host_main(argv) | ||
| except BaseException as error: |
| except BaseException: | ||
| try: | ||
| os.close(fd) | ||
| except OSError: |
| os.dup2(devnull, self.fd, inheritable=False) | ||
| finally: | ||
| os.close(devnull) | ||
| except OSError: |
| finally: | ||
| try: | ||
| os.close(sink_write) | ||
| except OSError: |
…ut, and tighten the channel tests When the host's stdout and stderr are the same open file (a 2>&1 wrapper, a terminal), routing native fd-1 writes to stderr cannot keep them out of the report stream. The host now notices this before redirecting, writes one stderr line, "report channel not isolated: stdout and stderr are the same file", and keeps running: only the isolation claim is lost, so a run that worked before still works. The docs and the changelog say the isolation holds only when the two streams stay separate. argparse help printed through sys.stdout, which in host mode is now fd 1 and therefore stderr; help for run_json.py and its csv2bin form now goes to the original stdout again. A rejected second report publication in library mode no longer points the caller's stdout at the null device. The wording about how the library reaches the report descriptor now says it can write it because it runs in this process, and that the descriptor is non-inheritable so exec'd children do not get it. Reports are ASCII, so the UTF-8 encoding of the channel changes no bytes. New cases: merged streams, help on stdout, descriptor relocation with 3 and 4 occupied, non-ASCII failure text on the channel, and the second library publication. The doc citations that the source shift moved are re-pointed. Two sentences introduced by 67ade9d ("run_json: reserve the report channel, so strategy output on fd 1 cannot corrupt a report") are replaced on purpose. In docs/pages/memory-management.md, "Native code runs in the same process and inherits the saved descriptor, so code that deliberately writes through that inherited file description can s..." is replaced by the sentence that the library can write the descriptor because it runs in this process and that the descriptor is non-inheritable. In docs/pages/scratch-tables.md, "The saved report descriptor is inherited by native code in this process, so this separation is transport isolation rather than a security boundary." is replaced by the same corrected wording; the limit it states (not a security boundary) is kept. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526
added a commit
that referenced
this pull request
Oct 28, 2026
…aling, UDT graphs, metrics replay and the CI admission fix (#616) This lands twelve engine pull requests as one change, verified and measured once on the combined tree: - #593 scratch: a fixed write-through read cache for the adapter's id tables - #594 live runner: a delta report stream by default, and report-path work that does not rescan closed trades - #597 live runner: typed pre-growth stop for the ledger budget and flat pending counts - #608 run_json: reserve the report channel, so strategy output on fd 1 cannot corrupt a report - #586 Pine UDT object graphs on the paged object Store - #578 native order: dormant working book, flat per-point cost for never-filled orders - #579 adapter: skip roster passes and empty-family scans that cannot change anything - #599 adapter: read projected pending orders through one cached duplicate index - #563 adapter: gate the reversal exit cancel on candidate owners instead of every historical id - #591 report sink: retire sealed closed-trade rows on sink routes when the module opts in - #596 metrics: stream the bar Sharpe and Sortino through an ordered, packed return replay - #615 ci: accept release.yml's matrix runner under its fetched path in PR admission Fixes found while testing the combined tree are in the final commits: the retiring ledger's cache sizing, a residency witness that bounds sink-run heap by the declared cache cap and checks zero growth at 100k, 200k and 1M trades, a test fix for a descriptor reuse race, and a standing-orders timing grid whose 20,000-order point is tracked in #621. The member pull requests are closed as landed here.
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
run_jsonreserves the report channel, so strategy output written to fd 1 can no longer corrupt a report.sys.stdout.Lab: all 8 legs pass, and every old harness test case passes unchanged. The paired parity sweep completed 134/134.
Draft until review corrections are folded and main is merged in.
🤖 Generated with Claude Code