Repository navigation
ci/tests: remove the CI time limits; a platform-maximum stop is a HANG to re-run (EN-116) - #378
Merged
Merged
Conversation
| self.end_headers() | ||
| self.wfile.write(payload) | ||
| except (BrokenPipeError, ConnectionResetError, TimeoutError): | ||
| except (BrokenPipeError, ConnectionResetError): |
luisleo526
force-pushed
the
lv/en116-eng-final
branch
from
October 13, 2026 03:23
7c287af to
e29ecb8
Compare
…r and its verifier stage (red first, EN-116) Owner rule (EN-105 G9, EN-116): slowness never fails a required check. scripts/check_native_include_independence.py compile_object() kills a compile after 120 seconds, raises InfrastructureError, and scripts/ci_verify.py records the stage as failed. This test runs the real helper through the real Driver.invoke()/default_runner with a compiler wrapper that sleeps 122 seconds and then runs the real compiler with the same arguments, and requires exit 0, the object and dependency files, a passed stage with no recorded failure, and no timeout or deadline text in the captured diagnostic or the stage log. It takes about two minutes on purpose. No production code, bound or constant is changed or patched by the test, and no CMake registration or row count changes (the test lives in the already registered scripts/test_ci_verify.py). On this commit it FAILS (the helper's 120 second compile deadline expires); the commits that follow remove the deadlines and it passes. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…r rule) Owner rule (2026-10-12 15:52, verbatim): 不要做任何時間限制 類似的問題已經出現太多次了 很煩 — no time limits of any kind; problems of this kind have come up too often. Two CI jobs failed on 2026-10-12 only because a bound was too tight for a slow row: hosted kernel-only run 38213895073 was killed at 1800 s, and main push run 38217844683 (native-live) died with "test_ci_verify: TIMEOUT 3600 must be positive and less than stage deadline 2820". This removes every `timeout=` bound on a subprocess a CI test or check runs, together with the two local helpers that carried a timeout default (`run_command` in scripts/check_native_include_independence.py, `run` in scripts/prepare_settlement_cpp_abi_base.py), the `GIT_SECONDS` constant of scripts/ci_docs_only.py, and the `timeout` parameter of `run_release` in scripts/pf_release_run.py. Synchronization waits (threading Event/Barrier, Thread.join, Queue.get, sqlite3 connect, socket settimeout, wait_for poll helpers) and product configuration named timeout (connect_timeout_ms, total_timeout_ms, --feed-idle-timeout, --feed-message-timeout) are untouched: they are behaviour under test, not CI limits. scripts/ci_verify.py, scripts/ci_preflight.py, scripts/test_ci_verify.py, scripts/test_ci_preflight.py, tests/CMakeLists.txt, runner/CMakeLists.txt, .github/workflows/*.yml and docs/ci.md belong to the sibling lane and are not touched here. The platform's 360-minute job limit is the only hang catcher left. Bounds removed per file (count): benchmarks/speed/aggregate.py: 1 benchmarks/tests/test_run_all.py: 1 docker/run_json_provenance_test.py: 12 scripts/check_kernel_residuals.py: 1 scripts/check_native_include_independence.py: 6 scripts/check_pending_order_prefix_compiler.py: 1 scripts/check_runtime_budget.py: 2 scripts/ci_docs_only.py: 2 scripts/cpp_abi_pairing.py: 6 scripts/pf_release_run.py: 2 scripts/prepare_settlement_cpp_abi_base.py: 4 scripts/test_ci_docs_only.py: 1 scripts/test_cmake_version_source.py: 1 scripts/test_example_runner.py: 2 scripts/test_native_cpp_versions.py: 1 scripts/test_oracle_fixture_checks.py: 1 scripts/test_private_fixture_reads.py: 4 scripts/test_run_json_outputs.py: 1 scripts/test_twin_parity.py: 1 scripts/test_two_world_registration.py: 1 scripts/update_twin_parity_fixtures.py: 1 tests/checked_settings_e2e.py: 6 tests/confirmed_capabilities_e2e.py: 1 tests/confirmed_capabilities_refusals.py: 1 tests/confirmed_receipt_versions.py: 1 tests/extract_native_engine_complete_host.py: 1 tests/live_phase_a_migration.py: 3 tests/native_live_chart_calendar_e2e.py: 1 tests/native_live_chart_input_e2e.py: 7 tests/native_live_d7_fixtures.py: 3 tests/native_live_e2e.py: 4 tests/native_live_equivalence_e2e.py: 4 tests/native_live_report_e2e.py: 1 tests/native_live_routing_e2e.py: 12 tests/native_live_service_e2e.py: 7 tests/native_live_startup_e2e.py: 1 tests/native_live_tick_tape.py: 4 tests/native_live_websocket_e2e.py: 2 tests/same_calc_bracket_e2e.py: 1 tests/strategy_capabilities_e2e.py: 7 tests/test_compile_inventory_pipeline.py: 3 TOTAL: 122 bounds removed in 41 files. (An earlier version of this message listed scripts/check_native_cpp_abi.py: 8 and a total of 130 in 42 files; that file has no timeout in this commit's parent or in its tree, so it was never part of the change.) Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Owner rule (EN-105 G9 AMENDED, EN-116): slowness never fails a required check. No verifier stage, CTest row, workflow step or profile carries a deadline of ours; the only hang catcher is the CI job envelope at the platform maximum (360 minutes), and a job stopped there is a HANG to re-run, not a failed check. Exit-code, signal, missing-summary and diagnostic checks stay. * scripts/ci_verify.py: the options --test-timeout and --ctest-timeout, their config fields, validators, the row-versus-stage deadline inventory check, the summary fields testTimeoutSeconds and ctestTimeoutSeconds, the runner timeout parameters nobody passes any more and the dead TimeoutExpired handlers are removed; ctest is called with --timeout 0 (no default per-test deadline of ours) instead of 21600; a row's own TIMEOUT property is untouched. * scripts/test_ci_verify.py: the deadline-contract tests assert the removed options, the absent summary fields and unbounded dispatch; the deliberate CTest Timeout case now drives a row's own TIMEOUT 1 property and keeps its exit-8, witness, failed-stage, log and JUnit checks. * scripts/ci_preflight.py, scripts/test_ci_preflight.py, .github/workflows/native-live.yml: the native verification step carries no step deadline; the job-level 360 minutes stay and are still required. * Docs reworded so each sentence is true. Sentences named for the doc-revert gate (old text, first words): "The opt-in --test-timeout N gives CTest a default per-test deadline in seconds"; "Neither is set anywhere in CI any more (owner rule, 2026-10-12"; "With no explicit stage deadline the driver refuses no row for its speed"; "--test-timeout alone is accepted; both flags together still refuse a run whose per-test deadline"; "The native workflow passes neither flag." "The summary records both bounds (null when unset)"; "Their sanitizers CTest stage gets no bound of its own any more"; "Its sanitizer job ran 61.5 min and hit the verifier's then-30-minute CTest stage timeout (no stage carries a bound of ours any more)"; "No bound of the verifier's own is left to bind first"; "Now no stage, row, step, profile or matrix entry carries a time limit of this repository's: every row runs until it finishes"; "Its stages carry no timeout of their own any more -- the old 2400 s stage bound is gone"; "No row, stage or step bound remains: its old 3600 s row bound is what collided with the 2820 s stage deadline"; "Its old per-row 900 s and stage 1200 s deadlines are gone; a row now runs until it finishes"; "Its old 900/1200-s row/stage bounds are gone like live-sanitizers'" (docs/ci.md); "fails on a nonzero exit, a signal, a timeout or a missing summary line" (docs/adr/0001-kernel-adapter-boundary.md, docs/pages/pine-to-native.md twice, docs/pages/native-engine.md). The citations of ci_verify.py:794 became :789 in CONTRIBUTING.md:162, docs/pages/contributing-llm.md:136 and :284 and docs/pages/native-engine.md:4616. Not changed: the CPU-ratio and runtime-budget measurement gates, benchmarks/run_all.sh, the intentional timeout-contract fixtures, runner/ product timeouts and .claude/settings.json. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Owner rule (EN-105 G9, EN-116): slowness never fails a required check; the only hang catcher is the CI job envelope, and an expiry there is a HANG (re-run), not a failed test. Exit codes, results and assertions on what the code under test did stay. Polling intervals and deliberate delays stay; only the overall deadlines go. * tests/native_live_service_e2e.py: the condition helper wait_for() and the delivery/keepalive event waits are unbounded (two caller overrides of 30 s removed; the dead TimeoutExpired cleanup handler removed). * tests/native_live_routing_e2e.py: the receiver barrier, the two newer-action event waits, the receiver and reader joins, the condition helper and the queue read are unbounded; the ledger read retries a locked database (only "database is locked", every other error propagates) instead of failing after 30 s of busy wait. * tests/native_live_tick_tape.py, tests/native_live_websocket_e2e.py: the 30 s and 20 s socket read deadlines and the dead TimeoutError handler are removed. * tests/test_native_live_equivalence_harness.py: seven completion/reap waits, the log-readiness loop and the helper subprocess are unbounded; their exit-code and exact-log assertions stay. * tests/native_live_test_process.py: wait() with no timeout polls until the child ends; the explicit-timeout path stays for the intentional timeout-contract test. Not changed here (kept for a ruling): the elapsed-time assertions that prove a behaviour is prompt or that an idle timeout fired (native_live_service_e2e.py "elapsed < 10", native_live_routing_e2e.py drain bound and two "< 10" SIGTERM checks, native_live_websocket_e2e.py "elapsed >= 14"), the crash-window searches that need the child to still be running (native_live_equivalence_e2e.py accepted.wait(120) against the 120 s ACK hold, native_live_tick_tape.py deadline 600), and the explicit-timeout witness of the process helper. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…t deadlines (EN-116) Owner rule (EN-105 G9, EN-116): slowness never fails a required check; the only hang catcher is the CI job envelope, and an expiry there is a HANG (re-run). Polling intervals stay; the intentional timeout-contract cases stay. * tests/test_native_live_websocket.cpp, tests/test_native_live_store.cpp: the mock servers' socket receive and accept polls wait with no limit (poll(..., -1)) instead of throwing "receive/accept timeout" after 2-5 s; a failing poll still throws. * tests/test_native_live_service.cpp: the callback rendezvous and the periodic-status observation wait without a deadline; their asserts stay. * tests/native_live_service_strategy.cpp: the resume gate waits without a 10 s expiry; its 5 ms poll stays. * scripts/derive_corpus_feeds_self_test.py: the writer barrier and the future results wait without 5 s limits. Kept on purpose: the 20 ms connect / 30 ms total timeout cases and the 1000 ms total with a 50 ms cancellation predicate (they test the product's timeout), and the mock servers' configured delays. Not changed here (kept for a ruling): tests/test_local_time_lock_free_hit.cpp, a witness that releases the timezone lock when its 5 s rendezvous expires. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…psed time (EN-116) Owner rule (EN-105 G9, EN-116): slowness never fails a required check. Four rows asserted that a SIGTERM or a fatal drain finished within a pinned number of seconds, so a slow or loaded host failed them: tests/native_live_service_e2e.py (`elapsed < 10`), tests/native_live_routing_e2e.py (the fatal drain bound of three total timeouts, and the two `< 10` SIGTERM checks). The timing predicates are removed. Every return-code, stopped-state, blocked-receiver, outbox, unsent-action, pending-count and report assertion of those rows stays, so each row still proves that the runner stops, keeps its ledger and loses nothing. The row that asserts a MINIMUM (tests/native_live_websocket_e2e.py `elapsed >= 14`, an idle timeout that must not fire early) is not touched: slowness cannot fail it. Kept on purpose, not verdict bounds: the crash-window searches (tests/native_live_equivalence_e2e.py accepted.wait(120) against the 120 s ACK hold, tests/native_live_tick_tape.py deadline 600), the explicit timeout-contract witness (tests/test_native_live_equivalence_harness.py with TimedProcess.wait(timeout=...) in tests/native_live_test_process.py) and the timezone-lock rendezvous witness (tests/test_local_time_lock_free_hit.cpp). A follow-up redesigns the first two and the last without a wall-clock deadline. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ANG to re-run (EN-116) docs/ci.md now says a job stopped by the platform at its 360-minute maximum is recorded as a HANG to re-run, not as a failed check. The native-live aggregate job saw only the result `cancelled` and printed nothing about it. When any native profile job result is `cancelled` it now prints `HANG (re-run)` with the reason and still exits 1: the check is not green, it is classified. `success` still exits 0; `failure`, `skipped` and an empty result still exit 1 without the HANG line. scripts/ci_preflight.py still pins the `PROFILES_RESULT` environment line and the `test "$PROFILES_RESULT" = success` line; scripts/test_ci_preflight.py asserts the HANG line appears exactly for `cancelled`. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…say the HANG rule exactly (EN-116) A Claude gating read of the EN-116 engine diff found no faked pass and five mechanical leftovers; this commit fixes them and nothing else. * scripts/test_ci_preflight.py: the two subprocess time limits left in this required test are removed (the `--check-ci-workflow` run and the aggregate-script run). The aggregate-script test now also requires exit code exactly 1 for every result other than success, beside the existing success and HANG checks. * tests/native_live_routing_e2e.py, tests/native_live_service_e2e.py: four `Event.wait()` guards that could not fail (a wait without a timeout always returns true, so the `raise` or `assert` after it never fired) are plain waits; three PASS lines no longer advertise a "3x global-timeout guard" or "within 10 s" guard that is gone. The asserts on the behaviour (exit codes, ledger rows, reports) are untouched. * docs/ci.md: the sentence that said a job stopped at the platform's 360-minute maximum "must be recorded as a HANG to re-run, not as a failed check" (the prose and the `preflight` table row) is replaced by what the workflow does: the native-live aggregate job names such a stop `HANG (re-run)` in its log, and the check still fails. Kept on purpose, not time limits of the verifier: the equivalence harness waits (tests/native_live_equivalence_e2e.py, a file no registered row runs), the explicit-timeout contract witness (tests/test_native_live_equivalence_harness.py and tests/native_live_test_process.py), and product configuration named timeout. Known residuals, named in the pull request: tests/native_live_tick_tape.py (600 s mid-tape kill wait, feeds the registered native_live_genuine_retry_e2e row) and tests/test_local_time_lock_free_hit.cpp (5 s read bound). Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526
force-pushed
the
lv/en116-eng-final
branch
from
October 13, 2026 04:15
e29ecb8 to
0703d36
Compare
luisleo526
marked this pull request as ready for review
October 13, 2026 05:39
luisleo526
added a commit
that referenced
this pull request
Oct 14, 2026
* docs: the 1.6.0-rc.1 changelog entry The changelog section for the first candidate of 1.6.0, tag v1.6.0-rc.1 on commit 04607c1: the selected-window contract revision 1.3 and the typed failures behind it (#388), finite trade and summary doubles in every report writer (#384, #388), the favicon of the API documentation (#355), the removed CI time limits and command-line options (#378), and the tests, CI and benchmarks changes since 1.5.0 (#375, #376, #377, #382, #383, #385, #386, #387). The entry is inserted above the 1.5.0 section; no existing line changes. The date is the UTC date of the tag. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * README: link the apex pineforge.dev, not the www host The README header link pointed at https://www.pineforge.dev. The product site lives at the apex, https://pineforge.dev; the www host is being redirected to it, and search indexes both forms today. One link changed, nothing else. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * CHANGELOG: add the selected-window mode and typed failures to the 1.5.0 notes Two things that are in the v1.5.0 tag were missing from its notes: the opt-in selected-window mode with retained pre-roll bars (#363, already in v1.5.0-rc.1) and the typed failures that docker/run_json.py raises for exceptions from the selected-window helper modules (_typed_failure). The new subsection "Additions to the 1.5.0 notes" only states what the code and the #363 text say, with the same "development-only parity candidate" limits (contract revision 1.3-parity, no claim of complete value-ownership coverage, cannot be promoted to stable). It adds lines and changes none. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> --------- Co-authored-by: Claude Sonnet 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Owner rule (2026-10-12): a slow required test is never failed. No verifier stage, CTest row, workflow step or profile carries a time limit of this repository's; the only hang catcher is the platform's own 360-minute job maximum, and a job stopped there is a HANG to re-run, not a failed test (EN-105 G9 as amended, EN-116). Two checks went red on 2026-10-12 only because a bound was too tight for a slow row: hosted kernel-only run 38213895073 (killed at the old 1800 s stage bound) and main push run 38217844683 (
test_ci_verify: TIMEOUT 3600 must be positive and less than stage deadline 2820).This is the engine half of EN-116. The codegen half is pineforge-codegen-oss#201.
What changed (8 commits on 2dbd72f, tests/scripts/.github/docs only; no
src/,include/,runner/or CMake file)9e9de395fred first:scripts/test_ci_verify.pygains a test that runs the realscripts/check_native_include_independence.pythrough the realDriver.invoke()/default_runnerwith a compiler wrapper that sleeps 122 s and then runs the real compiler. On this commit it fails because the helper's 120 s compile deadline expires (lab jobrj-20261012t173608-233ef9, release profile:99% tests passed, 1 tests failed out of 910,ci_verify release: failed). The commits that follow make it pass.b2b2eb0c8everytimeout=bound on a subprocess a CI test or check runs is removed, with the two local helpers that carried a timeout default (122 bounds in 41 files).379deb02ascripts/ci_verify.py: the options--test-timeoutand--ctest-timeout, their config fields, validators, the row-versus-stage deadline inventory check and the summary fieldstestTimeoutSeconds/ctestTimeoutSecondsare removed; ctest is called with--timeout 0; the native verification step carries no step deadline (the job-level 360 minutes stay and are still required byscripts/ci_preflight.py). Docs reworded so each sentence is true.ad0da67f4,192288399,167860142the native-live Python tests, the C++ test servers and the routing/service rows wait without deadlines instead of failing on elapsed time.83baa36cathe native-live aggregate job printsHANG (re-run)when a native profile job iscancelled(the platform's 360-minute stop) and still exits 1;successexits 0;failure,skippedand an empty result exit 1 without the HANG line.scripts/test_ci_preflight.pyasserts this exactly, in both directions.7c287af52leftovers from the Claude gating read (see Review below): two more subprocess limits inscripts/test_ci_preflight.py, fourEvent.wait()guards that could not fail, three PASS lines that advertised guards that are gone, anddocs/ci.mdwording of the HANG rule.Evidence
ctest --timeout 0means no default per-test deadline, proven by a run. Throwaway lab jobrj-20261012t191150-4bcd9c(kernel profile, tree = this branch's earlier head plus one throwaway row, never pushed): the rowtest_throwaway_sleep_1600sleeps 1600 s with noTIMEOUTproperty andPassed 1600.01 sec;100% tests passed out of 338; the command line shows--timeout 0. 1600 s is above CTest's 1500 s built-in default.7c287af52, lab jobs (absent = private fixtures not present; overlay = private fixtures5abcf8a6overlaid):rj-20261012t194644-3a5e4e: preflight passed (49 stages), Doxygen warnings 0, releasectest ran 604 rows (floor 604), 306 skipped,100% tests passed out of 910, kernelctest ran 307 rows (floor 307), 30 skipped,100% tests passed out of 337;ci_verify releaseandci_verify kernelpassed with 0 failures.5abcf8a6, 10266 files, every blob verified) preflight/release/kernelrj-20261012t194646-6cae41: releasectest ran 910 rows (floor 910),100% tests passed out of 910, runtime budget ratio 3.948x against a 15.000x limit; kernelctest ran 337 rows (floor 337),100% tests passed out of 337; bothci_verifypassed with 0 failures.rj-20261012t194651-63922e: debug100% tests passed out of 891, sanitizers100% tests passed out of 882, bothci_verifypassed with 0 failures.rj-20261012t194653-2bb05a:Verified 312 strategies — excellent=311 ... anomaly=1,TradingView parity holdson all 312 probes (baseline match=312, moved=0) and on the 54-probe subset (match=54, moved=0).9878bfc6(same tree except the 4-file fix commit): absent release 604 of 910 rows (306 skipped), kernel 307 of 337 (30 skipped); overlay release 910/910 and kernel 337/337; debug 891/891 and sanitizers 882/882; parity 312/312 and subset 54/54.Review
A Claude gating read of the engine diff (Sonnet 5.5, raw
permission_denialsempty) found no faked pass, confirmed the red-first test fails for the right reason (a real 122 s compiler against the old 120 s limit) and that the HANG classifier is exact, and listed five mechanical leftovers, which7c287af52fixes; a short Claude re-read of that fix (Sonnet 5.5, rawpermission_denialsempty) returned ACCEPT: R1-R6 implemented exactly, no faked pass, the rebuilt commit chain has the same trees as before (only one commit message changed), andscripts/check_doc_reverts.py --base 2dbd72fb7reports 0 unnamed sentence deletions or reverts across the 8 commits.Squash note. The messages of
379deb02aand83baa36castill say a stop at the platform maximum is "not a failed check"; the final docs, workflow and test say the check fails and the aggregate names the stopHANG (re-run). A squash message must not repeat the old wording, and the docs-revert namings (scripts/check_doc_reverts.py) must be kept: build the squash body from the commit messages and re-run the check on it.Kept on purpose (not time limits of the verifier)
tests/native_live_equivalence_e2e.pywait(120)lines (:532, :690): the file is excluded from the registered rows (scripts/ci_verify.py,scripts/test_ci_verify.py).tests/test_native_live_equivalence_harness.py:404-407withtests/native_live_test_process.py:67-72; slowness cannot fail it, it expectsTimeoutExpired.connect_timeout_ms,total_timeout_ms,--feed-idle-timeout,--feed-message-timeout), the CPU-ratio and runtime-budget measurement gates,benchmarks/run_all.sh,runner/service.cpp.PUBLICATION_RESERVE_SECONDS = 120in the campaign workflow repo (cloudrun/native-proof/run_proof.py), untouched and out of this repository.Known residuals (honest): two bounds remain on REGISTERED required rows
tests/native_live_tick_tape.py:345/347: a 600 s wait for the mid-tape SIGKILL to be proven.genuine_casefeeds the registered rownative_live_genuine_retry_e2e(scenariosrestart-batch,combined-batch, also under ASan and TSan), so a run slower than 600 s can still fail a required row.tests/test_local_time_lock_free_hit.cpp:93: a 5 s read bound on a registered row; a read slower than 5 s fails it.Their redesign is tracked as LVB-69 and they are not changed here. Failures that used to go red in seconds (a broken barrier, a mock-server accept) now wait to the job maximum: an accepted consequence of the owner rule. Cleanup in
tests/native_live_service_e2e.pylost its kill fallback; I could not test macOS.Not done
No
src/,include/or product behavior changes. The merge gate's own verification (lab verify recordwith all lab jobs cited) is the supervisor's step after the push.🤖 Generated with Claude Code