Skip to content

Commit 116c461

Browse files
committed
Document advisory PR CI and required verification statuses
1 parent 57c89ff commit 116c461

5 files changed

Lines changed: 102 additions & 50 deletions

File tree

‎AGENTS.md‎

Lines changed: 20 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,13 @@
66

77
Both C++ unit tests and full corpus verification must pass.
88

9+
The merge ruleset requires `pineforge/verify` and `pineforge/parity` commit
10+
statuses on the PR head's exact tree. The maintainers' `lab verify` tooling
11+
posts them after full verification on their x86-64 build hosts and a parity
12+
verdict. GitHub Actions is advisory: PR Debug, sanitizers and native jobs
13+
exclude the measured `slow` CTest rows; push-to-main and manual CI dispatch
14+
run every row. The commands below run the full sets.
15+
916
```bash
1017
# Fast workflow/source checks first (requires actionlint 1.7.12 + ShellCheck).
1118
# This does not replace either verification step below.
@@ -122,23 +129,17 @@ dispatches, reviews and measures; this section is what binds you here.
122129
engine closer to an independent backtest + forward-execution state
123130
machine, and Pine-parity behaviour kept in codegen with this kernel clean.
124131

125-
## Parity campaign gate (applies on EVERY harness)
126-
127-
Pushes and PRs from this repo are gated by the PineForge parity campaign: a
128-
fresh (≤6h) PASS verdict must bind the exact (engine, codegen) HEADs, recorded
129-
on the campaign registry. Under Claude Code a PreToolUse hook
130-
(`.claude/settings.json`, calls `pineforge-workflow/campaign/hooks/pr-gate.mjs`)
131-
enforces this on `git push` / `gh pr create|ready|merge`. Codex, OpenCode, and
132-
other harnesses run NO hook — the discipline is exactly as binding there: before
133-
any push, run the gate and record the verdict (see the `pr-gate` skill in
134-
`pineforge-workflow/.claude/skills/` — plain markdown, readable anywhere):
135-
136-
```sh
137-
gcloud run jobs execute pineforge-pr-gate --project gen-lang-client-0864094636 \
138-
--region asia-east1 --args '^|^--pipeline|pr_gate|--conf|<conf-json>'
139-
lab gate record --verdict <verdict.json> --engine <sha> --codegen <sha>
140-
```
132+
## Merge gate and parity campaign
133+
134+
The `PineForge strict CI base` ruleset requires `pineforge/verify` (full
135+
`ci_verify.py` profiles on the PR head's exact tree) and `pineforge/parity`
136+
(no parity regression, or no engine behaviour change). The maintainers'
137+
`lab verify` tooling posts both commit statuses; GitHub Actions does not post them.
138+
Its PR jobs provide faster advisory feedback, while push-to-main and manual
139+
dispatch run the full CI profiles. A campaign PASS verdict still binds the
140+
exact engine and codegen HEADs for baseline promotion.
141141

142-
Merged single-axis PRs advance the campaign baseline automatically
143-
(`.github/workflows/promote-baseline.yml`); a squash/rebase that rewrites the
144-
sha defers and must be re-gated.
142+
Merged single-axis PRs advance the campaign baseline automatically through
143+
`.github/workflows/promote-baseline.yml` only when the exact-head guard, both
144+
required statuses on that head, and the campaign verdict pass. A squash or
145+
rebase that rewrites the SHA defers promotion and needs new verification.

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -260,7 +260,7 @@ Every script is exported from TradingView as-is (its own inputs, its own default
260260
- **strong** — ≥ 95% matched, trade count within 6%, entries within 0.1% and exits within 0.5% at p90;
261261
- **moderate / weak** — ≥ 75% coverage, or less.
262262

263-
Published parity results use a fixed population and reproducible Cloud Run measurements. The formal gate requires **no hard-surface regression** and strictly positive pooled movement across the target excellent and excellent+strong bands. A documented native-correctness exception permits exactly zero target-band movement with no individual regression, after full comparison, independent review and CI; its actual FAIL remains recorded and baseline promotion is deferred. Negative movement is outside this exception. Baseline promotion requires a recorded PASS and an exact-head merge with green CI.
263+
Published parity results use a fixed population and reproducible Cloud Run measurements. The formal gate requires **no hard-surface regression** and strictly positive pooled movement across the target excellent and excellent+strong bands. A documented native-correctness exception permits exactly zero target-band movement with no individual regression, after full comparison and independent review; its actual FAIL remains recorded and baseline promotion is deferred. Negative movement is outside this exception. The merge ruleset requires the maintainers' `pineforge/verify` and `pineforge/parity` commit statuses on the exact PR head; GitHub Actions CI is advisory. Baseline promotion also requires a recorded PASS and an exact-head merge.
264264

265265
### What the closed test taught the engine
266266

‎docs/ci.md‎

Lines changed: 76 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Local verification and CI
22

3-
Start with the fast preflight used by GitHub Actions. Install actionlint 1.7.12
3+
Start with the preflight used by GitHub Actions. Install actionlint 1.7.12
44
and ShellCheck, then run:
55

66
```sh
@@ -9,7 +9,8 @@ python3 scripts/ci_preflight.py
99

1010
This checks the CI/native workflow syntax, expressions and shell commands,
1111
source ABI/hash/schema guards, and the verifier's failure-handling tests. A
12-
failure blocks all compilation lanes and retains logs in `build-ci-preflight/`.
12+
failure fails the advisory aggregate `build` job and retains logs in
13+
`build-ci-preflight/`. The compilation lanes start alongside preflight.
1314
It does not compile the engine or replace any complete verification profile.
1415

1516
Then use the same verification entrypoint as GitHub Actions before publishing:
@@ -93,6 +94,17 @@ release rows. Raise the constant when a row lands, and pass `--min-tests N` to
9394
override it for one run (the flag gates any profile; `kernel` and `release`
9495
have a default).
9596

97+
For PR runs with `--exclude-label slow`, the verifier asks CTest to enumerate
98+
the registered rows both with `ctest -N` and with `ctest -N -LE slow`. It then
99+
requires the rows that actually ran to equal the second count, and records
100+
`registered`, `labelled = registered - selected`, and `ran` in
101+
`ctest-exclusion.log` and `ci-summary.json`. A skip, missing executable, lost
102+
registration below the PR registration floor, unreadable enumeration, or
103+
missing label fails. The PR registration floors at the INT24 base are 653
104+
for Debug and sanitizers and 662 for native. The existing full-run release and
105+
kernel floors remain 672 and 271 rows that ran; full runs do not exclude a
106+
label.
107+
96108
Preflight also runs the detached-comment census of the kernel compile closure
97109
(`detached-comments`: `scripts/measure_detached_comments.py --check-ceiling`)
98110
against the checked-in `DETACHED_LINE_CEILING`, which is 0: R5 lane B-DOCS
@@ -316,18 +328,19 @@ probe, machine load average 100–160 from sibling builds). Judging the result
316328
and the dominant term is single-core serial work that a 4-vCPU hosted runner
317329
runs slower, not faster.
318330

319-
So the whole population cannot be held to the ~25 min a required pull-request
320-
check is budgeted for, and it runs nightly (01:30 UTC), on `workflow_dispatch`,
331+
So the whole population cannot be held to the ~25 min pull-request CI target,
332+
and it runs nightly (01:30 UTC), on `workflow_dispatch`,
321333
and on pull requests that touch the corpus pin or the parity tooling. ccache
322334
(2 GB, keyed per commit with a prefix restore) makes a re-run at an unchanged
323335
pin almost entirely cache hits; the Git-LFS chart feed (~176 MB, one
324336
full-history 1m CSV) is pulled once per job.
325337

326-
### The subset that does block a merge
338+
### The advisory pull-request subset
327339

328-
A nightly gate cannot stop a merge: a `src/**` change that moves a TradingView
329-
trade merges green and is caught the next night at the earliest. The blocking
330-
half is a named population, not a weaker oracle:
340+
The GitHub Actions subset gives early feedback on a `src/**` change that moves
341+
a TradingView trade. The maintainers' separate parity verdict posts the
342+
required `pineforge/parity` status on the exact PR head. The subset is a named
343+
population using the same byte oracle:
331344

332345
```sh
333346
./scripts/check_corpus_parity.sh --subset
@@ -365,19 +378,18 @@ judge <1 s — **94 s** end to end over an up-to-date build directory, against
365378
1792 s for the run phase alone in full mode.
366379

367380
`corpus-parity.yml` exposes it as a reusable workflow (`mode: subset`) and
368-
`ci.yml` calls it on every CI run and lists it in `build-gate`'s `needs`. The
369-
"PineForge strict CI base" ruleset requires the contexts `build` and
370-
`sanitizers`; `build` **is** `build-gate`, so the parity subset blocks a merge
371-
through the context that is already required, with no ruleset change. A
372-
skipped dependency is not a success, so the job is deliberately unfiltered by
373-
path.
381+
`ci.yml` calls it on every CI run and lists it in `build-gate`'s `needs`. A
382+
skipped dependency fails that advisory aggregate job, so the subset remains
383+
unfiltered by path. The merge ruleset requires `pineforge/verify` and
384+
`pineforge/parity`, posted by the maintainers' `lab verify` tooling after
385+
full verification on the PR head's exact tree.
374386

375387
What the subset does not prove, and what therefore stays with the nightly
376388
sweep: the other 282 probes, and the tier headline —
377389
`scripts/verify_corpus.py` grades the whole population, so 30 re-runs cannot
378390
print its line, and grading the 282 untouched tapes beside them would judge the
379-
corpus's own older generation rather than this engine. The subset is a blocking
380-
floor, not a replacement.
391+
corpus's own older generation rather than this engine. The subset is an early
392+
signal alongside the full parity verdict.
381393

382394
The subset job checks the submodule out with `GIT_LFS_SKIP_SMUDGE=1` and
383395
restores `corpus/data` from a cache keyed by the gitlink, so the ~176 MB feed
@@ -504,14 +516,51 @@ identity are retained even if preparation fails before the verifier starts.
504516
Compiler objects, binaries and dependency caches are not uploaded
505517
as diagnostics.
506518

507-
Superseded pull-request runs are canceled. CI/native main/post-merge and manual proof runs
508-
use distinct concurrency groups and remain uncanceled. Native verification is a
509-
reusable workflow called once by CI, with a separate concurrency namespace; it
510-
also supports manual dispatch. The required `build` check passes only when
511-
preflight, all four standard builds, sanitizers, native verification and the
512-
TradingView parity subset succeed. A failed, canceled or skipped dependency
513-
cannot produce a green `build` check.
514-
The separate required `sanitizers` status remains available.
515-
516-
These checks do not run the parity campaign. Fixed-population Cloud measurement,
517-
the actual gate and post-merge evidence remain separate acceptance steps.
519+
Superseded pull-request runs are canceled. Main/post-merge and manual CI runs
520+
use distinct concurrency groups and remain uncanceled. A PR runs preflight,
521+
both Release jobs, kernel-only and the parity subset with their full sets;
522+
both Debug jobs, sanitizers and native-live run the registered set excluding
523+
the 27 CTest rows labelled `slow` in `tests/CMakeLists.txt`. These rows were
524+
chosen from INT23/INT24 job logs: over 60 seconds in either sanitizer run or
525+
over 30 seconds in either Debug run. Preflight and all proof jobs start in
526+
parallel. The advisory `build` aggregate succeeds only if every job succeeds.
527+
528+
A push to `main` and a manual dispatch of `ci.yml` run every profile without
529+
the exclusion. Standalone manual dispatch of `native-live.yml` is also full.
530+
The maintainers run the full `ci_verify.py` profiles on their x86-64 build
531+
hosts before the PR merge gate and post `pineforge/verify` to the PR head.
532+
Their parity verdict posts `pineforge/parity`, reporting no regression or that
533+
the PR changed no engine behaviour. The `PineForge strict CI base` ruleset
534+
requires these two commit statuses. GitHub Actions jobs, including `build`,
535+
`sanitizers`, docs and the parity subset, are advisory. Baseline promotion
536+
requires both statuses to be successful on the exact merged PR head, as well
537+
as its existing exact-head and campaign verdict guards.
538+
539+
The full corpus sweep remains a separate acceptance step. The nightly and
540+
manual corpus workflow, and the maintainers' full parity verification, keep
541+
the broader population running.
542+
543+
### CI-LITE timing estimate
544+
545+
The INT24 PR run `36105656107` measured 9.1 min preflight, 30.2/27.8 min
546+
macOS/Ubuntu Debug, 25.1 min native, 20.5/16.5 min Ubuntu/macOS Release,
547+
15.8 min kernel, and 3.8 min parity subset. Its sanitizer job ran 61.5 min
548+
and hit the verifier's 30 min CTest timeout. The complete INT23 sanitizer
549+
job `107895095058` measured 42.1 min: 20.0 min CTest and 22.1 min outside
550+
CTest, including a 21.1 min build. The 27 labelled rows consumed 34.8 and
551+
42.3 min of summed test time in the INT24 Ubuntu and macOS Debug jobs. The
552+
26 rows then present in INT23 sanitizers consumed 57.5 of its 76.7 min of
553+
summed test time; the 27th row landed by INT24.
554+
555+
At four concurrent CTest jobs, the remaining rows have scheduling lower
556+
bounds of 1.1 min Ubuntu Debug, 1.6 min macOS Debug, 0.5 min native, and
557+
4.8 min INT23 sanitizers. Adding observed non-CTest time and a small CTest
558+
scheduling allowance estimates about 19 min Ubuntu Debug, 20 min macOS
559+
Debug, 16 min native, and 28 min sanitizer on the complete INT23 run. Release
560+
and kernel keep their measured full durations. With preflight running in
561+
parallel, the modeled PR wall time is about 28 min on INT23 conditions. The
562+
INT24 sanitizer build-to-CTest interval alone was 30.3 min, so its conditions
563+
imply a wall time above 35 min even after excluding the rows. The 25 min
564+
target needs a measured sanitizer build improvement; the current job logs do
565+
not separate compile from link time or report a ccache hit rate, so CI-LITE
566+
does not assume a build optimization without evidence.

0 commit comments

Comments
 (0)