Repository navigation
Grade corpus trade counts by gross unmatched trades - #827
luisleo526 wants to merge 1 commit into
Conversation
Keep relative count differences net, explain unmatched trades on both sides, and align the corpus grade expectation with the recorded tape regrade. Co-Authored-By: Claude Opus 5.5 <[email protected]>
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; formal verification proved 0 of 2 changed function(s) (1 not verified).
Formal verification. 1 change(s) alter behavior, breaking input(s) attached. PR-changed functions: 1/2 verified (0 proven, 0 may-equivalent, 1 distinguished) · 1 not verified (1 vacuous).
Behavior changes: \_emit changes behavior, here is the input that shows it.
The verifier found a concrete input on which \_emit behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.
Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.
Evidence: On input \{"rows":"\[\]","engine\_sha":"'a'","corpus\_sha":"'a'","blob\_url":"'a'"\}, the old code produced "\# PineForge Corpus Validation Report\\n\\n\*\*Generated:\*\* 2026\-11\-11 \(UTC\) — engine a, corpus a\\n\\nAll probes live under corpus/validation/; categories below are derived\\nfrom each slug's… but the new code produces "\# PineForge Corpus Validation Report\\n\\n\*\*Generated:\*\* 2026\-11\-11 \(UTC\) — engine a, corpus a\\n\\nAll probes live under corpus/validation/; categories below are derived\\nfrom each slug's…. Paste that input straight into a regression test.
Not verified on this run: analyze\_strategy (vacuous: never exercised).
Graphify review — findings
Changes the G1 count rule to count unmatched trades on both sides instead of netting them, which regrades nine corpus strategies from excellent to strong. check_corpus_parity.sh now pins the expected headline at excellent=302, strong=9, anomaly=1 (previously 311/0/1), and the comment records the regrade of all 312 committed tapes, with zero exceptions. Adds a g1_recorded_count_cases.json fixture of archived TV/engine tapes, pinned by archive SHA-256 with their recorded tier and metrics, so the count rule is exercised against real recorded data. Adds tests for the surrounding grading behaviour: reading TradingView's Return % column (including an explicit zero), declared-tier overrides keeping normal-path precedence, and entry-identity checks that stop fragment consolidation from hiding a merged entry.
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 206 functions depend on the 172 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
analyze_strategy()— 16 callers, 23 callees
Verification — 206 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 206 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
1417 of 1417 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety (
noncode-change,uncovered-code) — the selection is not trustworthy on its own (see below). CI should run the whole suite.
benchmarks/tests/test_aggregate.py— full-run-safetybenchmarks/tests/test_check_provenance.py— full-run-safetybenchmarks/tests/test_compare.py— full-run-safetybenchmarks/tests/test_public_names.py— full-run-safetybenchmarks/tests/test_run_all.py— full-run-safetybenchmarks/tests/test_run_strategy_feeds.py— full-run-safetydocker/pineforge_sink_finalize_test.py— full-run-safetydocker/pineforge_sink_replay_rq_test.py— full-run-safetydocker/pineforge_sink_replay_test.py— full-run-safetydocker/pineforge_sink_to_json_test.py— full-run-safetydocker/quota_columns_test.py— full-run-safetydocker/quota_convert_test.py— full-run-safetydocker/quota_finalize_test.py— full-run-safetydocker/replay_rq_engine_test.py— full-run-safetydocker/run_json_artifact_window_test.py— full-run-safetydocker/run_json_bin_test.py— full-run-safetydocker/run_json_bounded_transport_test.py— full-run-safetydocker/run_json_bulk_equity_fuzz_test.py— full-run-safetydocker/run_json_bulk_equity_test.py— full-run-safetydocker/run_json_codes_test.py— full-run-safetydocker/run_json_converter_metadata_test.py— full-run-safetydocker/run_json_converter_session_test.py— full-run-safetydocker/run_json_diagnostics_test.py— full-run-safetydocker/run_json_equity_scan_test.py— full-run-safetydocker/run_json_fused_test.py— full-run-safetydocker/run_json_harnesses_test.py— full-run-safetydocker/run_json_metrics_only_test.py— full-run-safetydocker/run_json_misc2_test.py— full-run-safetydocker/run_json_misc_test.py— full-run-safetydocker/run_json_pfbar_test.py— full-run-safetydocker/run_json_provenance_test.py— full-run-safetydocker/run_json_quota_supervisor_test.py— full-run-safetydocker/run_json_replay_test.py— full-run-safetydocker/run_json_selected_sink_test.py— full-run-safetydocker/run_json_sink_lexical_test.py— full-run-safetydocker/run_json_sink_summary_test.py— full-run-safetydocker/run_json_sink_test.py— full-run-safetydocker/run_json_sink_vectors_test.py— full-run-safetydocker/run_json_spool_work_test.py— full-run-safetydocker/run_json_stringio_test.py— full-run-safetydocker/run_json_symbol_feeds_test.py— full-run-safetydocker/run_json_syminfo_test.py— full-run-safetydocker/run_json_trade_scan_test.py— full-run-safetydocker/run_json_window_budget_test.py— full-run-safetydocker/run_json_wrapper_work_test.py— full-run-safetyinclude/pineforge/native_run_spec.hpp— full-run-safetyscripts/derive_corpus_feeds_self_test.py— full-run-safetyscripts/fingerprint_self_test.py— full-run-safetyscripts/run_corpus_self_test.py— full-run-safetyscripts/test_abi_receipt_skips.py— full-run-safety- … and 1367 more
non-code file(s) changed (
scripts/fixtures/g1_recorded_count_cases.json) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
scripts/check_corpus_parity.sh) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Formal verification
Behavior changes: \_emit changes behavior, here is the input that shows it.
The verifier found a concrete input on which \_emit behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.
Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.
Evidence: On input \{"rows":"\[\]","engine\_sha":"'a'","corpus\_sha":"'a'","blob\_url":"'a'"\}, the old code produced "\# PineForge Corpus Validation Report\\n\\n\*\*Generated:\*\* 2026\-11\-11 \(UTC\) — engine a, corpus a\\n\\nAll probes live under corpus/validation/; categories below are derived\\nfrom each slug's… but the new code produces "\# PineForge Corpus Validation Report\\n\\n\*\*Generated:\*\* 2026\-11\-11 \(UTC\) — engine a, corpus a\\n\\nAll probes live under corpus/validation/; categories below are derived\\nfrom each slug's…. Paste that input straight into a regression test.
Could not verify: Could not verify analyze\_strategy.
The verifier did not have enough to check analyze\_strategy, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 9 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly NameError — names the real obstacle, not a sampling gap)
· 1 more finding(s) on lines outside this diff (see the check run).
|
Closed: this change will be resubmitted with a corrected test fixture. |
Grade the corpus trade-count test by gross unmatched trades. A probe is excellent only when no trade is unmatched on either side, so a trade the engine misses and a trade TradingView does not have no longer cancel each other out. The relative count difference stays net.
scripts/verify_corpus.py: the count rule uses gross unmatched trades, and the reason note names how many TradingView and engine trades are unmatched.scripts/regen_validation_report.py: the Reason column prints that note.scripts/check_corpus_parity.sh: the corpus expectation follows the recorded-tape re-grade (302 excellent, 9 strong, 1 anomaly).scripts/test_verify_corpus_count_rule.py,scripts/test_verify_corpus_metrics.py, plus the recorded fixturescripts/fixtures/g1_recorded_count_cases.json. They fail on the previous rule and pass on this one, including two recorded probes whose net count was 0 with 2 unmatched trades.Stored grades stay net until the baseline is re-graded under this rule. One follow-up is not part of this change: a stale probe note in the corpus repository.
🤖 Generated with Claude Code