Skip to content

Catalog freshness: see prerelease releases of a pinned prerelease stream (Nautilus 2.0.0rcN; trading PR-6) - #707

Merged
seathatflowsinourveins merged 21 commits into
mainfrom
shared/nautilus-prerelease-freshness-20261004
Oct 5, 2026
Merged

seathatflowsinourveins merged 21 commits into
mainfrom
shared/nautilus-prerelease-freshness-20261004

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator

Scope

A published NautilusTrader 2.x prerelease can be absent from releases/latest, leaving a v2.0.0rc5 pin without a drift report when rc6 is published. This change adds bounded release-list reads for prerelease pins and applies the same selection and comparison policy to the workflow summary, manifest, and trading report.

  • Base commit: 3b8f9c8a1b938358d65bf422f61592dc3b89407d
  • Lane: lane:shared; the other lane's acknowledgement is required before merge.
  • Owned paths touched: .github/workflows/catalog-freshness.yml, tools/sota-convergence/github_freshness.py, tools/sota-convergence/build_manifest.py, tests/test_catalog_freshness_pins.py, tests/test_catalog_freshness_trading.py, docs/decisions/2026-10-04-nautilus-prerelease-freshness.md, and manifests/evidence.json.

Prerelease pins select the newest published, non-draft release in the pinned major, including prereleases and final releases. A later stable 1.x backport does not hide a 2.x RC. Stable pins retain their existing releases/latest behavior. Reads are capped at three pages of 100 records; a full third page reports unknown beyond cap, even when a candidate appears within the window.

SOTA sources

Evidence-class table

Claim Evidence class Command / receipt
rc6 produces drift for an rc5 pin despite a later 1.x backport synthetic Freshness pins and trading fixtures
Drafts are ignored and cap exhaustion produces an explicit unknown synthetic Freshness pins and trading fixtures
Workflow summary, fetch, manifest, and report share the release policy local_integration Four freshness modules: 238 tests passed
REST endpoint semantics and prerelease ordering have cited references source_review GitHub documentation and packaging 26.0 source above
Final artifacts and registered hashes are consistent local_integration validate.py and evidence_manifest.py --check

Local commands run

All resumed checks used rtk nice -n 19 with TMPDIR="$PWD/.tmp-build/.pytest_cache/freshness".

python3 -m unittest tests.test_catalog_freshness_pins tests.test_catalog_freshness_trading tests.test_catalog_freshness_runtime tests.test_catalog_freshness_propose -v
exit 0; 238 tests passed in 66.887 seconds

python3 scripts/validate.py
exit 0; 9961 hashed files; status passed

python3 scripts/evidence_manifest.py --check
exit 0; 9961 files; status passed

actionlint .github/workflows/catalog-freshness.yml
nv; unavailable on PATH

git diff --check
exit 0

scripts.host_receipts.register_file(root, path)
exit 0; four covered files registered last; manifest bytes unchanged

Historical full-suite command python3 -m unittest -v --durations 50 exited 1 with ENOSPC before completion and workspace-TMPDIR-sensitive fixture failures. It was not rerun under the latest instruction, so full-suite acceptance remains unqualified. Earlier interrupted or failed attempts are retained in the decision record.

Decision record

docs/decisions/2026-10-04-nautilus-prerelease-freshness.md records the endpoint choice, pagination bound, alternatives, regression evidence, completeness review, and conditions for revisiting the cap or supported version grammar.

Host evidence

Not applicable: no files under evidence/hosts/ changed. These checks establish local integration and artifact consistency. Live upstream rc6 observation and hosted CI remain unperformed.

Checklist

  • Workflow actions retain full commit SHA pins and version comments.
  • Workflow retains top-level permissions: {} and job-specific grants.
  • No secrets were printed, logged, or committed; no required secret was added.
  • No paid hosting, subscription, or billing surface was introduced.
  • Peer-owned files and worktrees were preserved.

No commit or push was performed; the coordinator owns integration.

Review chain

  • Claude Opus first read: FINDINGS (1 p1, 2 p2).
  • GPT-6.1 Sol repair round through the packaged SDK worker; the coordinator committed it.
  • Claude Opus delta read of the repair: ACCEPT at d3f068a0593a (0 p1, 0 p2).

🤖 Generated with Claude Code

Scout and others added 6 commits October 4, 2026 20:35
…eam (Nautilus 2.0.0rcN)

github_freshness.py and catalog-freshness.yml read a bounded, paginated release list beside releases/latest and,
for a pin in a prerelease stream, report the newest published, non-draft release of the pinned major stream,
prereleases included; reaching the page cap reports unknown, never a silent miss. A stable 1.x backport created
after an rc no longer hides it. Fixtures cover rc6 after rc5, a later stable backport, drafts and the cap.
GPT-6.1 Sol build freshness-1 through the packaged SDK worker; the coordinator commits.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…fore the final hot-file commit

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… (hot-file protocol: every hot-file edit in the last commit)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…it unknown/no-release states (Opus p1 + p2s)

GPT-6.1 Sol repair round freshness-r1 (answers the Claude Opus batch-7 read) through the packaged SDK worker; the
coordinator commits.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…fore the final hot-file commit

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… (hot-file protocol: every hot-file edit in the last commit)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@seathatflowsinourveins seathatflowsinourveins added the lane:shared Touches files owned by both lanes; needs both lanes' acknowledgement label Oct 5, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-05T02:40:29.214247Z d3f068a PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d3f068a059

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Scout and others added 3 commits October 4, 2026 23:23
…egistry plus the owned rows)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…fore the final hot-file commit

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… (hot-file protocol: every hot-file edit in the last commit)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@socket-security

socket-security Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Dependency limit exceeded — report not shown.

This pull request scan exceeded the 10,000-dependency limit applied to this scan, so the results are incomplete and may be inaccurate. To avoid reporting false positives, Socket has not posted a report.

Upgrade your plan to raise the dependency limit and get complete reports, or view the partial scan in the dashboard.

Socket is always free for open source. If this is a non-commercial open source project, contact us to request a free Team account.

Scout and others added 9 commits October 5, 2026 00:56
…anking, dormancy, fetch scope, aliases, anti-pattern log)

- github_freshness.py ranks release candidates by parsed version first, with publication
  time only as a tie-breaker, so a republished rc5 can no longer hide rc7 from an rc6 pin.
- Runtime entries that declare tags no longer require a release-list fetch. A slug needs the
  list only when one of its entries actually uses the release-stream path, so a failed
  unneeded fetch cannot suppress valid drift.
- build_manifest.py counts the selected release-list publication as dormancy activity, so a
  row cannot show a fresh RC and dormant: true together.
- A refetch removes every retained record with the same normalized slug, aliases included,
  before storing the new one. Unrelated repositories are preserved.
- docs/harness-defaults.md's anti-pattern log gains rows for the proven mistakes:
  import-time execution, uncertainty reporting and runtime-tag comparison. Each row has its
  prevention and verification path.

Each fix has a regression test that was red before it. The decision record follows the
behaviour.

Built by GPT-6.1 Sol through the packaged SDK worker (round freshness-t1); committed by the
coordinator.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…fore the final hot-file commit

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… (hot-file protocol: every hot-file edit in the last commit)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ues, row checks, documented dormancy, fail-closed tag misses)

- The alias cleanup guards dictionary access and normalizes aliases with github_slug.
  Unrelated null, string and list records survive refetch and resume, which fixes the t1
  regression. Mixed-case and .git release URLs resolve to the fresh record.
- The anti-pattern log regression now covers all seven prior repair rows and the three new
  rows. It checks uniqueness, non-empty fields, and every cited method and its owning class
  by AST, with twelve negative controls. The decision record's coverage claim is corrected.
- Dormancy keeps its per-row channel semantics, documented: stable-pinned and tag-declared
  rows ignore RC activity fetched for another row. The rendered explanations say what
  activity each row considered.
- For a tag-declared prerelease pin, a missing or unmatched tag list now publishes
  not_compared, with pin_behind_upstream null and an explicit tag_pattern_unfetched or
  tag_pattern_unmatched reason, instead of comparing against another package's
  releases/latest. The behaviour is fail-closed and covered by regressions.

Built by GPT-6.1 Sol through the packaged SDK worker (round freshness-t2); committed by the
coordinator.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…fore the final hot-file commit

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… (hot-file protocol: every hot-file edit in the last commit)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ing; anti-pattern controls reach the AST checks; stable-pin fallback covered

- tests/test_catalog_freshness_trading.py:587 asserts the reworded dormant summary sentence. It
  is shared through the module, so wording and test cannot drift apart again. The trading report
  module joins the record's targeted acceptance.
- The anti-pattern verification gains source-side negative controls with the log unchanged: a
  renamed cited method, and a cited method moved to another class. Each control asserts the
  exact failing subtest and message, so deleting the class or method ownership check now fails a
  control.
- A tag-declared stable pin with a tag miss keeps its documented fallback comparison against
  releases/latest, and a regression now covers it.

Built by GPT-6.1 Sol through the packaged SDK worker (round freshness-t3); committed by the
coordinator. The coordinator re-ran the trading, runtime, propose and sota_convergence suites
(exit 0) and validate (exit 0).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…fore the final hot-file commit

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… (hot-file protocol: every hot-file edit in the last commit)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

Trading-lane acknowledgement (lane:shared, docs/lanes.md) at 66e49151402ae07115bb63d4c64cac10a43ab67f.

The trading side of this change is acknowledged:

  • the trading freshness rows;
  • the per-row dormancy wording in the trading drift report;
  • prerelease stream handling for the NautilusTrader rc pins.

The basis is the cross-family review chain. The Claude Opus delta reads t1, t2 and t3 cover every round through this head. t3 returned ACCEPT with no P1 or P2, and confirmed that the t2 trading report assertion (tests/test_catalog_freshness_trading.py) follows the new wording. The coordinator's runs of the trading, runtime, propose and sota_convergence suites and of scripts/validate.py exited 0 at this head. All five review threads are answered and resolved.

The one CI failure at this head is tests.test_adaptive_paper_metrics FileSdRegistrationTests, which hit its 30-second exporter registration deadline. That module is outside this change, and the failed job has been re-run.

🤖 Generated with Claude Code

Scout and others added 3 commits October 5, 2026 03:50
…egistry plus the owned rows)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…fore the final hot-file commit

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… (hot-file protocol: every hot-file edit in the last commit)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

Claude session native-agent-stack-5f: landing at head d220de9318f9ec25ef1cee2989a9a344ae57845d. The command center (wsl-architecture-design) gave its ACK at 66e49151402a, which reaches this head only through recorded mechanical refreshes whose owned non-registry patch-ids are equal (its carry rule); under the user's 2026-10-04 decision it owns merges that touch the hot registry. The read-only cross-family review (GPT-6.1 Sol max, the packaged Codex SDK worker) returned ACCEPT at d3f068a0593a; this head is reached from it through recorded carry edges (equal owned patch-ids, or a cross-family delta read returning ACCEPT at the edge's target), with no P1 item.

Observed main cab2566fae0093f4c1ac9330158d7bbbf1d17299. coordination/merge_tree_landing_check.py (sha256 a11649fcf2e5844c…) exit 0:

main cab2566fae0093f4c1ac9330158d7bbbf1d17299 head d220de9318f9ec25ef1cee2989a9a344ae57845d base 4af7417b7d4db8936c0136235a86717f31d16f27 merged-tree e147caa3359f0dafc8622456498e19ee8313fbb0 merge-tree-exit 0
ok   1: clean three-way merge
ok   2: merged-vs-main paths 13, outside PR-owned 0 []
ok   3: main drift 8 paths, overlap with PR-owned inputs (registry excluded) []
ok   4: registry foreign rows equal True, order preserved True, PR-owned rows 12, unowned top-level keys differing []
ok   5: merged files[] sorted by path with no duplicates: True (10098 rows)
LANDABLE
condition 5: merged files[] sorted and unique (tree e147caa3359f0dafc8622456498e19ee8313fbb0)

Required checks at this head: 7 pass . Unresolved review threads: 0.

@seathatflowsinourveins
seathatflowsinourveins merged commit f3ebf46 into main Oct 5, 2026
25 checks passed
@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

Claude session native-agent-stack-5f: post-merge observation. Landed as f3ebf469b49c3e8a41efd28d3d30d53b2edbbac1 (parent cab2566fae0093f4c1ac9330158d7bbbf1d17299), tree e147caa3359f0dafc8622456498e19ee8313fbb0. The landed tree equals the checked merged tree and the parent is the observed main.

@seathatflowsinourveins
seathatflowsinourveins deleted the shared/nautilus-prerelease-freshness-20261004 branch October 5, 2026 08:28
seathatflowsinourveins pushed a commit that referenced this pull request Oct 5, 2026
… its own readiness message, not by a set difference

The FileSdRegistrationTests helper start() treated a reused address as missing registration. An unfinished trial keeps its target registered after a clean stop. When a later exporter in the same test binds the same address, update_file_sd replaces that address's group, so set(listed()) - before stays empty and the helper waited out its 30-second deadline. This is the recurring CI failure "exporter never registered its target" (tests/test_adaptive_paper_metrics.py:467), seen on #707 and #725 today.

Root cause, confirmed by measurement:
- CPython's HTTPServer sets allow_reuse_address, and TCPServer.server_bind sets SO_REUSEADDR.
- 1,000 sequential --port 0 exporters reused a retained address 378 times (37.8%). Every one registered, in 55.6 ms median and 79.3 ms maximum.

The fix:
- start() reads this child's own readiness message (its listening address and file_sd path) from non-blocking stderr, and confirms that the announced address is listed.
- It keeps the 30-second bound and fails at once if the child exits.

Regressions:
- Forced sequential port reuse: failed after 30.2 s with the old helper; passes now.
- A never-registering exporter (--no-file-sd on a reused address) is rejected within its bound.
- 20 consecutive runs of FileSdRegistrationTests: 380 tests, all passing.

Built by GPT-6.1 Sol through the packaged SDK worker (round filesd-flake-r1); committed by the coordinator. The builder's runs each exited 0: tests.test_adaptive_paper_metrics (42 tests), the 20-run loop, validate, evidence_manifest --check and git diff --check.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins pushed a commit that referenced this pull request Oct 5, 2026
…md gained #707's rows) and cite only what publication-validation.json records (gate read 730c b, c)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:shared Touches files owned by both lanes; needs both lanes' acknowledgement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant