Skip to content

feat(examples): add market data integrity verify loop - #541

Merged
JackChen-me merged 2 commits into
open-multi-agent:mainfrom
green3sf:feat/market-data-integrity-cookbook
Aug 20, 2026
Merged

JackChen-me merged 2 commits into
open-multi-agent:mainfrom
green3sf:feat/market-data-integrity-cookbook

Conversation

@green3sf

Copy link
Copy Markdown
Contributor

Summary

  • add a Binance L2 market-data integrity cookbook recipe built around the per-task verify loop
  • isolate MOCK depth, trade, snapshot, collector, and policy evidence across extractors and source-specific judges
  • seed a provisional ACCEPT that must be revised to an actionable QUARANTINE report
  • register the example in the catalog and examples README, including the updated inventory assertion

Why this is a verify-loop recipe

The proposer receives only locally continuous retained depth events and independent aggregate-trade activity. Snapshot-boundary metadata and collector lifecycle evidence are deliberately withheld for the judges:

  • protocol-continuity-judge checks snapshot-metadata.json
  • backtest-risk-judge checks collector-log.txt plus data-quality-policy.md

The task sets quorum: 2, maxRounds: 2, and onDissent: "revise". It uses judgePrompt: (judge) => string for source isolation and intentionally does not set mode, because the custom judge prompt replaces the built-in mode instruction.

Runtime assertions require the observable path ACCEPT -> QUARANTINE, both judges in round one, both judges accepting round two, the exact 900101-900104 gap, the reconnect time window, source citations, and replacement-data remediation.

All fixtures are synthetic and explicitly marked MOCK. The example makes no exchange request and uses no exchange credentials.

Validation

  • npm run test:example-catalog — 9 tests passed; 63 entries validated
  • strict standalone example typecheck with tsc --noEmit — passed
  • npm run lint -w @open-multi-agent/core — passed
  • npm run build -w @open-multi-agent/core — passed
  • focused consensus suite — 23 tests passed
  • npm run test -w @open-multi-agent/core — 1949 tests passed
  • deterministic fixture validation — retained sequence continuous, reconstructed books uncrossed, snapshot bridge fails, and independent trades occur in the gap window
  • git diff --check — passed

The real-provider example itself was not executed because it requires ANTHROPIC_API_KEY; no contributor or exchange credential was used. CI and the deterministic checks above cover compilation, catalog registration, verify-loop behavior, and fixture consistency.

Closes #540
Context: #147

@JackChen-me this implements the accepted verify-loop direction with quorum: 2, judge-specific source prompts, and no mode override.

@JackChen-me

Copy link
Copy Markdown
Member

assertExpectedPath throws before anything prints, so a wording near-miss burns a full run and leaves the user with one Fatal error line, no report, no audit trail. The filename asserts at :348 / :351 are the likely trip: neither judgePrompt nor the proposer prompt asks for the literal snapshot-metadata.json / collector-log.txt string, and the revision round gets task.description instead of the dependency-augmented prompt (consensus.ts:353), so the proposer only picks those names up if a judge quotes its own header. Ask for the filename in both prompts, and move the asserts after the printing using the PASS/FAIL + process.exit(1) shape from rare-disease-information-triage.ts.

Everything else checks out. Fixtures are internally consistent, and the quorum: 2 / no-mode reasoning matches consensus.ts.

@green3sf

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review. I addressed both failure modes in 0a1fefc:

  • Both source-specific judge prompts now quote the literal fixture filename in dissent and require it in the matching revised invariant; the proposer revision prompt explicitly preserves those filenames in violated_invariants[].source.
  • The runtime checks now run after the decision path, judge audit trail, and full report are printed. Every check emits PASS/FAIL, and the example calls process.exit(1) only after all assertions have been shown.

Validation completed:

  • npm run test:example-catalog (9 tests; 63 entries validated)
  • npm run lint -w @open-multi-agent/core
  • npm run build -w @open-multi-agent/core
  • focused consensus/verify tests (28 passed)
  • full core suite (1949 passed)
  • git diff --check

Thanks again for catching the nondeterministic filename handoff and the hard-to-debug failure path.

@JackChen-me
JackChen-me merged commit d7b4b80 into open-multi-agent:main Aug 20, 2026
7 checks passed
JackChen-me added a commit that referenced this pull request Aug 21, 2026
#546)

The published body was the changelog section and nothing else, so a reader
of a release page could not tell which packages shipped, how to install
them, or who contributed. green3sf landed #536 and #541 in 1.16.1 and the
release page credited nobody.

Three deterministic sections now follow the changelog section:

- `## Packages` names every workspace version and reports the ones this
  release left alone, derived by comparing the release commit's manifests
  against its first parent.
- `## Thanks` credits contributors from outside the project, resolved from
  the commits between the previous release tag and the release commit. Bot
  commits and the maintainer's own are excluded, and the section is omitted
  when nothing remains. Names are plain rather than mentions, because a
  published body that mentions an account notifies it.
- `## Install` is copy-pasteable.

The whole body is still composed before anything reaches the npm registry,
so a malformed changelog or an inconsistent manifest set keeps failing
ahead of publication rather than after it.

The previous tag is resolved from the release commit's first parent rather
than from HEAD. On an idempotent re-run this release's tag already exists,
and describing from HEAD would report an empty range and silently drop
every contributor from the body.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cookbook: Binance L2 market-data integrity verify loop

2 participants