feat(proto): unified OpenAPI bundle via sebuf v0.11.0#3341
Conversation
Adds a third protoc-gen-openapiv3 invocation that merges every service into a single docs/api/worldmonitor.openapi.yaml spanning all 68 RPCs, using the new bundle support shipped in sebuf 0.11.0 (SebastienMelki/sebuf#158). Per-service YAML/JSON files are untouched and continue to back the Mintlify docs in docs/docs.json. The bundle runs with strategy: all and bundle_only=true so only the aggregate file is emitted, avoiding duplicate-output conflicts with the existing per-service generator. Requires protoc-gen-openapiv3 >= v0.11.0 locally: go install github.com/SebastienMelki/sebuf/cmd/[email protected] Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
- Makefile: SEBUF_VERSION v0.7.0 → v0.11.0 (required for bundle support). - proto/buf.gen.yaml: point bundle_server at https://api.worldmonitor.app. - CONTRIBUTING.md: new "OpenAPI Output" section covering per-service specs vs the unified worldmonitor.openapi.yaml bundle, plus a note that all three sebuf plugins must be installed from the pinned version. - AGENTS.md: clarify that `make generate` also produces the unified spec and requires sebuf v0.11.0. - CHANGELOG.md: Unreleased entry announcing the bundle and version bump. Also regenerates the bundle with the updated server URL. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Mechanical output of the bumped protoc-gen-ts-client and protoc-gen-ts-server. Two behavioral improvements roll in from sebuf: - Proto enum fields now use the proper `*_UNSPECIFIED` sentinel in default-value checks instead of the empty string, so generated query-string serializers correctly omit enum params only when they actually equal the proto default. - `repeated string` query params now serialize via `forEach(v => params.append(...))` instead of being coerced through `String(req.field)`, matching the existing `parseStringArray()` contract on the server side. All files also drop the `// @ts-nocheck` header that earlier sebuf versions emitted — 0.11.0 output type-checks cleanly under our tsconfig. No hand edits. Reproduce with `make install-plugins && make generate`. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Greptile SummaryThis PR bumps sebuf to v0.11.1, adds a unified Confidence Score: 5/5Safe to merge — only remaining finding is a trivial v0.11.0→v0.11.1 documentation nit in AGENTS.md All P0/P1 concerns from prior rounds are resolved: repeated-string params are now correctly typed as arrays in both the bundle and per-service specs, CONTRIBUTING.md and CHANGELOG.md version references are accurate, and the TS codegen removes @ts-nocheck cleanly. The sole remaining finding (AGENTS.md still says v0.11.0) is a one-line doc fix with no behavioral impact. AGENTS.md line 60 — minor version string nit Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[proto/buf.gen.yaml] --> B[protoc-gen-ts-client\nsrc/generated/client/]
A --> C[protoc-gen-ts-server\nsrc/generated/server/]
A --> D[protoc-gen-openapiv3\nper-service .openapi.yaml]
A --> E[protoc-gen-openapiv3\nper-service .openapi.json]
A --> F[protoc-gen-openapiv3\nbundle=true / strategy: all\nworldmonitor.openapi.yaml NEW]
D --> G[Mintlify docs\ndocs/docs.json]
E --> G
F --> H[External consumers\nAPI explorers / SDK generators]
B --> I[TypeScript clients\nno @ts-nocheck]
C --> I
Reviews (3): Last reviewed commit: "Apply suggestions from code review" | Re-trigger Greptile |
|
@koala73 heads-up on Greptile's P2 finding re: It's a pre-existing sebuf codegen gap (same issue in the per-service specs), not a regression from this PR. The TS client/server codegen is already correct in v0.11.0 — only the OpenAPI schema emitter is out of sync. Will land the fix in a sebuf patch release and regenerate; no blocker for this merge. Separately, CI has one failing test ( |
…-param wire format - Bump SEBUF_VERSION to v0.11.1, pulling in the OpenAPI fix for repeated scalar query params (SebastienMelki/sebuf#161). `repeated string` fields now emit `type: array` + `items.type: string` + `style: form` + `explode: true` instead of `type: string`, so SDK generators consuming the unified bundle produce correct array clients. - Regenerate all 12 OpenAPI specs (unified bundle + Aviation, Economic, Infrastructure, Market, Trade per-service). TS client/server codegen is byte-identical to v0.11.0 — only the OpenAPI emitter was out of sync. - Update three tests that asserted the pre-v0.11 comma-joined wire format (`symbols=AAPL,MSFT`) to match the current repeated-param form (`symbols=AAPL&symbols=MSFT`) produced by `params.append(...)`: - tests/market-service-symbol-casing.test.mjs (2 cases: getAll) - tests/stock-analysis-history.test.mts - tests/stock-backtest.test.mts Locally: test:data 6619/6619 pass, typecheck clean, lint exit 0. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
|
@koala73 ready for another look. Pushed sebuf v0.11.0 → v0.11.1 (fixes SebastienMelki/sebuf#161):
Fixed the failing CI test ( End-to-end verified the new wire format against the generated server route in-process:
In-repo pipeline is safe — all internal callers go through the v0.11 TS client, which matches the v0.11 server. Grepped External risk: any non-repo consumer (MCP, Telegram bot, mobile, old SDK generations) that still sends Local checks: cc @greptile-apps for a re-review on the repeated-param schema fix. |
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
|
Let me know if all is well please @greptile-apps |
… proto change The shipping/v2 webhook alert_threshold field was flipped from `int32` to `optional int32` with an expanded doc comment in f333946. That comment now surfaces in the unified docs/api/worldmonitor.openapi.yaml bundle (introduced by #3341). Regenerated with sebuf v0.11.1 to pick it up. No behaviour change — bundle-only documentation drift. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…ve (#3343) * feat(agent-readiness): RFC 9727 API catalog + native openapi.yaml serve Closes #3309, part of epic #3306. - Serves the sebuf-bundled OpenAPI spec natively at https://www.worldmonitor.app/openapi.yaml with correct application/yaml content-type (no Mintlify proxy hop). Build-time copy from docs/api/worldmonitor.openapi.yaml. - Publishes RFC 9727 API catalog at /.well-known/api-catalog with service-desc pointing at the native URL, status rel pointing at /api/health, and a separate anchor for the MCP endpoint referencing its SEP-1649 card (#3311). Refs PR #3341 (sebuf v0.11.1 bundle landed). * test(deploy-config): update SPA catch-all regex assertion The deploy-config guardrail hard-codes the SPA catch-all regex string and asserts its Cache-Control is no-cache. The prior commit added openapi.yaml to the exclusion list; this updates the test to match so the guardrail continues to protect HTML entry caching. * fix(agent-readiness): address Greptile review on PR #3343 - Extract openapi.yaml copy into named script `build:openapi` and prefix every web-variant build (build:full/tech/finance/happy/ commodity). prebuild delegates to the same script so the default `npm run build` path is unchanged. Swap shell `cp` for Node's cpSync for cross-platform safety. - Bump service-desc MIME type in /.well-known/api-catalog from application/yaml to application/vnd.oai.openapi (IANA-registered OpenAPI media type). Endpoint Content-Type stays application/yaml for browser/tool compatibility. * fix(agent-readiness): P1 health href + guardrail tests on PR #3343 - status.href in /.well-known/api-catalog was pointing at https://api.worldmonitor.app/health (which serves the SPA HTML, not a health response). Corrected to /api/health, which returns the real {"status":"HEALTHY",...} JSON from api/health.js. - Extend tests/deploy-config.test.mjs with assertions that would have caught this regression: linkset structure, status/service- desc href shapes, and presence of build:openapi across every web-variant build script.
* chore(api-manifest): rewrite brief-why-matters reason as proper internal-helper justification Carried in from #3248 merge as a band-aid (called out in #3242 review followup checklist item 7). The endpoint genuinely belongs in internal-helper — RELAY_SHARED_SECRET-bearer auth, cron-only caller, never reached by dashboards or partners. Same shape constraint as api/notify.ts. Replaces the apologetic "filed here to keep the lint green" framing with a proper structural justification: modeling it as a generated service would publish internal cron plumbing as user-facing API surface. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * feat(lint): premium-fetch parity check for ServiceClients (closes #3279) Adds scripts/enforce-premium-fetch.mjs — AST-walks src/, finds every `new <ServiceClient>(...)` (variable decl OR `this.foo =` assignment), tracks which methods each instance actually calls, and fails if any called method targets a path in src/shared/premium-paths.ts PREMIUM_RPC_PATHS without `{ fetch: premiumFetch }` on the constructor. Per-call-site analysis (not class-level) keeps the trade/index.ts pattern clean — publicClient with globalThis.fetch + premiumClient with premiumFetch on the same TradeServiceClient class — since publicClient never calls a premium method. Wired into: - npm run lint:premium-fetch - .husky/pre-push (right after lint:rate-limit-policies) - .github/workflows/lint-code.yml (right after lint:api-contract) Found and fixed three latent instances of the HIGH(new) #1 class from #3242 review (silent 401 → empty fallback for signed-in browser pros): - src/services/correlation-engine/engine.ts — IntelligenceServiceClient built with no fetch option called deductSituation. LLM-assessment overlay on convergence cards never landed for browser pros without a WM key. - src/services/economic/index.ts — EconomicServiceClient with globalThis.fetch called getNationalDebt. National-debt panel rendered empty for browser pros. - src/services/sanctions-pressure.ts — SanctionsServiceClient with globalThis.fetch called listSanctionsPressure. Sanctions-pressure panel rendered empty for browser pros. All three swap to premiumFetch (single shared client, mirrors the supply-chain/index.ts justification — premiumFetch no-ops safely on public methods, so the public methods on those clients keep working). Verification: - lint:premium-fetch clean (34 ServiceClient classes, 28 premium paths, 466 src/ files analyzed) - Negative test: revert any of the three to globalThis.fetch → exit 1 with file:line and called-premium-method names - typecheck + typecheck:api clean - lint:api-contract / lint:rate-limit-policies / lint:boundaries clean - tests/sanctions-pressure.test.mjs + premium-fetch.test.mts: 16/16 pass Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(military): fetchStaleFallback NEG_TTL=30s parity (closes #3277) The legacy /api/military-flights handler had NEG_TTL = 30_000ms — a short suppression window after a failed live + stale read so we don't Redis-hammer the stale key during sustained relay+seed outages. Carried into the sebuf list-military-flights handler: - Module-scoped `staleNegUntil` timestamp (per-isolate on Vercel Edge, which is fine — each warm isolate gets its own 30s suppression window). - Set whenever fetchStaleFallback returns null (key missing, parse fail, empty array after staleToProto filter, or thrown error). - Checked at the entry of fetchStaleFallback before doing the Redis read. - Test seam `_resetStaleNegativeCacheForTests()` exposed for unit tests. Test pinned in tests/redis-caching.test.mjs: drives a stale-empty cycle three times — first read hits Redis, second within window doesn't, after test-only reset it does again. Verified: 18/18 redis-caching tests pass, typecheck:api clean, lint:premium-fetch clean. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * refactor(lint): rate-limit-policies regex → import() (closes #3278) The previous lint regex-parsed ENDPOINT_RATE_POLICIES from the source file. That worked because the literal happens to fit a single line per key today, but a future reformat (multi-line key wrap, formatter swap, etc.) would silently break the lint without breaking the build — exactly the failure mode that's worse than no lint at all. Fix: - Export ENDPOINT_RATE_POLICIES from server/_shared/rate-limit.ts. - Convert scripts/enforce-rate-limit-policies.mjs to async + dynamic import() of the policy object directly. Same TS module that the gateway uses at runtime → no source-of-truth drift possible. - Run via tsx (already a dev dep, used by test:data) so the .mjs shebang can resolve a .ts import. - npm script swapped to `tsx scripts/...`. .husky/pre-push uses `npm run lint:rate-limit-policies` so no hook change needed. Verified: - Clean: 6 policies / 182 gateway routes. - Negative test (rename a key to the original sanctions typo /api/sanctions/v1/lookup-entity): exit 1 with the same incident- attributed remedy message as before. - Reformat test (split a single-line entry across multiple lines): still passes — the property is what's read, not the source layout. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(shipping/v2): alertThreshold: 0 preserved; drop dead validation branch (#3242 followup) Before: alert_threshold was a plain int32. proto3 scalar default is 0, so the handler couldn't distinguish "partner explicitly sent 0 (deliver every disruption)" from "partner omitted the field (apply legacy default 50)" — both arrived as 0 and got coerced to 50 by `> 0 ? : 50`. Silent intent-drop for any partner who wanted every alert. The subsequent `alertThreshold < 0` branch was also unreachable after that coercion. After: - Proto field is `optional int32 alert_threshold` — TS type becomes `alertThreshold?: number`, so omitted = undefined and explicit 0 stays 0. - Handler uses `req.alertThreshold ?? 50` — undefined → 50, any number passes through unchanged. - Dead `< 0 || > 100` runtime check removed; buf.validate `int32.gte = 0, int32.lte = 100` already enforces the range at the wire layer. Partner wire contract: identical for the omit-field and 1..100 cases. Only behavioural change is explicit 0 — previously impossible to request, now honored per proto3 optional semantics. Scoped `buf generate --path worldmonitor/shipping/v2` to avoid the full- regen `@ts-nocheck` drift Seb documented in the #3242 PR comments. Re-applied `@ts-nocheck` on the two regenerated files manually. Tests: - `alertThreshold 0 coerces to 50` flipped to `alertThreshold 0 preserved`. - New test: `alertThreshold omitted (undefined) applies legacy default 50`. - `rejects > 100` test removed — proto/wire validation handles it; direct handler calls intentionally bypass wire and the handler no longer carries a redundant runtime range check. Verified: 18/18 shipping-v2-handler tests pass, typecheck + typecheck:api clean, all 4 custom lints clean. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * docs(shipping/v2): document missing webhook delivery worker + DNS-rebinding contract (#3242 followup) #3242 followup checklist item 6 from @koala73 — sanity-check that the delivery worker honors the re-resolve-and-re-check contract that isBlockedCallbackUrl explicitly delegates to it. Audit finding: no delivery worker for shipping/v2 webhooks exists in this repo. Grep across the entire tree (excluding generated/dist) shows the only readers of webhook:sub:* records are the registration / inspection / rotate-secret handlers themselves. No code reads them and POSTs to the stored callbackUrl. The delivery worker is presumed to live in Railway (separate repo) or hasn't been built yet — neither is auditable from this repo. Refreshes the comment block at the top of webhook-shared.ts to: - explicitly state DNS rebinding is NOT mitigated at registration - spell out the four-step contract the delivery worker MUST follow (re-validate URL, dns.lookup, re-check resolved IP against patterns, fetch with resolved IP + Host header preserved) - flag the in-repo gap so anyone landing delivery code can't miss it Tracking the gap as #3288 — acceptance there is "delivery worker imports the patterns + helpers from webhook-shared.ts and applies the four steps before each send." Action moves to wherever the delivery worker actually lives (Railway likely). No code change. Tests + lints unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * ci(lint): add rate-limit-policies step (greptile P1 #3287) Pre-push hook ran lint:rate-limit-policies but the CI workflow did not, so fork PRs and --no-verify pushes bypassed the exact drift check the lint was added to enforce (closes #3278). Adding it right after lint:api-contract so it runs in the same context the lint was designed for. * refactor(lint): premium-fetch regex → import() + loop classRe (greptile P2 #3287) Two fragilities greptile flagged on enforce-premium-fetch.mjs: 1. loadPremiumPaths regex-parsed src/shared/premium-paths.ts with /'(\/api\/[^']+)'/g — same class of silent drift we just removed from enforce-rate-limit-policies in #3278. Reformatting the source Set (double quotes, spread, helper-computed entries) would drop paths from the lint while leaving the runtime untouched. Fix: flip the shebang to `#!/usr/bin/env -S npx tsx` and dynamic-import PREMIUM_RPC_PATHS directly, mirroring the rate-limit pattern. package.json lint:premium-fetch now invokes via tsx too so the npm-script path matches direct execution. 2. loadClientClassMap ran classRe.exec once, silently dropping every ServiceClient after the first if a file ever contained more than one. Current codegen emits one class per file so this was latent, but a template change would ship un-linted classes. Fix: collect every class-open match with matchAll, slice each class body with the next class's start as the boundary, and scan methods per-body so method-to-class binding stays correct even with multiple classes per file. Verification: - lint:premium-fetch clean (34 classes / 28 premium paths / 466 files — identical counts to pre-refactor, so no coverage regression). - Negative test: revert src/services/economic/index.ts to globalThis.fetch → exit 1 with file:line, bound var name, and premium method list (getNationalDebt). Restore → clean. - lint:rate-limit-policies still clean. * fix(shipping/v2): re-add alertThreshold handler range guard (greptile nit 1 #3287) Wire-layer buf.validate enforces 0..100, but direct handler invocation (internal jobs, test harnesses, future transports) bypasses it. Cheap invariant-at-the-boundary — rejects < 0 or > 100 with ValidationError before the record is stored. Tests: restored the rejects-out-of-range cases that were dropped when the branch was (correctly) deleted as dead code on the previous commit. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * refactor(lint): premium-fetch method-regex → TS AST (greptile nits 2+5 #3287) loadClientClassMap: The method regex `async (\w+)\s*\([^)]*\)\s*:\s*Promise<[^>]+>\s*\{\s*let path = "..."` assumed (a) no nested `)` in arg types, (b) no nested `>` in the return type, (c) `let path = "..."` as the literal first statement. Any codegen template shift would silently drop methods with the lint still passing clean — the same silent-drift class #3287 just closed on the premium-paths side. Now walks the service_client.ts AST, matches `export class *ServiceClient`, iterates `MethodDeclaration` members, and reads the first `let path: string = '...'` variable statement as a StringLiteral. Tolerant to any reformatting of arg/return types or method shape. findCalls scope-blindness: Added limitation comment — the walker matches `<varName>.<method>()` anywhere in the file without respecting scope. Two constructions in different function scopes sharing a var name merge their called-method sets. No current src/ file hits this; the lint errs cautiously (flags both instances). Keeping the walker simple until scope-aware binding is needed. webhook-shared.ts: Inlined issue reference (#3288) so the breadcrumb resolves without bouncing through an MDX that isn't in the diff. Verification: - lint:premium-fetch clean — 34 classes / 28 premium paths / 489 files. Pre-refactor: 34 / 28 / 466. Class + path counts identical; file bump is from the main-branch rebase, not the refactor. - Negative test: revert src/services/economic/index.ts premiumFetch → globalThis.fetch. Lint exits 1 at `src/services/economic/index.ts:64:7` with `premium method(s) called: getNationalDebt`. Restore → clean. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * refactor(lint): rate-limit OpenAPI regex → yaml parser (greptile nit 3 #3287) Input side (ENDPOINT_RATE_POLICIES) was flipped to live `import()` in 4e79d02. Output side (OpenAPI routes) still regex-scraped top-level `paths:` keys with `/^\s{4}(\/api\/[^\s:]+):/gm` — hard-coded 4-space indent. Any YAML formatter change (2-space indent, flow style, line folding) would silently drop routes and let policy-drift slip through — same silent-drift class the input-side fix closed. Now uses the `yaml` package (already a dep) to parse each .openapi.yaml and reads `doc.paths` directly. Verification: - Clean: 6 policies / 189 routes (was 182 — yaml parser picks up a handful the regex missed, closing a silent coverage gap). - Negative test: rename policy key back to /api/sanctions/v1/lookup-entity → exits 1 with the same incident-attributed remedy. Restore → clean. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * chore(codegen): regenerate unified OpenAPI bundle for alert_threshold proto change The shipping/v2 webhook alert_threshold field was flipped from `int32` to `optional int32` with an expanded doc comment in f333946. That comment now surfaces in the unified docs/api/worldmonitor.openapi.yaml bundle (introduced by #3341). Regenerated with sebuf v0.11.1 to pick it up. No behaviour change — bundle-only documentation drift. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
Summary
docs/api/worldmonitor.openapi.yaml— a single OpenAPI 3.1 spec spanning every WorldMonitor RPC (190 operations across all services), generated via the new origin-level bundle support in sebuf v0.11.0 (SebastienMelki/sebuf#158 — landed specifically for this project).SEBUF_VERSIONin theMakefilefromv0.7.0→v0.11.0somake install-pluginspicks up the bundle-capableprotoc-gen-openapiv3alongside matchedprotoc-gen-ts-client/protoc-gen-ts-serverbuilds.sed '1s;^;// @ts-nocheck\n;'post-processing — sebuf v0.11.0 TS output type-checks cleanly under our tsconfig (npm run typecheckpasses), so the workaround is no longer needed.CONTRIBUTING.md,AGENTS.md, andCHANGELOG.mdto document the version bump and the per-service vs. unified spec split.Per-service specs (
{Service}.openapi.yaml/.json) are untouched and continue to back the Mintlify docs indocs/docs.json. The bundle is emitted by a thirdprotoc-gen-openapiv3invocation inproto/buf.gen.yamlwithbundle=true,bundle_only=true, andstrategy: all, server URL pinned tohttps://api.worldmonitor.app.Heads-up for reviewers
The second commit (
chore(codegen): regenerate TS client/server with sebuf v0.11.0) is pure codegen — 68 files. The behavioral deltas from sebuf are:*_UNSPECIFIEDsentinels instead of empty strings when deciding whether to omit query params.repeated stringquery params now serialize viaforEach(params.append(...))instead ofString(req.field).Both are strict improvements (the empty-string check was always a bug for enums; the repeated-string path actually matches what the server expects).
Follow-up
This PR is exactly the shape of change that motivates #3340 — moving sebuf codegen out of contributor PRs and into CI/CD. The generated diff is ~69 files / 181 deletions and has nothing to do with the actual intent of the change. Verified that
.github/workflows/proto-check.ymlcurrently only checks freshness, it does not produce the generated files. We should prioritize #3340 before the next sebuf version bump.Test plan
make install-pluginsinstalls v0.11.0 for all three pluginsmake generateproduces the unified bundle (190 operations, spans every service) plus per-service specs with no driftnpm run typecheckpasses without@ts-nocheckon generated filesproto-check.ymlCI job (freshness check) — verify it's green after push🤖 Generated with Claude Code