Skip to content

docs(gateways): send OpenRouter Broadcast traces with the X-Project-Name header - #1033

Draft
nik13 wants to merge 3 commits into
mainfrom
docs/openrouter-broadcast-direct
Draft

nik13 wants to merge 3 commits into
mainfrom
docs/openrouter-broadcast-direct

Conversation

@nik13

@nik13 nik13 commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Status: draft — real Broadcast delivery still required

TH-8244 / TH-8103. Do not merge or publish yet. The Future AGI receiver and the documented model request have been exercised separately; neither proves that a configured OpenRouter Broadcast destination delivered a trace.

The shared OpenRouter API key is used by other teams. No Broadcast toggle, destination, filter, sampling, privacy or account/workspace setting was changed. No accounts were created. No application browser was driven.

What changed and why

  • Merged current main (ddca13930bd9c4b9cc80384b7cf2b1a0d71fa43d) into the existing PR branch without rebasing, force-pushing or renaming it. Resolved the page conflict against the cleaned-up main version, not the old verification diary.
  • Replaced the on-page copied collector recipe with the native OpenTelemetry Collector destination: https://api.futureagi.com/tracer/v1/traces, OTLP/HTTP JSON, and X-Api-Key, X-Secret-Key, X-Project-Name.
  • Made the project-header contract self-contained: workspace-scoped key pair, resource-attribute precedence, one project name, length and invalid-character rules, self-hosted upgrade requirement. Removed the dependency on the absent #naming-the-project anchor in still-open docs#1028.
  • Linked the canonical API-keys documentation, removed obsolete release blockers and unsupported claims that direct exports always have empty panes/zero cost/no users. Current receiver source has newer OpenRouter mappings; the guide limits display promises to exported attributes rather than asserting an unverified deployment version.
  • Kept credential-sharing, shared-account, key-filter and privacy decisions explicit. Current OpenRouter documentation says Privacy Mode removes provider error text as well as prompt/completion content; metadata and model parameters remain. Switching away from a filtering collector bypasses its filters, and FI_CAPTURE_CONTENT does not control direct export.
  • Updated the integration catalog to match the direct setup. Replaced page/config-byte-parity checks with direct-route contract tests; retained collector behavior tests. Clarified the internal example README so synthetic fixtures are not presented as actual Broadcast delivery or a customer prerequisite.

Sources and claim scope

Read the relevant current OpenRouter OTLP destination, Broadcast, and Webhook documentation. Cross-checked Future AGI's current fi-collector/pkg/auth/projectheader.go, pkg/server/server.go, pkg/server/openrouter_broadcast_test.go, pkg/adapter/adapter.go, and exporter/clickhouse25exporter/converter.go.

This is not a claim that a synthetic fixture reproduces every current vendor payload, that the provider's privacy behavior was tested, or that HTTP 200 proves storage/readback.

Verification

All builds and tests ran in E2B or GitHub Actions; none ran locally.

  • E2B th8103-or1033-1109-r1: executed the guide's curl block unchanged, one openai/gpt-4o-mini request capped at 16 output tokens; succeeded with 14 input / 7 output tokens. The Future AGI header JSON was exercised against a loopback OTLP/HTTP collector. Two explicitly synthetic Broadcast-shaped JSON bodies were captured, with current and legacy token names and no project_name or project_type resource attribute. Provider-key comparison was false for both bodies before Cloud forwarding. The staged provider key was removed after this one run; Future AGI keys never entered E2B.
  • Cloud receiver qualification to th8103-docs: both captured bodies returned HTTP 200 using X-Project-Name. The same body with a blank project header returned 400; a comma-separated project name also returned 400. Trace IDs: a9ef26b4d7478f2271ea36ae16a3f0b6 and a0a779964686289ef96a49df2f8c6a65. These are synthetic replay IDs, not real Broadcast-delivery IDs. No storage or UI readback is claimed.
  • E2B th8103-or1033-1109-r2: original main page tests 12/12; final page tests 10/10; replacing the final page with main's old page makes 8 final tests fail, proving regression sensitivity. Full Broadcast suite 71/71; integration suite 273/273; internal link audit 0 broken nav and 0 broken content links; integration type check passed; full Astro/Pagefind build passed.
  • Harness retries are recorded rather than hidden: r1's build exhausted Node's default approximately 2 GiB heap; r2 built successfully with a 6 GiB heap in the 8 GiB sandbox, then the temporary rendered-anchor checker incorrectly rejected the valid href="#" top-of-page link. The checker was corrected without changing product code. Final confirmation run th8103-or1033-1109-r3 completed with exit 0: 71/71 Broadcast tests, 273/273 integration tests, the same base/reversion checks, type check, link audit, full build and Pagefind (850 indexed pages), and all 10 rendered in-page links resolved (including the top-of-page link). All three E2B sandboxes were deleted and confirmed gone.
  • Final GitHub CI on 64c51f1f7fe09e049ef9befba1618cc58d1fffbc: all eight checks finished green. https://github.com/future-agi/docs/actions/runs/42546229267 (PR Checks, including build/Pagefind and the repository's docs-browser suite), https://github.com/future-agi/docs/actions/runs/42546229274 (Temporal Go), https://github.com/future-agi/docs/actions/runs/42546229471 (FastMCP). The final branch is clean and contains current main (0 commits behind).

Re-run the committed checks with python -m pytest examples/gateways/openrouter-broadcast/tests -q, npm run test:integrations, npm run audit-links, npm run check:integrations, and npm run build in a cloud runner. The guide itself contains the real-request command and the destination configuration.

E2E coverage and review

Not applicable: no e2e/ application coverage harness in this docs repository. Python test changes were run against both base and final page; no Ruff-configured pyproject, migrations, frontend API contracts or backend behavior changed. Reviewed the final four-file diff against internal minimal-code, PR, naming/security and testing standards. The remaining release blocker is external delivery qualification, not a claimed code defect. No approval or request-changes review was posted.

Exact remaining human step

The OpenRouter account owner or organization administrator must explicitly approve an isolated qualification destination. Use a dedicated test API key rather than the shared cross-team key:

  1. In Settings > Observability, enable Broadcast if required and configure an OpenTelemetry Collector destination with the Cloud endpoint and the three documented headers, selecting project th8103-docs.
  2. Restrict the destination to that dedicated key, use sampling 1.0, and deliberately choose Privacy Mode. Do not change other teams' destinations or filters.
  3. Click Test Connection, save, run the guide's short request with the dedicated key, and confirm the actual exported trace and generation attributes in the intended Future AGI workspace/project.
  4. Attach that real Broadcast-delivery trace ID/evidence. Only then resolve any observed mapping issue, mark this PR ready, and consider closing TH-8244 after publication.

Until then this PR remains draft; TH-8244 is not complete. No merge, release or ticket closure was performed.

🤖 Generated with Claude Code

…Broadcast page

Option 1 points Broadcast's OpenTelemetry Collector destination straight
at Future AGI with X-Api-Key, X-Secret-Key and X-Project-Name. It needs
the collector release that reads X-Project-Name (future-agi/future-agi#3590)
and has not been run live. Option 2 is the existing collector recipe from
docs#949, kept byte for byte, and works today.

The page now has a route comparison, What shows up split into rows shared
by both routes and rows that differ (span types, token names, cost, users,
prompt and response, error text), privacy notes per route, the response
table for the direct route, and troubleshooting for both. The stand-in's
straight-to-Future AGI results stay, framed as a collector without the
header.

Replaces 3a6ef17, which added the direct route as a separate page at the
same path.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@nik13
nik13 force-pushed the docs/openrouter-broadcast-direct branch from 3a6ef17 to 233e585 Compare October 25, 2026 23:38
@nik13
nik13 changed the base branch from main to feat/th-8244-openrouter-broadcast October 25, 2026 23:38
Base automatically changed from feat/th-8244-openrouter-broadcast to main October 29, 2026 21:00
nik13 and others added 2 commits November 8, 2026 17:29
Replace the obsolete collector recipe with the direct OTLP/HTTP destination and self-contained project-header contract. Keep privacy and shared-account boundaries explicit, align the catalog and test the native page contract. Real Broadcast delivery remains a draft release gate.

Co-Authored-By: Claude Code <[email protected]>
Keep synthetic replay and collector regression assets explicitly separate from real Broadcast-delivery proof.

Co-Authored-By: Claude Code <[email protected]>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant