feat(v2): Firebase token attachment + WS frame tracing for cloud-brain (Switch 2-CLI)#9
Merged
Merged
Conversation
Writes `scripts/cloud-brain-smoke.mjs` per V2 runbook §"Validation
script". Exercises the cloud-brain end-to-end path once Phase 1B
merges (BenchAGI #872 W1 + #874 W4 + #878 W2 + #988 relay +
openclaw#24 W3) and an agent's deployment is flipped to
runtime: 'remote-brain'.
What the script does:
- Lists known agents from the local openclaw gateway.
- Queries Firestore (admin REST + gcloud token per the documented
recipe) for `agentDeployments/{instanceId}_{agentId}.runtime`.
- For each agent with `runtime === 'remote-brain'`, spawns
`benchagi --agent <name> --liveness off "respond: smoke-ok"`
with stdout captured and a 60s timeout.
- Asserts: exit 0, non-empty stdout, no error markers, latency
under 60s.
- Emits JSON summary; exits 0 only if all tested agents passed.
Required env: INSTANCE_ID. Optional: GCP_PROJECT (default
benchagi-8ea90), SMOKE_AGENT_FILTER, SMOKE_PROMPT, SMOKE_TIMEOUT_MS.
Gated on cloud-brain Phase 1B merging + a deployment flipped to
remote-brain. Until then, the script reports "no remote-brain
agents found — Phase 1B may not be merged + flipped yet" and exits
0 (not a failure).
Stays as a draft PR until Cory merges Phase 1B and flips at least
one deployment, at which point we run the smoke + capture the
transcript for ANVIL-5.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Codex Anvil 4 finding: HOLD. Two P0s + two P1s + a missing-test
flag.
Most important: Codex couldn't find a readable local gateway path
that routes chat.send through cloud-brain when an agent's
deployment has runtime: 'remote-brain'. Without that dispatch
bridge, the smoke script will pass on local execution without
exercising cloud-brain at all. The bridge MAY be in W2 (#878)
which Codex couldn't see in full diff via the connector; needs
verification.
Other findings:
- P1: bench-cli's chat.history call expects events/frames but the
readable OpenClaw chat.history handler returns transcript
messages and ignores sinceSeq. V1.1 reconnect replay is
best-effort no-op until this contract is reconciled.
- P1: Firestore doc id format assumption in the smoke
(instances/{instanceId}/agentDeployments/{instanceId}_{agentId})
may not match real W1 backfill output if deploymentIds differ.
- P2: Firestore failures recorded as skips can produce false
exit-0 if no agents tested.
- P2: smoke assumes repo-root cwd.
Stronger smoke proposed: assert directive artifacts in Firestore
(relayDirectives doc with directiveType: llm_turn), assert directive
reaches completed, AND assert CLI saw normal chat/agent.lifecycle
frames. That proves both halves of ADR-006 transparency.
Doesn't change the smoke script in this commit — the structural
concern (missing dispatch bridge) needs resolution first. The
smoke script is preserved as-is (P0 #2 acknowledged as a review
finding) so Cory can decide which path to take.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…n (Switch 2-CLI)
Pairs with the BenchAGI mono cloud bridge endpoint + the OpenClaw
gateway bridge to enable transparent cloud-brain dispatch from the
CLI. The CLI side is small: attach the user's Firebase ID token to
chat.send so the gateway can authenticate to the cloud bridge.
## What's new
- `src/v2/auth/firebase-token.ts` — loads Firebase Direct creds
from the existing keychain. Refreshes ID token using the public
Firebase Identity Toolkit when one of these is set:
BENCHAGI_FIREBASE_API_KEY
NEXT_PUBLIC_FIREBASE_API_KEY
FIREBASE_API_KEY
- `src/v2/chat-runner.ts` — sendMessage now resolves the Firebase
token (best-effort) and adds `cloudAuth.firebaseIdToken` to the
chat.send request. If no creds are available, chat.send fires
without the field and the gateway falls back to local dispatch
per ADR-006's transparency contract.
- `src/v2/transport/local-gateway.ts` — accepts the cloudAuth field
in chat.send param shape; passes through unchanged.
- `src/v2/cli.ts` — adds raw WS frame tracing for diagnostics:
--trace-frames <path>
BENCHAGI_TRACE_FRAMES=<path>
Each WS frame the CLI receives is appended as a JSONL line to
the file. Useful for correlating CLI events to relayDirectives
during the smoke test.
## Verification
- `npm run build` — clean
- Existing test suite still passes (no behavior change for the
default-local path; only adds the cloudAuth field when a token
is available).
## Limitations / follow-ups
- chat.send acceptance race (V1.1 ANVIL-2 P1 deferred) — still not
fixed in this PR. Re-issuing chat.send on reconnect with same
idempotencyKey is a separate cycle.
- Token refresh path silently no-ops if no Firebase API key env
var is set. CLI users who don't have one configured will fall
through to local-only dispatch even on remote-brain agents
(which is the safe default).
- Frame trace file is append-only; no rotation. Each smoke run
should use a fresh path.
## Pairs with
- BenchAGI mono PR [#1016](BenchAGI/BenchAGI_Mono_Repo#1016) — cloud bridge endpoint
- OpenClaw fork PR [#26](BenchAGI/openclaw#26) — gateway-side dispatch + polling
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CLI-side completion of the Switch 2-CLI bridge. Pairs with the BenchAGI mono cloud bridge endpoint and the OpenClaw gateway bridge to make ADR-006 transparency real: the CLI attaches its Firebase token to `chat.send`, the gateway uses it to authenticate to the cloud bridge endpoint, and remote-brain runs flow back as normal `chat`/`agent` events.
Pairs with:
What's new
Each frame received is appended as JSONL. Useful for correlating CLI events to `relayDirectives` during the smoke test.
Verification
Limitations / follow-ups
Anvil Handoff
This PR will get one Codex Anvil pass before merge.
🤖 Generated with Claude Code