Skip to content

gateway test-suite remediation (#2720): deferred cleanups, parity gaps, and skip-restorations #2724

Description

@alexey-pelykh

Summary

Tracking issue for the smaller deferred items surfaced while re-greening the gateway behavioral test suite. Each is low/medium severity and was intentionally kept out of the make-the-suite-green change. The three HIGH-severity security gaps from the same effort are tracked in their own issues.

Items

Parity / behavior

  • senderIsOwner on the OpenAI-compat endpoint. Derive senderIsOwner from request auth on the OpenAI-compatible HTTP endpoint. Requires machinery currently absent from the fork (usesSharedSecretGatewayMethod, an AuthorizedGatewayHttpRequest request type) plus wiring senderIsOwner into buildAgentCommandInput. The shared-secret-bearer → owner arm is already covered by a passing test. Covered test is it.skip-ped in src/gateway/openai-http.test.ts (~line 628).
  • Canonical default-agent id. src/gateway/assistant-identity.ts resolves the default agent id as the literal "default" while http-utils.ts uses "main". Route both through resolveDefaultAgentId so the canonical id has a single source.

Hardening (lower severity)

  • Agent-trust sanitization (Medium). Sanitize agent-hook system-event strings (the full string including the summary, not just the name) via sanitizeInboundSystemTags, and decide whether such events should be marked trusted: false. Prompt-injection surface. The covering test (src/gateway/server/hooks.agent-trust.test.ts) was deleted when the secrets runtime was gutted; restore it when implemented.
  • Plugin gutted-posture regression guard. Add a guard asserting that (a) plugin-route auth stays gutted (residual plugins-http.ts guard) and (b) the scope-confinement and per-origin rate-limit helpers stay called/referenced (positive-presence-AND-wiring, to catch a control that is present but no longer invoked). Also audit canvasHost.handleHttpRequest internal authorization for the HTTP-pass / WS-reject asymmetry. This guards against a future upstream merge silently re-introducing or de-wiring these controls.
    • PARTIAL (2026-06-17): sub-part (a) (plugin-route auth stays gutted) is now covered by a behavioral integration tripwire — src/gateway/server.plugin-gateway-auth-tripwire.test.ts (PR test(gateway): pin plugin auth:gateway gutted-posture tripwire (#2724) #2726): it drives an auth:"gateway" route through the real server pipeline and asserts the un-propagated-operator-scope guard still blocks dispatch. STILL OPEN: (b) the scope-confinement + per-origin rate-limit helper wiring guard, and the canvasHost.handleHttpRequest HTTP-pass / WS-reject authorization audit.
  • Cron unsafe-job advisory (LOW). Warn-once / rate-limit the "unsafe persisted job" log and provide a working doctor --fix quarantine, so a hand-edited store containing an unsafe job cannot flood logs on every tick.
  • SecretRef pre-validation (LOW). Validate SecretRef shape before use on the config path.

Dead code / cleanup

  • Remove orphaned policy stubs. VOID — CORRECTION (2026-06-17): src/agents/pi-tools.policy.ts is LIVE; do NOT remove. It is statically imported by src/security/audit-extra.sync.ts + src/security/audit-extra.async.ts (isToolAllowedByPolicies) — a gutted-in-place security-audit stub, not dead code. The 3 undefined-returning shims are deliberately tolerated by callers (MODULE_ATTESTATIONS: partial). The original "zero importers" premise was wrong; removing it would break the security-audit deny/allow path.
  • Remove talk.speak orphan declarations. ✅ DONE (PR chore(gateway): remove talk.speak orphan method declarations (#2724) #2727, 2026-06-17): removed from BASE_METHODS (server-methods-list.ts) + WRITE_SCOPE (method-scopes.ts). No handler existed (talkHandlers = talk.config + talk.mode only); method-scopes consistency test 23/23 confirmed orphan-safety.
  • Remove the allowGatewaySubagentBinding dead field (no live consumer).
  • Evaluate secrets.resolve declaration (orphan vs intentional). ✅ RESOLVED → REMOVE (PR chore(gateway): drop dead secrets.resolve method + reconcile stale talk.speak/secrets docs (#2724) #2728, 2026-06-17). Evaluation outcome: true orphan. No handler (server-methods/secrets.ts gutted) and zero production callers (grep finds only the BASE_METHODS + method-scopes.ts registry sites; requiredMethods itself has no production caller). The "intentionally advertised for graceful fallback" hypothesis was refuted: advertising a handler-less method makes the requiredMethods pre-check (call.ts) pass, then dispatch hard-errors unknown method — strictly worse than honest non-advertisement. The fork's SecretRef resolution is local (src/secrets/resolve.ts), independent of the RPC; the call.test.ts/message.test.ts references are fixture strings, not real callers (verified — message.test.ts results are identical with/without the removal). Gateway behavioral test suite is gated out of CI — recently-merged fixes have no behavioral coverage #2720 had deliberately kept it, deferring to this evaluation. Removed from both registries, mirroring talk.speak.
  • Reconcile stale talk.speak doc references. ✅ DONE (PR chore(gateway): drop dead secrets.resolve method + reconcile stale talk.speak/secrets docs (#2724) #2728, 2026-06-17). docs/nodes/talk.md (×2) + docs/gateway/protocol.md rewritten to the fork's actual Talk-mode playback: the native apps (macOS / iOS / Android) synthesize on-device via the ElevenLabs API directly, falling back to the platform's system TTS (AVSpeechSynthesizer / Android TextToSpeech) when no ElevenLabs key — the gateway is config/state only (talk.config, talk.mode), never synthesis. (Forensics confirmed all three platforms call ElevenLabs directly; the gutted talk.speak RPC was never the live path.)
  • (follow-up A) Reconcile secrets.md + qr.md command-path SecretRef-resolution docs. (Surfaced 2026-06-17 by PR chore(gateway): drop dead secrets.resolve method + reconcile stale talk.speak/secrets docs (#2724) #2728.) docs/gateway/secrets.md § Command-path resolution (+ the reloader-degraded section) and docs/cli/qr.md (~line 49 version-skew note; ~line 44) still describe SecretRef resolution as going through a gateway secrets.resolve snapshot RPC. That RPC is now gone; the fork resolves SecretRefs locally (src/secrets/resolve.ts, gateway/resolve-configured-secret-input-string.ts — neither touches a gateway/snapshot RPC). Rewrite to the fork's actual local-resolution story (incl. whether remoteclaw secrets reload / the reloader section still applies). Subsystem-accurate rewrite, not mechanical.
  • (follow-up B) Reconcile protocol.md "Common RPC method families" against BASE_METHODS. (Surfaced 2026-06-17 by PR chore(gateway): drop dead secrets.resolve method + reconcile stale talk.speak/secrets docs (#2724) #2728.) The hand-maintained method-families reference in docs/gateway/protocol.md still documents methods Gateway behavioral test suite is gated out of CI — recently-merged fixes have no behavioral coverage #2720 removed from BASE_METHODS: models.list, skills.search/skills.detail/skills.bins, the plugin.approval.* methods (the events plugin.approval.requested/resolved survive in GATEWAY_EVENTS — handle the methods-vs-events distinction), commands.list, and likely the sessions.usage* family / exec.approval.get/list (classified but not advertised — verify each against handlers). Do one reconciliation pass with per-method handler verification (an inaccurate "correction" is worse than a flagged gap). PR chore(gateway): drop dead secrets.resolve method + reconcile stale talk.speak/secrets docs (#2724) #2728 already removed the talk.speak + secrets.reload + secrets.resolve bullets.

Kept-capability gaps to evaluate (fix-prod vs accept-divergence)

  • Non-round-trip agent-directory discovery (two cases) — directory names that do not round-trip through normalizeAgentId.
  • loadSessionEntry should prefer the freshest duplicate with exact-match-first ordering.
  • Disk-only agent under a custom session-store root: confirm whether a disk-discovered agent under a non-default store root should be resolvable, or whether the divergence is intentional.

Test harness capability (to un-skip the remaining deferrals)

  • Add a harness capability to originate a remote (non-local-direct) node connect that requires explicit pairing approval. This is needed to un-skip the two test.skip cases in src/gateway/server.roles-allowlist-update.test.ts (~lines 400, 461) and the related control-UI pairing cases. Silent local auto-pairing is the secure-on-loopback behavior (no fail-open), which is why these cannot be exercised with the current local-only harness.
  • Design the legacy main-alias remap boundary so sessions-resolve.ts can remap agent:main:main onto the live default agent (currently rejected). Un-skip src/gateway/sessions-resolve-store.test.ts (~line 17) once the boundary is reviewed.

Optional coverage

  • Add a gateway-level integration test for "a method is reported unavailable when a run does not stop."

Context

All items above are documented inline at their it.skip / deletion sites in the gateway test suite, with the rationale and un-skip condition. This issue aggregates them so they are not lost.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions