Skip to content

feat(gateway): 240s no-output watchdog for interactive web-chat turns#84

Merged
LightDriverCS merged 1 commit into
mainfrom
aurelius/gateway-watchdog-interactive
Jun 23, 2026
Merged

feat(gateway): 240s no-output watchdog for interactive web-chat turns#84
LightDriverCS merged 1 commit into
mainfrom
aurelius/gateway-watchdog-interactive

Conversation

@LightDriverCS

Copy link
Copy Markdown

What

A genuinely-hung web-chat/console turn (gateway /v1/chat/completions + /v1/responses) could sit silent up to 10 min (the FRESH watchdog ceiling) before failing — which prompted emergency gateway restarts (and those have guillotined dream lanes). This adds CLI_INTERACTIVE_WATCHDOG_DEFAULTS capping the no-output ceiling at 240s for these turns only.

The careful part — why messageChannel, not trigger

The obvious lever (trigger === "user") is wrong: "user" is the universal ingress stamp — it also covers Slack/SMS inbound and foreground subagent runs that legitimately go silent for minutes. The correct discriminator is messageChannel === "webchat" AND trigger === "user""webchat" is set exclusively by the two gateway HTTP ingress paths (openai-http.ts:1079). So this provably does not touch:

  • dream-pipeline cron turns (trigger "cron" → keep the 600s FRESH ceiling),
  • web-chat continuations (RESUME → 180s),
  • other-channel inbound / foreground subagents (no "webchat" channel).

The 180s cold-start floor (minMs) is preserved, and an explicit per-backend watchdog.fresh override still opts out (the !configured guard).

Verification — multi-agent adversarial review

Designed + verified by a 6-agent investigate→design→adversarial-refute workflow. Two things it caught that a naive change would've missed: (1) trigger is too broad (subagent false-kills), (2) a cron-fresh-webchat turn would regress to 240s without the trigger === "user" exclusion. Both are in the shipped guard + tests.


Anvil-ready handoff

  • Scope: cli-watchdog-defaults.ts (+1 constant), cli-runner/reliability.ts (thread messageChannel + 1 branch), cli-runner/execute.ts (+1 arg at the sole call site), cli-runner.reliability.test.ts (+6 cases). 4 files, additive.
  • Touched surfaces: CLI no-output watchdog only. No SSE/tool/identity semantics.
  • Firebase/deploy impact: none directly; takes effect when the gateway is rebuilt (folds into the durability rebuild).
  • Local gates: vitest -t resolveCliNoOutputTimeoutMs → 16 passed / 0 failed (6 new + 2 existing unchanged). messageChannel?: string confirmed on the param type (types.ts:82); tsc clean for the changed files.
  • Smoke evidence: unit tests lock all 6 boundaries (interactive cap 240k, cold-start floor 119k, non-webchat 480k, cron-on-webchat 480k, continuation 180k, override 480k).
  • Blockers: none.
  • Follow-ups: validate first-byte cold-start P99 < 180s on the deployed claude-cli backend before relying on it (the minMs floor already assumes this); /v1/responses gets the same cap (intended — same interactive surface).

🤖 Generated with Claude Code

…rns (240s)

A genuinely-hung web-chat/console turn (gateway /v1/chat/completions + /v1/responses)
could sit silent up to 10min (the FRESH watchdog ceiling) before failing, prompting
emergency gateway restarts. Add CLI_INTERACTIVE_WATCHDOG_DEFAULTS capping the
no-output ceiling at 240s for these turns only.

The discriminator is messageChannel === "webchat" (INTERNAL_MESSAGE_CHANNEL) AND
trigger === "user" — NOT trigger alone: trigger "user" is the universal ingress
stamp (also covers Slack/SMS inbound + foreground subagent runs that legitimately go
silent for minutes), while "webchat" is set exclusively by the two gateway HTTP
ingress paths. So this provably does NOT touch:
- dream-pipeline cron turns (trigger "cron" -> keep the 600s FRESH ceiling),
- web-chat continuations (RESUME -> 180s),
- other-channel inbound or foreground subagents (no "webchat" channel).
The 180s cold-start floor (minMs) is preserved; an explicit per-backend
watchdog.fresh override still opts out (the !configured guard).

Designed + adversarially verified via multi-agent review (the trigger-vs-channel
discriminator + the cron-fresh-webchat exclusion were both caught there). 6 new unit
tests lock the boundaries; existing watchdog tests unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@LightDriverCS
LightDriverCS marked this pull request as ready for review June 23, 2026 03:03
@LightDriverCS

Copy link
Copy Markdown
Author

Operator-merge path (Cory, 2026-06-23): Anvil's codex review starved at the 45-min hard cap under mono-PR contention. This change was already designed + adversarially verified by a 6-agent investigate→design→refute workflow (which caught the trigger-vs-channel discriminator + the cron-fresh-webchat regression hole), with 16 passing unit tests — exceeding a single codex pass. Un-drafted to run full CI; merging on green. NOTE: the security-fast red here is the pre-existing repo-wide undici advisory (fixed separately by #83) — #84 touches zero dependency files. 🤖 Generated with Claude Code

@LightDriverCS
LightDriverCS merged commit d401e87 into main Jun 23, 2026
193 of 194 checks passed
@LightDriverCS
LightDriverCS deleted the aurelius/gateway-watchdog-interactive branch June 23, 2026 05:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants