Skip to content

fix(infra): treat undici terminated aborts as transient#31860

Closed
liuxiaopai-ai wants to merge 1 commit into
openclaw:mainfrom
liuxiaopai-ai:codex/undici-terminated-nonfatal-31855
Closed

fix(infra): treat undici terminated aborts as transient#31860
liuxiaopai-ai wants to merge 1 commit into
openclaw:mainfrom
liuxiaopai-ai:codex/undici-terminated-nonfatal-31855

Conversation

@liuxiaopai-ai

Copy link
Copy Markdown
Contributor

Summary

Describe the problem and fix in 2–5 bullets:

  • Problem: Gateway could exit on unhandled rejection when Node/undici surfaced TypeError: terminated from Fetch.onAborted.
  • Why it matters: transient upstream disconnect/abort paths should not crash the whole gateway process.
  • What changed: added targeted detection for undici-style TypeError("terminated") (stack/cause-aware) and classify it as transient network error.
  • What did NOT change (scope boundary): no retry policy, no provider-specific request logic, no change to fatal/config error handling paths.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

User-visible / Behavior Changes

  • Gateway no longer exits on undici TypeError: terminated abort/disconnect rejections; it logs as non-fatal transient network error and continues running.

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/tokens handling changed? (No)
  • New/changed network calls? (No)
  • Command/tool execution surface changed? (No)
  • Data access scope changed? (No)
  • If any Yes, explain risk + mitigation:

Repro + Verification

Environment

  • OS: macOS
  • Runtime/container: Node 25.x local test run
  • Model/provider: N/A
  • Integration/channel (if any): N/A
  • Relevant config (redacted): N/A

Steps

  1. Trigger installUnhandledRejectionHandler with an unhandled rejection reason matching undici abort shape: TypeError("terminated") with Fetch.onAborted stack.
  2. Observe gateway unhandled rejection handling path.
  3. Run infra unhandled-rejection unit tests.

Expected

  • Reason is classified transient; process is not exited.

Actual

  • Before fix: generic unhandled rejection branch could exit process.
  • After fix: non-fatal branch logs warning and continues.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

pnpm test src/infra/unhandled-rejections.test.ts src/infra/unhandled-rejections.fatal-detection.test.ts

  • Passed: 2 files, 36 tests
  • Added regression cases for TypeError: terminated and fatal-detector non-exit behavior

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios:
    • isTransientNetworkError returns true for undici TypeError: terminated with Fetch.onAborted stack.
    • plain TypeError: terminated without undici/abort context remains non-transient.
    • installUnhandledRejectionHandler does not exit for undici-terminated transient case.
  • Edge cases checked:
    • Existing TypeError: fetch failed and nested-cause transient paths still pass.
    • Non-transient generic errors still exit.
  • What you did not verify:
    • Live upstream reproduction from real gateway runtime logs.

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (No)
  • Migration needed? (No)
  • If yes, exact upgrade steps:

Failure Recovery (if this breaks)

  • How to disable/revert this change quickly: revert commit 28d93597a.
  • Files/config to restore:
    • src/infra/unhandled-rejections.ts
    • src/infra/unhandled-rejections.test.ts
    • src/infra/unhandled-rejections.fatal-detection.test.ts
  • Known bad symptoms reviewers should watch for:
    • Unexpected suppression of truly fatal non-network TypeError: terminated errors (guarded by stack/cause checks).

Risks and Mitigations

List only real risks for this PR. Add/remove entries as needed. If none, write None.

  • Risk: over-classifying generic TypeError: terminated as transient.
    • Mitigation: detection requires undici-specific stack marker or abort/UND_ERR cause evidence; added negative regression test for plain TypeError: terminated.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 28d93597a4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

return true;
}
const causeMessage = getErrorMessage(cause).toLowerCase();
if (causeMessage.includes("aborted") || causeMessage.includes("terminated")) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Tighten cause-message heuristic for terminated errors

The new isUndiciTerminatedFetchError path treats TypeError("terminated") as transient whenever cause.message contains "aborted" or "terminated", even without undici-specific evidence. That over-matches non-network failures (for example, an app-level TypeError("terminated") caused by a business-rule cancellation message), which then makes installUnhandledRejectionHandler suppress process exit for a genuinely unhandled runtime error. Please limit this fallback to stronger undici indicators (stack markers, AbortError, or UND_ERR_* codes) instead of generic message substrings.

Useful? React with 👍 / 👎.

@greptile-apps

greptile-apps Bot commented Mar 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR addresses a bug where the gateway could crash on undici TypeError: terminated from Fetch.onAborted during client disconnects. The implementation adds targeted detection in isUndiciTerminatedFetchError() with multiple fallback mechanisms: stack trace inspection for undici markers, AbortError cause detection, UND_ERR_ error code checking, and cause message pattern matching. The detection is comprehensive but not overly strict - it will classify TypeError("terminated") as transient if it has abort-related evidence in the stack or cause chain, which appropriately balances safety (preventing crashes on transient network issues) with specificity. Test coverage includes both positive cases (undici terminated errors) and negative cases (plain TypeError without undici context).

Confidence Score: 4/5

  • Safe to merge with minimal risk - addresses a real crash scenario with appropriate guardrails
  • The implementation is well-structured with comprehensive detection logic and good test coverage. The risk of over-classifying generic TypeError("terminated") is mitigated by requiring undici-specific evidence (stack markers, UND_ERR_ codes, or abort-related cause patterns). The broader cause message check (lines 191-194) could theoretically catch non-undici errors, but in the context of a gateway handling HTTP requests, this is likely intentional and acceptable.
  • No files require special attention - changes are focused and well-tested

Last reviewed commit: 28d9359

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gateway crashes on undici Fetch.onAborted (TypeError: terminated)

2 participants