Skip to content

fix(session): avoid stalls when parent token probing hangs#101932

Open
LiLan0125 wants to merge 2 commits into
openclaw:mainfrom
LiLan0125:fix/101718-session-fork-token-timeout
Open

fix(session): avoid stalls when parent token probing hangs#101932
LiLan0125 wants to merge 2 commits into
openclaw:mainfrom
LiLan0125:fix/101718-session-fork-token-timeout

Conversation

@LiLan0125

@LiLan0125 LiLan0125 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Closes #101718

What Problem This Solves

Fixes an issue where users creating or forking a session from a parent could wait indefinitely when parent transcript token probing did not settle.

Why This Change Was Made

The parent fork decision now uses fresh persisted token counts immediately, and otherwise bounds transcript token probing to a short wait. If probing times out or cannot verify the parent size, OpenClaw starts the child with isolated context instead of continuing into parent transcript copy. Quickly resolved small parents still fork, and fresh or quickly resolved oversized parents still use the existing isolated-session skip behavior.

User Impact

Session creation and parent-fork workflows no longer stay blocked behind a stuck parent token probe or continue into another unbounded parent transcript read after the probe timeout. Users with oversized or unverified parent sessions get isolated context instead of inheriting unsafe parent history.

Impact Assessment

  • Scope: parent-session fork decision logic shared by session creation, parent-fork prepare, and subagent spawn paths.
  • Downstream: callers already handle skip decisions by avoiding transcript copy; timeout and unavailable-size probes now use that existing isolated-context path.
  • Edge cases: fresh token counts bypass transcript probing; quickly resolved oversized counts still skip; stalled probes clear their timer after the bounded wait; unresolved size probes do not fork.
  • Config surface: no new configuration, environment variable, or storage format.

Evidence

  • node scripts/run-vitest.mjs src/auto-reply/reply/session-fork.test.ts
[test] starting test/vitest/vitest.auto-reply.config.ts
[test] passed 1 Vitest shard in 22.71s
  • pnpm tsgo:prod
passed
  • pnpm check:test-types
passed

Real behavior proof

Behavior addressed: Parent fork caller paths return instead of waiting indefinitely when parent token probing does not settle, and the timeout path avoids the later parent transcript copy.
Environment tested: Linux, Node v22.22.0, local source checkout on branch fix/101718-session-fork-token-timeout, isolated temporary OPENCLAW_HOME and OPENCLAW_STATE_DIR.
Steps run after the patch: Called the patched forkSessionEntryFromParent production caller path with a parent transcript whose token-probe fs.promises.stat never resolves, then verified fresh oversized parent tokens still produce the existing skip decision.
Evidence after fix:

node --import tsx --no-warnings -e '
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";

const root = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-101718-proof-")));
process.env.OPENCLAW_HOME = path.join(root, "home");
process.env.OPENCLAW_STATE_DIR = path.join(root, "state");
const { forkSessionEntryFromParent } = await import("./src/auto-reply/reply/session-fork.js");
const activeStoreDir = path.join(root, "active-store");
await fs.mkdir(activeStoreDir, { recursive: true });
const storePath = path.join(activeStoreDir, "sessions.json");
const parentSessionKey = "agent:main:main";
const sessionKey = "agent:main:subagent:child";
const parentSessionFile = path.join(activeStoreDir, "parent.jsonl");
const now = Date.now();
await fs.writeFile(parentSessionFile, [
  JSON.stringify({ type: "session", version: 3, id: "parent-session", timestamp: "2026-07-08T00:00:00.000Z", cwd: root }),
  JSON.stringify({ type: "message", id: "assistant-1", parentId: null, timestamp: "2026-07-08T00:00:01.000Z", message: { role: "assistant", content: "parent branch" } }),
].join("\n") + "\n", "utf-8");
await fs.writeFile(storePath, JSON.stringify({
  [parentSessionKey]: { sessionId: "parent-session", sessionFile: parentSessionFile, updatedAt: now },
  [sessionKey]: { sessionId: "", updatedAt: now },
}, null, 2), "utf-8");

const originalStat = fs.stat;
let parentStatCalls = 0;
fs.stat = async (filePath, ...args) => {
  if (String(filePath) === parentSessionFile) {
    parentStatCalls += 1;
    return await new Promise(() => {});
  }
  return await originalStat.call(fs, filePath, ...args);
};

const started = Date.now();
try {
  const result = await forkSessionEntryFromParent({
    agentId: "main",
    decisionSkipPatch: () => ({ forkedFromParent: false }),
    fallbackEntry: { sessionId: "", updatedAt: now },
    parentSessionKey,
    sessionKey,
    storePath,
  });
  const elapsedMs = Date.now() - started;
  const allFiles = await fs.readdir(activeStoreDir);
  console.log(JSON.stringify({
    elapsedMs,
    parentStatCalls,
    resultStatus: result.status,
    decisionReason: result.status === "skipped" ? result.decision?.reason : null,
    resultSessionEntry: result.status === "skipped" ? result.sessionEntry : null,
    activeStoreFiles: allFiles.sort(),
    createdForkTranscriptCount: allFiles.filter((name) => name.endsWith(".jsonl") && name !== "parent.jsonl").length,
  }, null, 2));
} finally {
  fs.stat = originalStat;
  await fs.rm(root, { recursive: true, force: true });
}
'
{
  "elapsedMs": 1032,
  "parentStatCalls": 1,
  "resultStatus": "skipped",
  "decisionReason": "parent-size-timeout",
  "resultSessionEntry": {
    "sessionId": "",
    "updatedAt": 1783476977699,
    "forkedFromParent": false
  },
  "activeStoreFiles": [
    "parent.jsonl",
    "sessions.json"
  ],
  "createdForkTranscriptCount": 0
}
node --import tsx --no-warnings --input-type=module -e '
process.env.OPENCLAW_HOME = "/tmp/openclaw-101718-proof-home";
process.env.OPENCLAW_STATE_DIR = "/tmp/openclaw-101718-proof-state";
const { resolveParentForkDecision } = await import("./src/auto-reply/reply/session-fork.js");
const decision = await resolveParentForkDecision({
  parentEntry: {
    sessionId: "parent-session",
    updatedAt: Date.now(),
    totalTokens: 170_000,
    totalTokensFresh: true,
  },
  storePath: "/tmp/openclaw-sessions.json",
});
console.log(JSON.stringify({
  status: decision.status,
  reason: decision.status === "skip" ? decision.reason : null,
  maxTokens: decision.maxTokens,
  parentTokens: "parentTokens" in decision ? decision.parentTokens : null,
}, null, 2));
'
{
  "status": "skip",
  "reason": "parent-too-large",
  "maxTokens": 100000,
  "parentTokens": 170000
}

Observed result after the fix: The stalled parent token-probe caller path returned skipped with parent-size-timeout after roughly one second, kept the original parent.jsonl, and created zero fork transcripts. Fresh oversized tokens still returned parent-too-large.
Not tested: A live gateway session creation with a real remote filesystem or transcript handle blocked indefinitely was not exercised locally.

@clawsweeper

clawsweeper Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 7, 2026, 10:43 PM ET / 02:43 UTC.

Summary
The branch bounds parent-fork token probing, uses fresh persisted token counts as a fast path, adds skip decisions for timed-out or unavailable parent sizing, updates gateway/reply callers, and adds regression coverage.

PR surface: Source +88, Tests +122. Total +210 across 5 files.

Reproducibility: yes. Source inspection on current main shows resolveParentForkDecision awaits resolveParentForkTokenCount without a timeout, and the PR body includes after-fix terminal proof for a never-resolving parent stat path.

Review metrics: 1 noteworthy metric.

  • Parent-fork decision variants: 2 skip reasons added. The internal decision union now distinguishes too-large, timed-out, and unavailable parent sizing, so each fork caller must intentionally handle non-too-large skips.

Stored data model
Persistent data-model change detected: serialized state: src/auto-reply/reply/session-fork.test.ts, serialized state: src/auto-reply/reply/session-fork.ts, serialized state: src/config/sessions/session-accessor.ts, serialized state: src/gateway/session-create-service.ts, unknown-data-model-change: src/auto-reply/reply/session-fork.test.ts, unknown-data-model-change: src/auto-reply/reply/session-fork.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #101718
Summary: This PR is a candidate fix for the canonical parent-fork token-probe timeout issue; two sibling PRs target the same root problem, while the accessor refactor is adjacent infrastructure.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Get maintainer acceptance of fail-closed handling for timed-out or unavailable parent sizing before merge.

Risk before merge

  • [P1] Timed-out or unavailable parent sizing now fails closed: reply/subagent paths can start isolated and explicit gateway forks can return INVALID_REQUEST, so existing unknown-size parent forks may stop inheriting context.
  • [P1] The token probe is raced rather than cancelled, so a truly hung filesystem operation may continue in the background even after the caller is unblocked.
  • [P1] The PR body proves the production fork caller path with a simulated stuck stat, but not a full live gateway sessions.create flow on a slow or hung real filesystem.

Maintainer options:

  1. Accept fail-closed parent sizing (recommended)
    Maintainers can accept that timed-out or unavailable sizing starts isolated, or rejects explicit gateway forks, because inheriting unverified context is the state and availability risk this PR avoids.
  2. Preserve unknown-size forks
    Ask for a narrower rework that keeps unknown-size inheritance possible while bounding the transcript-copy path so session creation still cannot stall indefinitely.
  3. Pause for live gateway proof
    Ask for a sessions.create or equivalent full-caller proof run if owner confidence depends on seeing the gateway behavior outside the focused terminal harness.

Next step before merge

  • No automated repair is needed; the remaining blocker is maintainer acceptance of the fail-closed unknown-size fork semantics before merge.

Maintainer decision needed

  • Question: Should parent forks with timed-out or unavailable size probes start isolated, or reject explicit gateway forks, instead of attempting a bounded transcript copy?
  • Rationale: The code fixes the availability stall by changing unknown-size parent-fork policy, which is safer for bounded execution but can change context inheritance for existing users whose parent size cannot be verified.
  • Likely owner: jalehman — jalehman most recently reorganized the parent-fork accessor and storage boundary that this timeout decision now crosses.
  • Options:
    • Accept fail-closed sizing (recommended): Merge this shape after owner review, treating unverified parent size as unsafe to inherit and relying on the timeout tests plus terminal proof.
    • Preserve unknown-size inheritance: Require a follow-up patch that bounds or avoids the later transcript-copy stall while still allowing unknown-size parent forks to inherit context when possible.
    • Require live gateway proof: Hold merge until the contributor adds redacted sessions.create or equivalent gateway proof for the explicit fork behavior after timeout.

Security
Cleared: The diff changes session-fork logic, an internal decision type, gateway/reply callers, and tests; it does not change dependencies, workflows, secrets, package resolution, or other supply-chain surfaces.

Review details

Best possible solution:

Land one canonical timeout fix that bounds parent sizing, preserves fresh and oversized behavior, and explicitly accepts fail-closed handling for unverified parent size.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection on current main shows resolveParentForkDecision awaits resolveParentForkTokenCount without a timeout, and the PR body includes after-fix terminal proof for a never-resolving parent stat path.

Is this the best way to solve the issue?

Yes, with a policy caveat. The latest patch fixes the prior timeout-to-fork defect by making unresolved sizing skip inheritance, but maintainers still need to accept that fail-closed behavior versus preserving unknown-size transcript inheritance.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 6765eb0166fc.

Label changes

Label changes:

  • add merge-risk: 🚨 compatibility: Unknown-size parent forks that previously attempted inheritance can now start isolated or fail explicit gateway fork creation.
  • add proof: sufficient: Contributor real behavior proof is sufficient. Sufficient terminal proof: the PR body shows the patched production fork caller returning skipped/parent-size-timeout after about one second with zero fork transcripts created; full live gateway slow-filesystem proof was not included.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): Sufficient terminal proof: the PR body shows the patched production fork caller returning skipped/parent-size-timeout after about one second with zero fork transcripts created; full live gateway slow-filesystem proof was not included.
  • remove rating: 🧂 unranked krab: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.
  • remove status: 📣 needs proof: Current PR status label is status: 👀 ready for maintainer look.

Label justifications:

  • P2: The PR targets a normal-priority session-fork availability and session-state bug with limited blast radius and no active outage evidence.
  • merge-risk: 🚨 compatibility: Unknown-size parent forks that previously attempted inheritance can now start isolated or fail explicit gateway fork creation.
  • merge-risk: 🚨 session-state: The patch changes when child sessions inherit parent transcript state versus starting isolated after unverified parent sizing.
  • merge-risk: 🚨 availability: The fix intentionally trades exact parent sizing and inheritance for bounded caller availability when token probing stalls.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): Sufficient terminal proof: the PR body shows the patched production fork caller returning skipped/parent-size-timeout after about one second with zero fork transcripts created; full live gateway slow-filesystem proof was not included.
  • proof: sufficient: Contributor real behavior proof is sufficient. Sufficient terminal proof: the PR body shows the patched production fork caller returning skipped/parent-size-timeout after about one second with zero fork transcripts created; full live gateway slow-filesystem proof was not included.
Evidence reviewed

PR surface:

Source +88, Tests +122. Total +210 across 5 files.

View PR surface stats
Area Files Added Removed Net
Source 4 99 11 +88
Tests 1 123 1 +122
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 5 222 12 +210

What I checked:

  • Root policy read: Read the full root policy and applied its PR review guidance for whole decision surfaces, session-state merge risk, real behavior proof, and likely-owner history. (AGENTS.md:17, 6765eb0166fc)
  • Scoped gateway policy read: Read the gateway scoped policy because the PR changes gateway session-create fork behavior. (src/gateway/AGENTS.md:1, 6765eb0166fc)
  • Current main still has the unbounded await: Current main awaits parent-token resolution before returning a fork/skip decision, so a stuck token probe can still block callers. (src/auto-reply/reply/session-fork.ts:107, 6765eb0166fc)
  • Filesystem callee is byte-bounded but not time-bounded: The async transcript-tail reader awaits file open/read with byte and line bounds but no timeout or abort parameter at this call site. (src/gateway/session-utils.fs.ts:290, 6765eb0166fc)
  • PR head bounds the decision path: At PR head, resolveParentForkDecision uses fresh tokens immediately, races stale token probing against a 1s timeout, and returns skip for timed-out or unavailable sizing. (src/auto-reply/reply/session-fork.ts:123, 36e6ece642f0)
  • Gateway caller handles expanded skip reasons: The explicit gateway fork path now formats non-too-large skip reasons with the decision message instead of assuming parentTokens exists. (src/gateway/session-create-service.ts:439, 36e6ece642f0)

Likely related people:

  • jalehman: PR metadata and git history show jalehman authored the recently merged accessor-boundary refactor that reorganized the parent-fork storage boundary and touched the same session-fork, accessor, gateway, and subagent paths. (role: recent adjacent refactor owner; confidence: high; commits: a68caa185bf1, 75f2d08c4a29, 003c51975c77; files: src/auto-reply/reply/session-fork.ts, src/config/sessions/session-accessor.ts, src/gateway/session-create-service.ts)
  • steipete: PR metadata and git show tie the current session-controls/operator-fork feature that introduced this decision surface to steipete's merged PR and merge commit. (role: prior feature author and recent area contributor; confidence: high; commits: 88f1ec38d4a5, e0852f05e320; files: src/auto-reply/reply/session-fork.ts, src/auto-reply/reply/session-fork.runtime.ts, src/gateway/session-create-service.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (6 earlier review cycles)
  • reviewed 2026-07-08T00:56:31.300Z sha afe5e5c :: needs real behavior proof before merge. :: [P1] Avoid forking after a timed-out size probe
  • reviewed 2026-07-08T01:06:22.384Z sha 424cbc0 :: needs real behavior proof before merge. :: [P1] Avoid forking after a timed-out size probe
  • reviewed 2026-07-08T01:15:34.482Z sha 424cbc0 :: needs real behavior proof before merge. :: [P1] Use a timer handle type that compiles | [P1] Avoid forking after a timed-out size probe
  • reviewed 2026-07-08T01:44:16.188Z sha 7b1f6a6 :: needs real behavior proof before merge. :: [P1] Avoid forking after token-probe timeout
  • reviewed 2026-07-08T01:52:08.277Z sha 7b1f6a6 :: needs real behavior proof before merge. :: [P1] Avoid forking after token-probe timeout
  • reviewed 2026-07-08T02:30:33.364Z sha 36e6ece :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. labels Jul 8, 2026
@LiLan0125
LiLan0125 force-pushed the fix/101718-session-fork-token-timeout branch from afe5e5c to 424cbc0 Compare July 8, 2026 01:00
@LiLan0125
LiLan0125 force-pushed the fix/101718-session-fork-token-timeout branch from 424cbc0 to 7b1f6a6 Compare July 8, 2026 01:36
@openclaw-barnacle openclaw-barnacle Bot added gateway Gateway runtime size: M and removed size: S labels Jul 8, 2026
@LiLan0125

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Fixed the P1 timeout-to-fork finding:

  • Timeout or unavailable parent sizing now returns a skip decision (parent-size-timeout / parent-size-unavailable) so callers do not enter parent transcript copy after the probe timeout.
  • Gateway and reply-session prepare paths now format non-token skip reasons without assuming parentTokens.
  • Added caller-path regression coverage for a stalled parent size probe.

Verified:

  • node scripts/run-vitest.mjs src/auto-reply/reply/session-fork.test.ts passed.
  • pnpm tsgo:prod passed.
  • pnpm check:test-types passed.
  • GitHub checks Real behavior proof, check-prod-types, check-test-types, and check-lint passed on the new head.

Updated the PR proof with forkSessionEntryFromParent caller-path output showing skipped / parent-size-timeout and zero fork transcripts created.

@clawsweeper

clawsweeper Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 8, 2026
@knoal

knoal commented Jul 13, 2026

Copy link
Copy Markdown

A/B verified locally against base SHA. Ran vitest with the PR's test file against the PR's fix (B) and against the base's fix (A).

Result: CLEAN A/B

A (PR test + base fix): FAIL - 1 test failed
B (PR test + PR fix):   PASS

The new test starts isolated when parent token probing stalls fails on base (the isolation is skipped when probing stalls), passes with the fix. Failing-first signature is correct. Looks ready to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gateway Gateway runtime merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: M status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

session-fork parent token count resolution has no timeout and blocks session creation indefinitely

2 participants