Skip to content

[Bug]: A remote delegated task finishes with the child's first turn, and the answer from its background-wake turn never reaches the parent #28114

Closed as not planned

Description

@TimFux11

Before submitting

Area

apps/server

Steps to reproduce

You need two linked environments: A (origin) and B (destination). B runs the Claude provider; any model works (I used claude-haiku-5-5).

  1. From a thread on A, call delegate_task with target.environmentId = B, mode: "async", and this task:

    Throwaway probe, change nothing. (1) Run the Bash command sleep 25; echo OK with run_in_background: true. (2) End your turn right away with the single word WAITING. Do not wait, poll or sleep yourself. (3) When the background command's finished notice wakes you, reply with the single word FINAL.

  2. End the parent's turn and wait a few minutes.

  3. Repeat, with two changes to the task. Replace step (1) with "start a Monitor with command sleep 25; echo OK, non-persistent, timeout 120000 ms". Replace step (3) with "When the monitor's finished notice wakes you, reply FINAL".

  4. Controls: repeat steps 1 and 3 without target.environmentId, so the child is local.

Expected behavior

The parent gets the child's final answer, FINAL, whichever machine the child runs on. It can come as the task's result, or as a follow-up task with its own completion wake. Since #23542, a local child's background-wake turn is delivered the second way.

Actual behavior

On B, the child thread does exactly what it was told:

  • run 1 replies WAITING;
  • the background command (or Monitor) finishes, and T3 starts run 2 with a Command "…" finished (or Monitor "…" finished) notification;
  • run 2 replies FINAL.

On A:

  • While the background command runs, task_status reports status: running, workState: working. That part is right.
  • The task then completes with summary: "WAITING". That is run 1's text.
  • Run 2's FINAL is never delivered.
    • No follow-up task is created, and no second completion wake reaches the parent.
    • A task_status read taken after run 2 on B had posted FINAL still returned WAITING.
    • The parent thread never received a later wake from either remote probe.
Child Waits with Task result on A Child's run 2 (FINAL)
Remote background command WAITING posted on B, never delivered
Remote Monitor WAITING posted on B, never delivered
Local background command WAITING delivered as a separate provider-continuation follow-up task, with its own "Delegated task … finished" wake
Local Monitor WAITING posted in the child thread, never delivered

Every machine should behave like the local background-command row. In the local Monitor case, the task completed with WAITING about 0.1 s before the Monitor's wake run was requested. Both timestamps come from the same server, and no second result followed.

Here is the remote background-command probe on each machine, in seconds from delegate_task. The two machines' clocks aren't synchronised to the millisecond.

B  t+0.2   run 1 starts
B  t+14.4  background command started
B  t+16.0  run 1 replies WAITING; run 1 completes at t+16.3
A  t+25.7  task_status -> running / working
B  t+40.2  "Command … finished" notification; run 2 requested
B  t+42.4  run 2 replies FINAL; run 2 completes at t+42.8
A  t+42.8  task_status -> completed / result_available, summary "WAITING"
A  later   no follow-up task, no second wake

Cause

From reading the source at cadf7d5fe3. None of the code cited here changed between that commit and 0.1.6.

  1. A remote child gets no follow-up.
  2. Why the follower takes run 1 (inferred).
    • followOnce waits with untilDelegatedWorkSettled: true.
    • On B, delegatedTaskProgress (packages/provider-core/src/server/subagentProjection.ts) returns result_available when no work run is active and pendingBackgroundTasks is empty. resultRun is then the latest finished run.
    • It looks as if the finished command leaves pendingBackgroundTasks before its wake run is created. For a moment, then, the thread reads as settled on run 1. I haven't confirmed this ordering in the Claude adapter.
    • What I did see once: a t3_thread_wait with untilDelegatedWorkSettled on the remote child's thread returned resultRun = run 1, and run 2 then produced the real answer.
  3. Runs that a Monitor wakes are excluded.
    • delegatedTaskProgress drops every run whose triggering message has notification.source.kind === "monitor" (monitorRuns).
    • fix(server): notify parents after delegated background runs #23542 treats monitor updates the same way, because they "report progress rather than asking for new work". That fits PR-watch and output-progress notices.
    • But when a Claude Monitor finishes, that is the agent's chosen way of waiting for its own work, and the run it wakes holds the answer.
    • As written, that run can never be a task's result or start a follow-up. This applies to local children as well as remote ones; see the local Monitor row above.

Impact

Major degradation or frequent failure. Any remote child whose work outlasts one turn loses its result. For us that's every long test run delegated to a second machine: the agent's foreground command limit ends the first turn while the suite is still running in the background. The parent then either acts on "still running…" as if it were the answer, or waits for a wake that never comes. It hit two real runs in one morning. One result went unseen for about 45 minutes and the other for about 80, until someone noticed by hand.

Version or commit

Reproduced on 0.1.5 (6d450a4284) on both servers. Both servers now run 0.1.6 (2359643cff), where the code paths above are unchanged. I haven't re-run the probes on 0.1.6.

Environment

Origin: Linux x64 server. Destination: Windows x64 desktop. Both children run Claude Code; the probes used claude-haiku-5-5. Seen with both background Bash commands and the Monitor tool.

What doesn't work

  • The child marks the work as awaited. One remote child called t3_background_wait on its background task before ending run 1. The outcome was the same: the task completed with WAITING, and run 2's FINAL was never delivered. This was observed once.
  • The parent waits on the thread instead. t3_thread_wait with untilDelegatedWorkSettled returned run 1. This was also observed once; see Cause, point 2.
  • The parent sends the child a follow-up to collect the result. A t3_thread_send to a remote child creates no task, so the reply never wakes the parent. That is Resuming a completed remote delegate does not create a follow-up task or notify its parent #26740.

Workaround

The remote child reports back by itself:

  • The parent gives the child its own thread ID and environment ID.
  • In the turn that has the real answer, the child's last action is a t3_thread_send (mode queue) to the parent, carrying the result line.
  • The parent treats a "Delegated task … finished" wake from a remote child as final only if it contains that result line. Otherwise it ends its turn and waits for the child's message.
  • A periodic check reads the child's thread with t3_thread_read, never task_status, to recover any result that still slips through.

Suggested fix

I'm happy to file a PR for this if you approve the direction.

Related: #26740, #26776, #23542, #16603, #22651

Filed by Claude Opus 5.5 via Claude Code.

Activity

  1. royalaid commented on Nov 7, 2026

    @royalaid

    A note on coverage, in case it helps before #31744 closes this: #31744 fixes the background-command case reported here (it follows the child's thread after the first result, and a run started by the child's own background wake counts as a follow-up), but it skips runs that a Monitor or scheduled task started (Orchestrator.ts:714 at its head 585db18, and its test "waitForThread with afterRunId skips runs a monitor or scheduled task started"). The finished-Monitor reproduction in this issue would still lose its final reply, so this issue may need to stay open for that case after #31744 lands.

    Evidence from our side for the background-command case: a linked child answered "waiting on the test suites", then woke itself a minute later when its background suites ended and wrote the final verdict; the parent got the interim answer and never the verdict. We patched it the same way (an afterRunId wait after the result); our tests for a late reply, a restart during the wait, and the end of the wait window fail on main and pass with that change. We have not tested a Monitor-started run.

  2. nuuls commented on Nov 11, 2026

    @nuuls

    Still happens with the current explicit-wait model, where the child marks the command awaited. This is on a fork build (0.1.9-fork.20261110) branched from main at 6c9adfb. The fork's only changes are to Codex usage limits, so the code paths involved here match main.

    Setup: Linux parent server, Windows child server, Claude Opus 5.5 parent and child, async delegate_task with target.environmentId. The child started a ~7 minute background Bash command and called t3_background_wait on it (matched). It then ended run 1 with "…I'll pick up once that job finishes."

    UTC times. The two machines' clocks aren't synced to the millisecond.

    Machine Time Event
    Child 15:08:34.66 Run 1 completes. The awaited command keeps the task open.
    Parent 15:09:46.63 Task completes with run 1's text. task_status returns completed / result_available.
    Parent 15:09:46.73 Parent woken with that text.
    Child 15:09:47.65 Command "…" finished notification arrives and run 2 is requested.

    So the awaited hold works until the command exits. Then the follower's untilDelegatedWorkSettled wait settles on run 1, about a second before the wake run exists.

    This confirms the ordering this issue infers under "Why the follower takes run 1". In ClaudeAdapterV2, the task_notification branch that runs while no turn is active calls applyBackgroundTaskRosterMessage, which drops the task and publishes the empty roster right away. bufferWakeMessage offers the wake continuation only once a root init, user, assistant or result frame arrives.

    Two open PRs each fix part of this, on different machines:

  3. juliusmarminge commented on Nov 12, 2026

    @juliusmarminge
    Member

    I tried to reproduce this on the current build and could not.

    Commit a18a458 (origin/main). A live two-environment run with Claude Code on the child side was not possible: the VM has no first-party Claude provider. I reproduced the parent-side defect in apps/server/src/peer/RemoteDelegation.test.ts, using its existing fake linked environment ('box'). The test delegates an async task there, the box run ends with reply 'WAITING', and the parent's task completes with 'WAITING' (that assertion passes; a temporary log confirmed it). Then a later run on the same box thread replies 'FINAL' (laterReply plus laterFinished), and the test waits for any parent task carrying 'FINAL'. On main nothing ever follows the box thread after the first result, so the test hangs until the 120 s project timeout (vp test run src/peer/RemoteDelegation.test.ts -t "background wake starts"). Code reading agrees: RemoteDelegation.followOnce calls complete(..., 'ran-out') once and the follower ends. The fix PRs #31744 (parent side, keeps following the thread) and #32223 (child side, keeps awaited Claude commands pending until their wake starts) are both still open; #26776, merged 2026-11-11, covers only follow-ups the parent sends.

    Closing as not planned. If it still happens for you, reply with the exact steps and the version you are on and it will be looked at again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions