Skip to content

opencode subprocess cleanup closes stdout before process exits, can orphan child and trigger EPIPE spin #4533

Closed

Description

@MSandro

Summary

server/pkg/agent/opencode.go closes the read end of the child stdout pipe as soon as the run context is cancelled, without first ensuring the opencode process has exited. This can leave the child orphaned and, depending on how OpenCode handles a closed stdout, cause it to spin at 100% CPU on EPIPE.

Code location

https://github.com/multica-ai/multica/blob/main/server/pkg/agent/opencode.go#L125-L128

// Close stdout when the context is cancelled so the scanner unblocks.
go func() {
    <-runCtx.Done()
    _ = stdout.Close()
}()

Problem

  1. A task times out or the daemon restarts, cancelling runCtx.
  2. Multica closes stdout to unblock its scanner loop.
  3. exec.CommandContext will eventually send a signal to the child, but there is a window where opencode is still alive with a closed stdout.
  4. If opencode tries to log or emit further output during that window, every write returns EPIPE.
  5. If the daemon itself exits/restarts without explicitly terminating children, the opencode process can be left orphaned and continue spinning.

Impact

  • Orphaned opencode processes consuming 100% CPU.
  • Degraded agent reliability when tasks are cancelled or the daemon restarts.

Suggested fixes

Option A — wait/kill before closing stdout:

Before closing stdout, ensure the command has exited. If it does not exit promptly after context cancellation, send SIGTERM/SIGKILL:

go func() {
    <-runCtx.Done()
    if cmd.Process != nil {
        _ = cmd.Process.Signal(syscall.SIGTERM)
        // give it a grace period, then SIGKILL
        time.AfterFunc(5*time.Second, func() { _ = cmd.Process.Kill() })
    }
    _ = stdout.Close()
}()

Option B — process-group / daemon shutdown tracking:

Track all spawned subprocesses and terminate them (or their whole process group) during daemon shutdown or restart before releasing pipe file descriptors.

Related

Activity

  1. Bohan-J commented on Jun 24, 2026

    @Bohan-J
    Collaborator

    Thanks for the precise report and the two-option breakdown — both the code location and the failure chain were spot on. Confirmed on main: cancellation closes the stdout read end while opencode is still alive, and exec.CommandContext only SIGKILLs the direct child, so a tool subprocess it spawned can be orphaned and spin on EPIPE.

    I've opened #4541. It's essentially your Option A made group-aware (Option B): on cancel/timeout we now run opencode in its own process group and drive a graceful group-wide SIGTERM → grace → SIGKILL, closing the stdout pipe only after the tree has been signalled. Since SIGKILL is uncatchable, once it's delivered no group member can write again, so the EPIPE window is gone; the group signal also reaps the descendants instead of orphaning them. The manual stdout.Close() stays as a last-resort unblock (it guards against a scanner deadlock when a descendant keeps the pipe open), with WaitDelay as the final backstop.

    Added unix tests for both the graceful path and the worst case — a fake that ignores SIGTERM and keeps writing stdout — asserting the whole process group is reaped and the run never hangs.

    On #3345: this should fix the cases where the idle spike followed a cancelled task or a daemon restart. If anyone still sees high idle CPU with no recent agent activity, that's likely a separate issue and a process sample + daemon.log would help pin it down. Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions