Location invalidation duplicates a directory's fff engine; on v2 releases the old engine is never destroyed #54631

Activity

1. opencode-agent commented on Oct 11, 2026

   

   opencode-agentBot

   on Oct 11, 2026 – with OpenCode Agent

   Contributor

   More actions

   Thanks for the detailed report. I reproduced this with a simulated model on Linux. The project was a git repo with 40 directories. In each round I started a session step, ran /reload while it was still running, let the step finish, then counted inotify watches in the server process.

   2.0.26: each round adds one inotify instance and one directory's worth of watches, and they are never released. Over three rounds the server went from 3 to 6 inotify instances and from 95 to 350 watches, while distinct inodes stayed at 90. This matches your case 2.

   Latest dev (707de58): the old engine is now destroyed once the step finishes, so the permanent leak is gone there. While the session is still running after /reload, though, the same directory still has two fff engines (4 instances and 180 watches, back to 3 and 95 afterwards). That is your case 1, and it still happens on dev. I didn't test the path-spelling case.

   The script below reproduces it and prints the inotify counts after each step.

   Related:

   • fix(core): share fff engine per directory #52964: open PR that shares one fff engine per directory, which should remove the temporary duplicate engine (case 1)
   • Multiple fff engines per process watch the same base path, multiplying inotify watches 2-3x #50966: the original report, closed as a v1 issue
   • watcher.ignore never reaches the embedded fff engine — per-dir inotify watches on every indexed dir (192K watches / 4.5GB RSS on a large root); root .ignore file is the only effective channel #37740: earlier fff/inotify watch-count issue

   Reproduction script (drive.ts)

   // Reproduces #54631: /reload during a running session leaves a second fff engine
   // (extra inotify instance + watches) for the same directory.
   import { Effect } from "effect"
   import { Llm, OpenCodeDriver } from "opencode-drive"
   import * as fs from "node:fs"
   
   const files: Record<string, string> = {}
   for (let i = 0; i < 40; i++) files[`dir${i}/sub/f.ts`] = `export const v${i} = ${i}\n`
   
   // Sum inotify watches / distinct inodes / inotify fds for every process on the box.
   const measure = (label: string) =>
     Effect.sync(() => {
       const rows: string[] = []
       for (const pid of fs.readdirSync("/proc").filter((p) => /^\d+$/.test(p))) {
         let fds: string[] = []
         try { fds = fs.readdirSync(`/proc/${pid}/fd`) } catch { continue }
         let instances = 0, watches = 0
         const inodes = new Set<string>()
         for (const fd of fds) {
           let link = ""
           try { link = fs.readlinkSync(`/proc/${pid}/fd/${fd}`) } catch { continue }
           if (!link.includes("inotify")) continue
           instances++
           let info = ""
           try { info = fs.readFileSync(`/proc/${pid}/fdinfo/${fd}`, "utf8") } catch { continue }
           for (const line of info.split("\n")) {
             if (!line.startsWith("inotify wd:")) continue
             watches++
             const m = line.match(/ino:(\S+)/)
             if (m) inodes.add(m[1])
           }
         }
         if (watches > 0) {
           let cmd = ""
           try { cmd = fs.readFileSync(`/proc/${pid}/cmdline`, "utf8").replace(/\0/g, " ").slice(0, 80) } catch {}
           rows.push(`pid=${pid} inotify_fds=${instances} watches=${watches} distinct_inodes=${inodes.size} cmd=${cmd}`)
         }
       }
       console.log(`=== ${label}\n${rows.join("\n")}`)
     })
   
   export default OpenCodeDriver.use(
     {
       ...(process.env.OC_DEV ? { opencode: { dev: process.env.OC_DEV } } : {}),
       project: { git: true, files },
       tools: ["shell"],
       tui: { recording: true },
     },
     ({ ui, llm, tools }) =>
       Effect.gen(function* () {
         yield* llm.queue(Llm.text("Warm up done."))
         yield* ui.submit("warm up")
         yield* ui.waitFor("Warm up done.")
         yield* Effect.sleep(5000)
         yield* measure("baseline")
         const shells = yield* tools.control("shell")
         for (const n of [1, 2, 3]) {
           yield* llm.queue(
             Llm.toolCall({ index: 0, id: `call_${n}`, name: "shell", input: { command: "sleep 1", description: "wait" } }),
             Llm.finish("tool-calls"),
           )
           yield* llm.queue(Llm.text(`Round ${n} done.`))
           yield* ui.submit(`run round ${n}`)
           const call = yield* shells.take(`call_${n}`)
           // The session is mid-step: reload every location while it runs.
           yield* ui.submit("/reload")
           yield* Effect.sleep(3000)
           yield* measure(`round ${n}: after /reload, session still running`)
           yield* call.succeed({ output: "ok\n", exit: 0 })
           yield* ui.waitFor(`Round ${n} done.`)
           yield* Effect.sleep(5000)
           yield* measure(`after round ${n} (reload during running session)`)
         }
       }),
   )

   Run it with opencode-drive: opencode-drive run ./drive.ts. It drives opencode from your PATH; set OC_DEV to the path of an OpenCode checkout to run that from source instead.
2. opencode-agent

   added

   reproduced

   on Oct 11, 2026
3. nshah1d commented on Oct 15, 2026

   

   nshah1d

   on Oct 15, 2026

   More actions

   Additional Linux observations for the original report, #50966. These measurements are from OpenCode 1.18.31, not a reproduction of the v2 lifecycle bug.

   The affected process was an interactive CLI session running for about six hours on Linux 7.0.0-38-generic, x86-64, with 16 logical CPUs. Its working directory was a non-repository development root containing several applications, scratch work, backups and an older Go toolchain.

   Two FFF watcher-owner groups were active. Comparing device/inode pairs from their inotify descriptors gave:

   FD 30:                 12,331 watches
   FD 33:                 12,316 watches
   Shared between both:   12,313 objects
   Only FD 30:                18
   Only FD 33:                 3
   

   Both sets covered the older toolchain, build cache and vendor directories while the session was working on a different project. The second owner group had appeared roughly two hours after the first; its creation trigger was not established.

   The owner threads each consumed roughly 85 to 90% CPU. A separate three-second whole-process sample measured 597.9% CPU. Per-thread I/O sampling recorded approximately 2.068 and 1.955 GB/s of read accounting, with zero additional physical read bytes charged during that interval. Those figures describe cached/kernel-served reads, not 4 GB/s of disk throughput.

   Content indexing was already disabled: the 1.18.31 search implementation sets both disableContentIndexing and disableMmapCache, and that option pair was present in the running binary. It bundles FFF 0.9.4.

   The pinned dependency still has a binary classifier that reads through to EOF. Post-scan directory registration and watcher processing provide a candidate path for repeated classification. No native stack profile was captured, so this is a source-backed explanation to investigate, not an attribution of all process CPU to that function or proof of a memory leak.

   A separate clean-process test on 1.19.0 with OPENCODE_DISABLE_FFF=1 passed file lookup, content search and direct reading with zero FFF threads. The affected 1.18.31 session was left running during the investigation; there was no controlled before/after comparison on that process.

   Sharing engine ownership should remove the duplicate coverage. It would also be useful to check whether post-rescan registration can repeatedly trigger full-file classification despite content indexing being disabled.