fff over-indexes Jujutsu (jj) workspaces — .gitignore silently dropped when no colocated .git, causing multi-GB indexing & sustained high CPU #35791

Summary

opencode's fff file indexer content-indexes everything a repo's .gitignore would normally exclude (.venv, node_modules, __pycache__, build output, *.db) whenever it operates on a Jujutsu working copy that has no colocated .git. This produces multi-GB indexing, sustained high CPU, and fan spin-up. It is especially bad for jj workspace add workspaces, which by design have a .jj pointer but no .git.

Root cause

fff (@ff-labs/fff-bun, upstream dmtrKovalenko/fff) walks the tree via the Rust ignore crate, whose require_git defaults to true — .gitignore rules are only applied when a .git directory marks the repo boundary. A jj-only working copy has .jj but no .git, so all .gitignore files are silently dropped and the whole tree is walked and content-indexed.

Note: the ignore crate's require_git=true is intentional and documented (it exposes require_git(false)), so this is not an ignore-crate or ripgrep bug — the fix belongs in how fff/opencode invoke the walk.

This is invisible to users who are using version control correctly: the main checkout may be a colocated jj+git repo, but the working copy opencode actually attaches to (a jj workspace add workspace) is jj-only.

Minimal reproduction

Demonstrates the crate behavior directly (no jj required):

T=$(mktemp -d); cd "$T"
printf "node_modules/\n.venv/\nbuild/\n*.log\n" > .gitignore
mkdir -p node_modules .venv build src
touch node_modules/dep.js .venv/lib.py build/out.o app.log src/main.py keep.txt

rg --files          # -> app.log, build/out.o, keep.txt, node_modules/dep.js, src/main.py   (.gitignore NOT applied)
git init -q .
rg --files          # -> keep.txt, src/main.py                                               (.gitignore applied)

The first rg --files mirrors what fff sees in a jj workspace (no .git): ignores dropped.

With actual jj:

jj git init --colocate main && cd main   # main copy: has .git + .jj
jj workspace add ../ws                    # ws: has .jj, NO .git   <- what opencode attaches to

Real-world impact (observed)

• An editspace containing 62 jj workspace add workspaces (each .jj + .gitignore, 0 with .git), attached to an always-on opencode serve daemon.
• sample on the daemon showed ~1000%+ CPU / 3+ GB RSS, with essentially 100% of on-CPU samples in:
  • fff_search::bigram_filter::BigramIndexBuilder::add_file_content
  • fff_search::types::FileItem::detect_binary_per_byte
  • fff_search::scan::ScanJob::run
• Content that .gitignore excludes but fff indexed anyway across those workspaces: .venv ~= 610 MB, node_modules ~= 179 MB, plus __pycache__, .ruff_cache, *.db, build dirs.

Does this affect the ripgrep fallback (OPENCODE_DISABLE_FFF=1)?

Partially, but far less severely — worth noting so the fix is scoped correctly:

• ripgrep shares the require_git behavior, so in a jj-only tree it also misses non-hidden gitignored paths (node_modules/, build/, __pycache__/).
• But ripgrep skips hidden dirs by default (.venv, .ruff_cache, .jj), and runs on-demand rather than as a persistent watched index. So it over-traverses per query but does not produce fff's sustained multi-core burn. fff's additional problems (no hidden-dir skip; proactive full-content index; continuous fs watcher) are fff-specific.

Suggested fixes (in fff / opencode's invocation — any of)

1. Treat .jj as a repository boundary for ignore processing (honor the working copy's .gitignore when .jj is present).
2. Configure the ignore walk with require_git(false) so .gitignore files are honored regardless of .git presence.
3. Skip .jj/ directories, and skip hidden dirs by default like ripgrep does.
4. Expose a project-level ignore-roots config knob so users can bound indexing.

User-facing workaround

Both fff and ripgrep honor .ignore / .rgignore files without requiring .git. Dropping a .ignore at the project root listing node_modules/, build/, scratch/, __pycache__/, etc. bounds indexing in jj-only trees today, without needing colocated git.

Environment

• opencode 1.17.10 (Homebrew), macOS ARM64
• Jujutsu workspaces created via jj workspace add (jj-only, no colocated git)

Root cause / upstream

• Manifests in: anomalyco/opencode (filing here)
• True root cause: @ff-labs/fff-bun -> dmtrKovalenko/fff
• Not an ignore-crate or ripgrep bug (require_git=true is intentional/configurable)

Related

• Jujutsu support? #5008 (general "Jujutsu support?" — closed stale; does not cover indexing/ignore behavior)
• opencode serve pegs CPU at ~1000% — fff content-indexes entire $HOME (search.ts enableHomeDirScanning/FsRootScanning) #32511 (fff enableHomeDirScanning full-home scan)