Skip to content

feat(config): add ignore_worktrees to skip linked git worktrees - #1430

Merged
DeusData merged 25 commits into
DeusData:mainfrom
vitalNohj:fm/cbm-worktree-ignore-1
Nov 2, 2026
Merged

DeusData merged 25 commits into
DeusData:mainfrom
vitalNohj:fm/cbm-worktree-ignore-1

Conversation

@vitalNohj

Copy link
Copy Markdown
Contributor

Problem

Agent workflows increasingly use git worktree add to run several branches of the same repo side by side. Every worktree is discovered as its own project, so the cache accumulates a near-duplicate graph per worktree and project pickers get noisy. Today there is no way to say "index the main checkout, leave the worktrees alone".

Change

Adds an opt-in ignore_worktrees config key. When enabled, a linked worktree is left alone across every indexing entry point:

  • auto-index on connect skips it (logged as autoindex.skip with reason=linked_worktree)
  • the daemon's background auto-index skips it
  • explicit index_repository refuses with an actionable message naming both ways forward
  • hook-augment guidance stops telling the agent to run index_repository in a directory where it would only be refused

The explicit call keeps an escape hatch — index_worktree=true indexes a worktree regardless of the setting, so the key never becomes a dead end:

repo_path is a linked git worktree and ignore_worktrees is enabled. Pass
index_worktree=true to index it anyway, or run:
codebase-memory-mcp config set ignore_worktrees false

Default is false, so nothing changes for existing users unless they opt in.

Detection

cbm_git_is_linked_worktree() is git plumbing only, no subprocess — it runs on every session start, so shelling out the way cbm_git_context_resolve() does was not an option.

<path>/.git must be a regular file holding a gitdir: pointer and that gitdir must contain a commondir file.

That second condition is the important one: a submodule's .git is also a gitlink file, so a naive "is .git a regular file?" check misclassifies every submodule as a worktree. A submodule's gitdir (<super>/.git/modules/<name>) has no commondir entry, which separates the two cleanly. The main checkout of a repo that has worktrees is never affected.

Tests

tests/test_git_context.c — detection truth table: linked worktree true; the main checkout of that same repo false; a submodule false; a plain non-git dir false.

tests/test_mcp.c — the config gate on explicit index_repository: worktree refused, main checkout not refused, and index_worktree=true escaping the gate.

Both follow the existing fixture patterns in those files (th_mktempdir/th_rmtree, SKIP_PLATFORM on Windows, graceful skip when git is unavailable).

Verification

  • scripts/build.sh — clean
  • scripts/test.sh — 6661 passed, 0 failed (116 suites)
  • make -f Makefile.cbm security — all 8 layers pass
  • clang-format (repo-wide lint-format) and cppcheck — clean; clang-tidy reports 0 findings on the added lines
  • manually verified end-to-end against the built binary: refusal on a worktree, normal indexing on the main checkout, index_worktree=true override, and unchanged default-off behaviour

Docs

docs/CONFIGURATION.md gets a table row plus a section covering motivation, the enable command, all three behaviours, and the submodule/main-checkout caveat. README.md gets a short paragraph next to auto_watch.

@vitalNohj
vitalNohj requested a review from DeusData as a code owner August 4, 2026 11:23
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

  • It will not be closed for inactivity. No stale bot touches pull requests here.
  • It may still sit a while before a human reads it. That is on us, not on you.
  • Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

  • Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
  • Get CI green, or say which failures you believe are pre-existing.
  • Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
  • Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

@vitalNohj
vitalNohj force-pushed the fm/cbm-worktree-ignore-1 branch from 1003528 to d21918c Compare August 6, 2026 18:25
@DeusData DeusData added enhancement New feature or request ux/behavior Display bugs, docs, adoption UX stability/performance Server crashes, OOM, hangs, high CPU/memory labels Aug 10, 2026
@DeusData DeusData added this to the 0.11.0-rc milestone Aug 10, 2026
@DeusData DeusData added the priority/normal Standard review queue; useful PR with ordinary maintainer urgency. label Aug 10, 2026
@DeusData

Copy link
Copy Markdown
Owner

Thank you for the focused ignore_worktrees configuration contribution. This adds public configuration and changes repository-discovery behavior, so I have routed it to 0.11.0-rc with normal priority for a full behavior review. The community queue is currently full, and detailed feedback may take a little time.

@DeusData

Copy link
Copy Markdown
Owner

@vitalNohj — the behavior review is done, and the answer is accepted: both surfaces ship — the ignore_worktrees config key and the index_worktree per-call override — once the rework below lands. Your detection design carried the decision: the commondir marker is the right discriminator (submodule-safe, since submodule gitdirs never carry it), the fail-open posture on any probe failure is exactly correct, and default-off is verifiably bit-for-bit today's behaviour.

Context you deserve on why this needed a maintainer decision at all: there is a parked direction (see #573) where worktrees eventually get a cheap view keyed by the git common dir rather than nothing. Your key is opt-in-off, so it forecloses none of that — and your commondir plumbing is genuinely reusable for that family-identity work later. But "skip worktrees on purpose" and "worktrees get an overlay" are opposite product answers to the same pain, so shipping yours ahead of that decision was a call, not a formality. It is now made, in your favour.

The rework map — one item is invisible and load-bearing:

  1. CONFIG_KEYS row (the invisible one). Since 0.10.0: two silent empty-result paths — tree format returns {} over MCP (default), and config get returns empty for all keys but auto_index #1522/Installer/updater scope and safety: no per-client selector, destructive update step before an interactive prompt, hidden ui_enabled, brittle legacy-marker cleanup #1558, that table in src/cli/cli.c (:6837) is the config set allowlist — config_key_lookup rejects unknown keys (:6997). Your PR predates this, patches the old printf blocks, and adds no row — so on current main every enable command your own docs and refusal message suggest would error unknown config key. Add {CBM_CONFIG_IGNORE_WORKTREES, "false", "<help text>"} with the default mirroring your runtime reader, per the table's keep-in-sync comment. Nothing will fail a test if you miss this — the Installer/updater scope and safety: no per-client selector, destructive update step before an interactive prompt, hidden ui_enabled, brittle legacy-marker cleanup #1558 pin covers only the UI keys — which is precisely why it leads this list.
  2. Rebase: conflicts in README, CONFIGURATION.md and the rewritten config region of cli.c; the other nine files auto-merge.
  3. The default-off pin: a test asserting that with the key unset, a linked worktree still indexes exactly as today — the regression class we guard hardest. Your explicit-gate probe with its positive control and override-escape is good; this is the missing direction.
  4. If you can: cover the auto-index path (your _for_testing wrapper hardcodes false, so the new branch is currently unreachable from the test hook), swap the hardcoded /tmp for th_mktempdir as your body describes, and let fixture failures SKIP rather than PASS.

The one red CI leg on your final SHA (test_daemon_ipc.c:2248, Windows) is a timeout-decided test on a loaded runner, mechanism-unrelated to your diff — it is now on our flake ledger, and a rebase re-runs everything anyway.

Take your time; the acceptance does not expire. This is a first contribution built the way we wish more were.

Agents that use `git worktree add` to run parallel branches end up with one
indexed project per worktree. Each is a near-duplicate of the main checkout,
so the cache fills with redundant graphs and project pickers get noisy.

Add an opt-in `ignore_worktrees` config key. When enabled, a linked worktree
is left alone:

  - auto-index on connect skips it (logged as autoindex.skip)
  - the daemon's background auto-index skips it
  - explicit index_repository refuses with an actionable message naming both
    ways forward (index_worktree=true, or turn the key off)
  - the hook-augment "not indexed" guidance stops telling the agent to run
    index_repository in a directory where it would only be refused

Detection is git plumbing, no subprocess: <path>/.git must be a regular file
holding a "gitdir:" pointer AND that gitdir must contain a `commondir` file.
The commondir check is what separates a linked worktree from a submodule,
whose .git is also a gitlink file but whose gitdir has no commondir. The main
checkout of a repo that has worktrees is never affected.

Defaults to false, so behaviour is unchanged unless the key is set.

Signed-off-by: Nohj <[email protected]>
@cursor
cursor Bot force-pushed the fm/cbm-worktree-ignore-1 branch from 6bb8ad6 to 0d2f840 Compare September 23, 2026 06:20

@DeusData DeusData left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vitalNohj, thank you, sincerely. You took a rework map with one invisible, untested trap at the top of it and came back with every item done, plus the tests we only asked for "if you can". I'm sorry, too: this landed on 09-23 and we didn't notice for two days. That's on us, not on you.

What we checked on 0d2f840c, item by item against the map:

  1. CONFIG_KEYS row: present in src/cli/cli.c with default false, mirroring the runtime reader. It's pinned by your new cli_ignore_worktrees_config_key_is_discoverable, so the enable command your docs and refusal message suggest now works.
  2. Rebase: the branch merges cleanly with today's main.
  3. Default-off pin: mcp_ignore_worktrees_default_off_still_indexes_worktree covers the direction that was missing. We also checked the code: all four call sites (explicit call, in-process auto-index, daemon auto-index, hook guidance) short-circuit on the config read before any filesystem probe. With the key unset, today's behaviour is untouched.
  4. Auto-index coverage, th_mktempdir, SKIP-not-PASS: all done. Your two auto-index tests drive the real initialize path with the count hook as the observable, which is exactly the shape we'd have asked for.

We also re-read the detector: the commondir discriminator, relative-pointer resolution, and fail-open on every probe failure. We confirmed the new read goes through cbm_fopen, that the change adds no raw allocation sites to src/, and that nothing in the diff goes beyond the feature.

Two optional wording nits, entirely up to you and fine to leave:

  • The one-line help text and the docs table row say "when indexing automatically", but the key also refuses explicit index_repository calls. The section below the table already explains that, so something like "Skip linked git worktrees when indexing (explicit calls need index_worktree=true)" would make the one-liner match.
  • The explicit default-off test asserts "not refused" rather than a successful index. The auto-index default-off test already covers the positive side, so this is only a note.

We'll run a fresh CI pass against current main and merge from there. Thank you again. This is a contribution built the way we wish more were, and it's already noted for the worktree-family work in #573.

Agents that use `git worktree add` to run parallel branches end up with one
indexed project per worktree. Each is a near-duplicate of the main checkout,
so the cache fills with redundant graphs and project pickers get noisy.

Add an opt-in `ignore_worktrees` config key. When enabled, a linked worktree
is left alone:

  - auto-index on connect skips it (logged as autoindex.skip)
  - the daemon's background auto-index skips it
  - explicit index_repository refuses with an actionable message naming both
    ways forward (index_worktree=true, or turn the key off)
  - the hook-augment "not indexed" guidance stops telling the agent to run
    index_repository in a directory where it would only be refused

Detection is git plumbing, no subprocess: <path>/.git must be a regular file
holding a "gitdir:" pointer AND that gitdir must contain a `commondir` file.
The commondir check is what separates a linked worktree from a submodule,
whose .git is also a gitlink file but whose gitdir has no commondir. The main
checkout of a repo that has worktrees is never affected.

Defaults to false, so behaviour is unchanged unless the key is set.

Signed-off-by: Nohj <[email protected]>
Upstream's Unicode path fix dropped stat() from this file because it follows
symlinks and rejects non-ASCII paths on Windows. Keep ignore_worktrees
detection on cbm_path_info_utf8 so the rebase does not restore that probe.

Signed-off-by: Nohj <[email protected]>
A session queued while capacity was full could still be indexed after the
setting was turned on. Retry admission now applies the same linked-worktree
gate as the initial path.

Signed-off-by: Nohj <[email protected]>
The production hook builds its server with cbm_mcp_server_new(NULL), so
ignore_worktrees stayed off and an unindexed linked worktree still told
the agent to run index_repository. Attach the runtime store for that
check, then drop it before return.

Signed-off-by: Nohj <[email protected]>
ignore_worktrees only inspected <path>/.git, so a session or index path
inside a linked worktree still passed the gate. Walk ancestors until a
git anchor, and stop at a main checkout or a submodule.

Signed-off-by: Nohj <[email protected]>
ignore_worktrees checked the setting only after project resolution, so a
linked worktree nested in an indexed checkout was reported as that parent
project. Drop an ancestor match and keep an index whose root is the
worktree itself.

Signed-off-by: Nohj <[email protected]>
The parent-graph drop treated any linked worktree root as a match, so an
unindexed worktree nested inside an indexed one still inherited that
parent. Compare the gitdir of each path's own anchor and keep the project
only when both paths are that same linked worktree.

Signed-off-by: Nohj <[email protected]>
A 64-level walk missed a deep session path, and a relative gitdir joined
onto a symlink spelling no longer matched the real project root. Walk to
the filesystem root and compare canonical gitdirs. Drop the unused project
resolver that failed the macOS -Werror build.

Signed-off-by: Nohj <[email protected]>
A symlink into a linked worktree has no .git on its lexical path, so the
hook told the agent to run index_repository. That call canonicalizes first
and then refuses. Resolve the cwd before project lookup and the
ignore_worktrees checks.

Signed-off-by: Nohj <[email protected]>
The ignore_linked initializer exceeded the formatter's assignment break.
Lint treats that as an error, so the line now matches .clang-format.

Signed-off-by: Nohj <[email protected]>
The memory-core ratchet rejects another raw free() in the hook. Keep the
parent name until the existing free(project) and skip it when the match
is a different worktree.

Signed-off-by: Nohj <[email protected]>
@vitalNohj
vitalNohj force-pushed the fm/cbm-worktree-ignore-1 branch from a43f07e to b2e1433 Compare September 30, 2026 23:31

@DeusData DeusData left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @vitalNohj, for continuing to work on this after the approval. The follow-ups fix real gaps (the hook note was dead without a runtime config), and keeping the detector as pure plumbing through cbm_fopen is exactly right.

The branch has grown a lot since the approved commit (19 more commits, +999/−24 against +597/−8), so the earlier approval no longer covers it. Before we can merge, could you do four things?

  1. Split out the two unrelated commits. adc2764a (fuzz build: link through the C++ driver) and 9588007b (free partial Cypher patterns after parse errors) are both welcome, but they are separate claims.

    • The Cypher fix looks sound and fixes a leak main still has. Please send it as its own PR.
    • The fuzz link rule is not built by PR CI at all; only the dry-run and nightly lanes run scripts/fuzz.sh. It needs its own PR so we can verify it on that lane.
  2. One linear series on current main. We merge with merge commits, never squash, so the branch history lands as it is. Right now it contains:

    • the feature commit twice;
    • a fork-internal merge without a sign-off;
    • a back-merge of main;
    • a merge (7ca336b5) that does not link, and that switched git_capture back from cbm_popen_git to cbm_popen until 62479ff9 repaired it.

    A rebase onto current main into a few signed-off commits would fix all of that.

  3. Keep the default path untouched. With ignore_worktrees unset, the hook now opens the runtime config store and runs realpath on the cwd for every lifecycle event. It also attaches and then clears a config on a server it does not own (the daemon passes its shared session server). The approval rested on every call site short-circuiting on the config read before any filesystem work. Could the hook read the key first and do nothing else when it is off? If the store has to be opened just to read the key, please say so and keep everything after that behind the key.

  4. A sentence on subdirectories. index_repository on any subdirectory of a linked worktree is now refused, and the ancestor walk runs to the filesystem root. That may well be what we want. Please state it in the PR description so that it is a decision and not a side effect.

We'll re-review as soon as the new head is up. Thanks again for the care you've put into this; the worktree identity work in particular is thorough.

@DeusData

Copy link
Copy Markdown
Owner

Thanks again for the worktree-ignore work. The October 1 review remains the current request: separate the unrelated fixes, keep the default-off hook path minimal, and document subdirectory behavior in the focused revision. The earlier approval covered an older head; no new changes are requested by this follow-up.

…ange

The review of 2026-10-01 asked to split these out of the ignore_worktrees
change. Neither belongs to it, and both are already on main in another
form, so this restores the three files to the branch merge-base:

- Makefile.cbm: main links cbm-fuzz with -fsanitize-link-c++-runtime.
- src/cypher/cypher.c and tests/test_cypher.c: main frees a partially
  parsed MATCH pattern since 70543ba, with its own regression test
  cypher_parse_error_frees_partial_pattern.

Signed-off-by: Martin Vogel <[email protected]>
… is off

The lifecycle hook now reads ignore_worktrees first and does nothing
else for it while the key is off; the project lookup is then the one
the hook did before the key existed.

- cbm_mcp_ignore_worktrees_runtime_enabled() replaces
  cbm_mcp_server_attach_runtime_config(). It never attaches a store to
  the server, which may be the daemon session server. A server with a
  store answers from it. Otherwise the runtime store is opened only to
  read this key, and only when it already exists, so a hook event never
  creates _config.db.
- cwd canonicalization, the linked-worktree probe and the parent-graph
  check moved into ha_resolve_project_ignoring_worktrees(), which runs
  only when the key is on.

Test cli_hook_ignore_worktrees_default_off_does_no_worktree_work was red
on the previous head (a default-off event created _config.db). It also
pins, through a test-API counter, that neither an absent store nor a
store without the key reaches the worktree gate, and that the key on
reaches it once. mcp_ignore_worktrees_runtime_read_leaves_server_and_store_alone
replaces the test of the removed helper.

Signed-off-by: Martin Vogel <[email protected]>
The review of 2026-10-01 asked for this to be a stated decision rather
than a side effect. Detection walks up from the given path to the
nearest .git entry, so a subdirectory of a linked worktree is treated
as that worktree: index_repository on it is refused, and a session root
or hook working directory inside it is handled the same way. The walk
stops at the first .git; outside any checkout it runs to the filesystem
root one directory at a time. None of it runs while the key is off.

Signed-off-by: Martin Vogel <[email protected]>
Brings the branch up to main a5b248f. Eight files conflicted, each a
same-spot insertion where main and this branch added lines next to the
same anchor. Each resolves to the main version plus the change of this
branch:

- src/cli/cli.h, src/cli/cli.c: CBM_CONFIG_IGNORE_WORKTREES and its
  CONFIG_KEYS row next to the new doc_candidates key.
- src/mcp/mcp.c: the index_worktree schema property next to the new
  doc_candidates property.
- README.md, docs/CONFIGURATION.md: the ignore_worktrees line and table
  row next to the rewritten auto_watch entries and the new
  doc_candidates and watch_non_git rows.
- src/git/git_context.c: the env-isolation comment after the new debug
  stage marker, directly above cbm_popen_git.
- src/cli/hook_augment.c: the new SubagentStart cli_hint ahead of the
  parent-graph check of this branch.
- tests/test_mcp.c: both RUN_TEST blocks; main already includes
  str_util.h.

Per file, the lines changed against main equal the changes of this
branch against the merge-base.

Signed-off-by: Martin Vogel <[email protected]>
cppcheck 2.20 (the CI version) reports shadowFunction: the local
cache_dir in cbm_mcp_ignore_worktrees_runtime_enabled() shadows the
static cache_dir() helper in mcp.c. Rename the local to store_dir; no
behavior change.

Signed-off-by: Martin Vogel <[email protected]>
@DeusData

DeusData commented Nov 2, 2026

Copy link
Copy Markdown
Owner

@vitalNohj, thank you for sticking with this one, and for all the work after the first approval. The commondir check, the fail-open probes, the symlink and nested-worktree handling and the default-off tests are careful work, and they made this easy to finish. To save you another round trip we pushed five small commits on top of your branch that cover the October 1 review:

  1. faf25ee - drops the fuzz-link and Cypher hunks from this PR (review item 1). Both landed on main in the meantime: the cbm-fuzz rule now links with -fsanitize-link-c++-runtime, and 70543ba frees a partially parsed MATCH pattern with its own regression test. So no separate PRs are needed. If you would like your two extra malformed-pattern queries as additional test cases, a small tests-only PR is very welcome.
  2. 77310d5 - keeps the hook default path untouched (review item 3). The lifecycle hook now reads ignore_worktrees first and does nothing else for it while the key is off: no realpath on the cwd, no worktree probe, and the same project lookup as before the key existed. The key comes from a new cbm_mcp_ignore_worktrees_runtime_enabled(), which replaces cbm_mcp_server_attach_runtime_config(). It never attaches a store to the server (the daemon lends its session server here) and answers from the server store when there is one. Otherwise it opens the runtime store only to read this key, and only if the store already exists, so a hook event never creates _config.db. To answer the question from the review: yes, without a server store the store has to be opened to read the key, and everything after that read is behind the key. A new test, cli_hook_ignore_worktrees_default_off_does_no_worktree_work, pins this; it failed on the previous head, where a default-off event created _config.db. The test for the removed helper is replaced by one for the new reader.
  3. 04ed024 - documents subdirectories in docs/CONFIGURATION.md (review item 4). For the record, this is now the stated behavior: a subdirectory of a linked worktree counts as that worktree, so index_repository on it is refused like the worktree root, and a session or hook directory inside it is treated the same way. The walk stops at the first .git it finds; outside any checkout it runs up to the filesystem root, one directory at a time. None of it runs while the key is off.
  4. 1976986 - merges current main. Eight files conflicted, all of them lines added next to the same spot on both sides (config keys, the tool schema, docs rows); each is resolved to main plus your change.
  5. 141b25a - renames one local variable in the new reader that cppcheck 2.20 flags as shadowing cache_dir().

On review item 2 (one linear series): we merge with merge commits and never force-push a contributor branch, so we left your history as it is rather than rewriting it.

We also checked it end to end on a clone of redis with a linked worktree beside it. With the key off, both checkouts index (34,539 and 34,544 nodes; the worktree carries one extra file). With the key on, the worktree and its src/ subdirectory are refused with your message, the main checkout indexes exactly as before, and index_worktree=true indexes the worktree to the same 34,544 nodes.

Please pull the branch before pushing again, so these commits are not lost. Thank you again; this is a contribution built the way we wish more were.

@DeusData
DeusData dismissed their stale review November 2, 2026 20:58

Superseded: items 1, 3 and 4 of this review are done on 141b25a (see the comment above), and item 2 (one linear series) is set aside because we merge with merge commits and do not rewrite contributor history.

@DeusData
DeusData merged commit 2921415 into DeusData:main Nov 2, 2026
34 checks passed
@DeusData

DeusData commented Nov 2, 2026

Copy link
Copy Markdown
Owner

Merged - thank you very much, @vitalNohj! ignore_worktrees is now on main: with the key on, a linked worktree (and any subdirectory of it) is refused with your actionable message and index_worktree=true still lets someone index it on purpose, while with the key off nothing changes at all - the hook path is exactly the old lookup. The commondir check, the fail-open probes and the symlink and nested-worktree handling made this easy to trust. We dismissed our old October 1 change request, since everything it asked for is covered by the commits above. Thanks again for sticking with it through all the rounds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request priority/normal Standard review queue; useful PR with ordinary maintainer urgency. stability/performance Server crashes, OOM, hangs, high CPU/memory ux/behavior Display bugs, docs, adoption UX

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants