Skip to content

fix(discover): skip .worktrees directory during indexing - #37

Closed
wassertim wants to merge 1 commit into
DeusData:mainfrom
wassertim:fix/skip-worktrees-directory
Closed

wassertim wants to merge 1 commit into
DeusData:mainfrom
wassertim:fix/skip-worktrees-directory

Conversation

@wassertim

Copy link
Copy Markdown
Contributor

Summary

  • Add .worktrees to IGNORE_PATTERNS in internal/discover/discover.go
  • Add test case verifying .worktrees/ contents are skipped during discovery

Problem

Git worktrees (stored in .worktrees/) duplicate the entire source tree. When indexing a monorepo that uses worktrees for parallel development, the knowledge graph ends up with duplicate nodes and edges for every function, class, and route. This causes:

  • Route searches returning 2-3x duplicate results
  • search_graph results pointing to worktree copies instead of the main source
  • Inflated node/edge counts (we saw 12,554 nodes where many were duplicates)

Context

Discovered while evaluating codebase-memory-mcp on a TypeScript monorepo with active worktrees. Related to #31 (.gitignore support would also solve this, but .worktrees should be unconditionally skipped like .git).

Test plan

  • go test ./internal/discover/ -v -run TestDiscoverSkipsWorktrees passes
  • Existing TestDiscoverBasic still passes

Git worktrees (stored in .worktrees/) duplicate the entire source tree,
causing the knowledge graph to contain duplicate nodes and edges for
every function, class, and route. This is especially problematic in
monorepos where worktrees are used for parallel development.

Add .worktrees to IGNORE_PATTERNS alongside other VCS/tool directories
like .git and .svn. Includes a test case verifying the directory and
its contents are skipped during discovery.

Fixes DeusData#31 (partial — .worktrees specifically)
@DeusData

Copy link
Copy Markdown
Owner

Thanks for this fix @wassertim! Great catch on the worktrees duplication issue — especially the detail about inflated node counts in monorepos.

I've incorporated your change into main in commit cab67eb along with a broader set of ignore improvements (.gitignore parsing, .cbmignore support, symlink skipping, expanded ignore list). Your .worktrees addition and test are included with attribution in the commit message.

Closing this as the fix is now in main. Thanks again for the contribution! 🙏

@DeusData DeusData closed this Mar 12, 2026
DeusData pushed a commit that referenced this pull request Mar 12, 2026
Git worktrees (stored in .worktrees/) duplicate the entire source tree,
causing the knowledge graph to contain duplicate nodes and edges for
every function, class, and route. Add .worktrees to IGNORE_PATTERNS
alongside .git and .svn.

Incorporates the fix from PR #37 by @wassertim. Closes #31 (partial).

Co-authored-by: wassertim <[email protected]>
@DeusData DeusData mentioned this pull request Mar 12, 2026
lilisir0722-crypto pushed a commit to lilisir0722-crypto/codebase-memory-mcp that referenced this pull request May 23, 2026
Git worktrees (stored in .worktrees/) duplicate the entire source tree,
causing the knowledge graph to contain duplicate nodes and edges for
every function, class, and route. Add .worktrees to IGNORE_PATTERNS
alongside .git and .svn.

Incorporates the fix from PR DeusData#37 by @wassertim. Closes DeusData#31 (partial).

Co-authored-by: wassertim <[email protected]>
ngallodev added a commit to ngallodev/codebase-memory-cli that referenced this pull request Oct 1, 2026
The codeql-gate job failed every run on 32 open alerts. Only one was a
CodeQL finding in this repository's own code; the rest were CodeQL
findings in vendored third-party grammars and OpenSSF Scorecard
repository-hygiene checks.

Gate (.github/workflows/_security.yml):
- Count only CodeQL alerts outside vendored/ and internal/cbm/vendored/.
  Scorecard keeps reporting in its own workflow; vendored grammar code is
  upstream code this repo does not patch.
- On pull requests, query the PR's own analysis
  (ref=refs/pull/N/merge). Without a ref the API answers for the default
  branch, so a fix inside a PR could never clear the gate before merging.
- Paginate (per_page=100, --paginate). The unpaginated call returned at
  most 30 alerts, so the count silently capped at 30.
- Fail-closed behavior is unchanged: an unreadable alert list or a
  non-numeric count still blocks. tests/test_security_gate_fail_closed.sh
  passes.

Findings fixed:
- src/simhash/minhash.c (CodeQL cpp/overflowing-snprintf, alert #13):
  cbm_minhash_to_hex advanced its offset by snprintf's return value
  without checking it. Stop and terminate on a negative or truncated
  write.
- .github/workflows/claude.yml, claude-code-review.yml (Scorecard
  TokenPermissions DeusData#36, DeusData#37): add a deny-all top-level `permissions: {}`;
  each job already grants what it needs.

Diagnostics:
- .github/workflows/_test.yml: upload build/c/test-logs/*.log from a
  failed Windows test shard. The harness prints only the last 15 lines of
  each failing suite, which cut off the ASan report for index_resilience
  on Windows.

Validation: security-audit.sh and the gate fail-closed contract pass;
make lint-ci lint-tidy-diff passes; scripts/test.sh --suites simhash 25
passed. The new filter evaluated against the live alert API counts only
alert #13 on the default branch and 0 for refs/pull/12/merge.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants