Skip to content

Respect nested .gitignore rules when mining project files#78

Merged
bensig merged 2 commits intoMemPalace:mainfrom
ac-opensource:feature/respect-gitignore-mining
Apr 7, 2026
Merged

Respect nested .gitignore rules when mining project files#78
bensig merged 2 commits intoMemPalace:mainfrom
ac-opensource:feature/respect-gitignore-mining

Conversation

@ac-opensource
Copy link
Copy Markdown
Contributor

@ac-opensource ac-opensource commented Apr 7, 2026

What does this PR do?

Project mining now respects .gitignore files as the directory walk descends, so nested repos and subdirectories can apply their own ignore rules instead of only reading the top-level file.

It keeps the implementation dependency-free, preserves Git-style precedence for stacked rules, adds --no-gitignore for full scans, and adds --include-ignored for project-relative escape hatches such as docs,libs or generated/keep.py.

It also expands the built-in skip list for common cache and IDE directories.

How to test

  1. Run python -m pytest tests/ -v
  2. Run ruff check .
  3. Create a parent directory with nested repos or subdirectories that have their own .gitignore files, then run mempalace mine <dir> --dry-run
  4. Confirm ignored paths are skipped, --no-gitignore indexes them again, and --include-ignored docs,libs brings back only the requested paths

Checklist

  • Tests pass (python -m pytest tests/ -v)
  • No hardcoded paths
  • Linter passes (ruff check .)

@sha2fiddy
Copy link
Copy Markdown
Contributor

Beat me by 5min lol. I submitted #80 for the same issue.

Quick comparison: this PR reads the root .gitignore, while mine uses a stack that picks up .gitignore files at each directory level during the walk. The nested approach matters when mining a parent dir with multiple repos, since each sub-repo has its own ignore rules. Mine also adds a --no-gitignore opt-out flag and expands SKIP_DIRS with common Python/IDE cache dirs.

@ac-opensource
Copy link
Copy Markdown
Contributor Author

@sha2fiddy yeah, I just kept it simple, I also didn't want to introduce additional dependencies

@ac-opensource ac-opensource changed the title Respect .gitignore when mining project files Respect nested .gitignore rules when mining project files Apr 7, 2026
@bensig
Copy link
Copy Markdown
Collaborator

bensig commented Apr 7, 2026

Looks solid — clean implementation, good test coverage, and no new dependencies. Will merge once CI goes green.

@ssasoglu
Copy link
Copy Markdown

ssasoglu commented Apr 8, 2026

When will we get a release that includes this change @bensig?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants