Skip to content

feat: discover all readable text files as skill resources (#54) - #55

Merged
DougTrajano merged 3 commits into
mainfrom
feat/resource-discovery-all-text-files
Jul 15, 2026
Merged

DougTrajano merged 3 commits into
mainfrom
feat/resource-discovery-all-text-files

Conversation

@DougTrajano

@DougTrajano DougTrajano commented Jul 15, 2026 •

Copy link
Copy Markdown
Owner

What & why

Closes #54. Resource discovery used a hardcoded extension allowlist (.md, .json, .yaml, .yml, .csv, .xml, .txt) with no way to extend it, so common text assets a skill ships — .sql, .toml, .jinja, .tmpl — were silently invisible to the model, even when SKILL.md referenced them by name. read_skill_resource(...) would then raise ModelRetry("... not found").

Rather than just widening the allowlist, this flips the model: any readable text file in the skill folder is a resource, with guardrails to keep discovery safe and quiet.

What changed

  • Allow-all-text discovery (directory.py): every file that decodes as UTF-8, in any subdirectory, becomes a resource. _discover_resources was rewritten from per-extension rglob loops to a single rglob('*') + filter.
  • Bounded text detection: _is_text_file decodes at most a 64 KB prefix (via an incremental UTF-8 decoder), matching the loader's read_text('utf-8') contract without reading whole files. Files that fit in the window are validated exactly; larger files are prefix-checked (a multibyte char split by the window is not a false reject). Binaries fail on the first invalid byte.
  • Scripts are never resources: files discovered by _discover_scripts are excluded from resources via a script_uris set (scripts discovered first, then passed into resource discovery).
  • New exclude_resources parameter: glob patterns that extend (never replace) the built-in DEFAULT_RESOURCE_EXCLUDES (__pycache__, *.pyc, *.pyo, .DS_Store, .git). A pattern matches the full skill-relative posix path or any single path component. Threaded through discover_skills, SkillsDirectory, Skill.from_file, SkillsToolset, and SkillsCapability (+ from_spec, so it works in agent specs). DEFAULT_RESOURCE_EXCLUDES is exported for easy extension.

Reviewer notes

  • Behavior change: skills may now expose more files by default (e.g. a stray .log or notes.org). This is intentional and matches our expectations that files in the skill bundle are readable; noise/binaries are filtered.
  • exclude_resources is a deny-list, not an allow-list — its default contains noise patterns, so allow-list semantics wouldn't make sense. User patterns are additive so noise stays excluded even when custom patterns are supplied.
  • UTF-8 is hardcoded in _is_text_file to stay in lockstep with the loader's read_text('utf-8'). If the loader's encoding ever becomes configurable, this must track it.
  • Docs updated in creating-skills.md; API-reference docstrings flow through mkdocstrings.

Tests

Added coverage for: arbitrary text extensions discovered by default, binary skip, invalid-UTF-8 rejection (a case the old approach would misclassify), multibyte acceptance, script exclusion, default noise exclusion, and additive exclude_resources — across discovery, toolset, and capability layers. Full suite: 459 passed; pre-commit (ruff + mypy) clean.

🤖 Generated with Claude Code

Resource discovery previously used a hardcoded extension allowlist
(.md/.json/.yaml/.yml/.csv/.xml/.txt) with no way to extend it, so common
text assets a skill ships (.sql, .toml, .jinja, ...) were silently invisible
even when SKILL.md referenced them (issue #54).

Replace the allowlist with allow-all-text discovery plus guardrails:

- Any file that decodes as UTF-8 is a resource, in any subdirectory. Detection
  decodes a bounded 64KB prefix (files within the window are validated exactly),
  matching the loader's read_text('utf-8') without reading whole files.
- Files discovered as scripts are excluded from resources (a file is never both).
- New `exclude_resources` glob-pattern parameter (extends, never replaces, the
  built-in DEFAULT_RESOURCE_EXCLUDES: __pycache__, *.pyc, *.pyo, .DS_Store, .git)
  threaded through discover_skills, SkillsDirectory, Skill.from_file,
  SkillsToolset, and SkillsCapability (+ from_spec).

This is a behavior change: skills may now expose more files by default.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Copilot AI review requested due to automatic review settings July 15, 2026 02:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

DougTrajano and others added 2 commits July 14, 2026 23:30
Each exception test constructed Mock() inside the pytest.raises block,
giving it two invocations that could throw. Assign ctx = Mock() before the
block so only the awaited call remains inside, satisfying python:S5778.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
@sonarqubecloud

Copy link
Copy Markdown

@DougTrajano
DougTrajano merged commit 79de88b into main Jul 15, 2026
17 checks passed
@DougTrajano
DougTrajano deleted the feat/resource-discovery-all-text-files branch July 15, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_discover_resources uses a hardcoded extension allowlist with no way to extend it

2 participants