Repository navigation
EA3 flags Apache-2.0 LICENSE boilerplate as scope creep (fires on 814/817 skills in one corpus) #312
Description
Summary
EA3 ("Scope Creep") matches the phrase "not limited to", which appears in the standard Apache-2.0 and MIT license texts. Any skill that ships a LICENSE file therefore gets EA3 findings for its license, not its behaviour.
Measured on main (34f6030, 2.5.0) across 817 skills from the public anthropic-cybersecurity-skills plugin:
- EA3 fires on 814/817 skills (99.6%) — the highest firing rate of any rule in the corpus.
- 1628 of 1630 total EA3 occurrences are in
LICENSEfiles.
Cause
src/skillspector/nodes/analyzers/static_patterns_excessive_agency.py:
(r"(?:not\s+limited\s+to|beyond\s+(?:the\s+)?(?:scope|stated|described|documented))", 0.7),Apache-2.0 contains "including, but not limited to" in several clauses (the warranty disclaimer, the definitions of "Contribution" and "Derivative Works"). There is no file-type exclusion, so LICENSE / LICENSE.txt is scanned as if it were skill instructions.
Reproduce
$ grep -c "not limited to" <any-skill>/LICENSE
4
$ skillspector scan <any-skill> --no-llm --format json | \
jq '[.issues[] | select(.id=="EA3")] | {n: length, files: [.[].location.file]}'
{ "n": 2, "files": ["LICENSE", "LICENSE"] }The finding text on each is literally "not limited to".
Not corpus-specific
It reproduces on an unrelated corpus: Anthropic's official anthropics/skills repo. Scanning skills/mcp-builder yields 2 EA3 findings, both in LICENSE.txt, both with finding text "not limited to". Six of that repo's 18 skills score 100/CRITICAL overall.
Impact
EA3 is the top-firing rule in the corpus and its signal is almost entirely license text. It inflates scores for the well-behaved case — shipping a license is good practice — and it is noise that a user cannot distinguish from a real finding, since severity and confidence are identical to a genuine scope-creep match.
It also has a compounding effect on anyone trying to use --baseline for false-positive management: the suppression file fills with license findings.
Suggested fix
Either exclude license files from the prose analyzers by filename (LICENSE*, COPYING*, NOTICE*), or require the phrase to co-occur with an imperative/instructional context rather than matching bare. The former is the smaller change and I would expect it to be correct for every prose rule, not just EA3 — a license is never skill instructions.
Happy to open a PR if you have a preference on which approach.
Environment
main @ 34f6030 (2.5.0), Python 3.12, macOS. Also reproduced on 2.3.7 (ab0431f), so this is not a recent regression.
Activity
RichCorbett commentedon Jul 28, 2026
Third corpus, and it turns out to be the cleanest evidence yet — because it's a negative control.
I scanned all 864 skills in ComposioHQ/awesome-claude-skills (71k stars). Results are near-silent: 7/864 score >50 (1%), median score 0, mean 1, and EA3 fires on only 9 skills (1%).
That is the opposite of the cybersecurity corpus, and the reason is exactly the mechanism in this report:
LICENSEfilesEA3's firing rate tracks per-skill
LICENSEpresence almost 1:1 across three unrelated corpora — 816 licenses to 814 EA3 hits in one, 13 to 9 in another. It is not tracking anything about the skills' behaviour.I want to be even-handed about what this does and doesn't say. It is not evidence that SkillSpector is broadly noisy — a corpus of 864 mostly-prose skills scoring a median of 0 is the tool being appropriately quiet, and that is worth something. The problem is narrow and mechanical: when a skill ships a license file, one specific prose pattern reads the license as instructions.
Which I think makes the filename-exclusion fix (
LICENSE*,COPYING*,NOTICE*) the right one, and low-risk — it targets precisely the files that are never skill instructions, and leaves behaviour on prose-only skills unchanged.Method:
skillspector scan <skill-dir> --no-llm --format jsonper skill directory, 2.3.7 binary; the 817-skill numbers were separately reproduced onmain@34f6030.RichCorbett commentedon Jul 28, 2026
Correction to my own impact claim above, in the interest of not leaving an overstated report standing.
I have now run the LLM-backed path (2.5.0,
claude_cliprovider) rather than--no-llm, and EA3 license findings do not survive the meta-analyzer. Scanninganalyzing-cyber-kill-chainfrom the same corpus with the LLM pass enabled:The two survivors are YARA hits in
scripts/agent.py; the license-derived EA3 findings are filtered out. Across a separate 25-skill LLM run, EA3 did not appear in the surviving set at all.So the accurate impact statement is narrower than what I wrote:
"noise that a user cannot distinguish from a real finding"— not true in the default LLM path; the meta-analyzer correctly discards it.--no-llm, where nothing filters it and it becomes the top-firing rule at 814/817 with 1628/1630 occurrences inLICENSEfiles.I still think it is worth fixing, because
--no-llmis not a corner case — it is the mode for CI gating, batch triage over large corpora, and any air-gapped or cost-sensitive use, and it is the only mode fast enough to run over hundreds of skills (~0.77s/skill versus ~4min/skill with the LLM). A user in that mode sees EA3 as the single loudest signal in their corpus and has no filter behind it. Fixing the pattern also saves the LLM the work of discarding ~1600 findings it should never have been handed.But I did not want the issue to read as "this corrupts SkillSpector's verdicts" when the tool's default path already handles it correctly. The measurement stands; my characterisation of the consequence was too strong.
fix(analyzer): scope EA3 license suppression to recognized boilerplate (
fix(analyzer): filter license boilerplate from EA3 static findings (N…
fix(analyzer): scope EA3 license suppression to recognized boilerplate (
fix(analyzer): filter license boilerplate from EA3 static findings (N…
fix(analyzer): scope EA3 license suppression to recognized boilerplate (
fix(analyzer): filter license boilerplate from EA3 static findings (N…
fix(analyzer): scope EA3 license suppression to recognized boilerplate (
rng1995 commentedon Aug 14, 2026
Active implementation: PR #328. The PR is linked back with Closes #312; keeping this issue open until it merges.
fix(analyzer): filter license boilerplate from EA3 static findings (#312