|
| 1 | +# Detect Whitespace Padding Injection (P9) |
| 2 | + |
| 3 | +Implements GitHub issue #20 — "Detect large whitespace padding used to hide prompt-injection instructions from review". |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +- Attackers can pad a skill file (e.g. `SKILL.md`) with a large block of whitespace — dozens of blank lines, or a long horizontal run of spaces — so that injected instructions sit below or to the right of anything a human reviewer sees, while the agent reads the whole file and acts on them. The text-file equivalent of white-on-white text in a PDF. |
| 8 | +- Existing patterns miss this: `P2` (Hidden Instructions) keys off zero-width chars/comments/base64, `TP2` (Unicode Deception) off homoglyphs/RTL, and `MP2` (Context Window Stuffing) off character *repetition* — its regex anchors on `\S`, so runs of blank lines slip through. |
| 9 | +- This plan adds a new combined rule **P9 "Whitespace Padding"** (category: Prompt Injection) covering three signals, plus the same detection over MCP manifest description fields. |
| 10 | +- **Critical requirement from the issue:** "whitespace" must mean Unicode whitespace (categories `Zs`, `Zl`, `Zp`, control chars `\t \n \r \v \f`, and the zero-width family U+200B/U+200C/U+200D/U+2060/U+FEFF) — not just ASCII space/tab. The zero-width set must be **one shared definition with P2**, not a drifting copy. |
| 11 | + |
| 12 | +### Decisions made during planning |
| 13 | + |
| 14 | +| Question (from issue #20) | Decision | |
| 15 | +|---|---| |
| 16 | +| Placement | `P`-series, inside `static_patterns_prompt_injection.py` next to P2 | |
| 17 | +| Rule id | **P9** — one combined id for all three signals (P6–P8 are already taken by System Prompt Leakage); per-signal confidence carries the weighting | |
| 18 | +| Severity | Vertical/horizontal: MEDIUM (HIGH when non-blank content follows a very large vertical gap); ratio: LOW | |
| 19 | +| MCP manifest fields | **In scope** — wire the shared detector into `mcp_tool_poisoning.py` description checks, reporting the same P9 id | |
| 20 | +| Testing approach | Regular (code first, then tests, within each task) | |
| 21 | + |
| 22 | +## Context (from discovery) |
| 23 | + |
| 24 | +- Files/components involved: |
| 25 | + - `src/skillspector/nodes/analyzers/static_patterns_prompt_injection.py` — P1–P4 live here; P9 joins them (module is already registered as analyzer node `static_patterns_prompt_injection`, so **no `ANALYZER_NODE_IDS`/`ANALYZER_NODES` changes needed**) |
| 26 | + - `src/skillspector/nodes/analyzers/mcp_tool_poisoning.py` — `_extract_metadata_texts()` (line ~78) yields `(text, source_field, is_identifier)` tuples; `node()` (line ~807) dispatches to `_check_tp1/2/3/4`; has its own `_ZERO_WIDTH_RE` (line ~134) that should converge on the shared definition |
| 27 | + - `src/skillspector/nodes/analyzers/pattern_defaults.py` — needs P9 entries in `DEFAULT_EXPLANATIONS`, `RULE_ID_TO_CATEGORY`, `PATTERN_NAMES`, `DEFAULT_REMEDIATIONS` |
| 28 | + - `src/skillspector/nodes/analyzers/common.py` — shared helpers (`get_line_number`, `get_context`) |
| 29 | + - `README.md` — Prompt Injection pattern table (5 → 6 patterns) |
| 30 | +- Related patterns found: pattern modules expose `analyze(content, file_path, file_type) -> list[AnalyzerFinding]`; findings carry `rule_id`, `message`, `severity`, `location`, `confidence`, `tags`, `context`, `matched_text` (`models.py:46-62`). `static_runner.run_static_patterns` already skips eval datasets and files > 1 MB. |
| 31 | +- Dependencies identified: none new — stdlib `unicodedata` for category classification. |
| 32 | + |
| 33 | +## Development Approach |
| 34 | + |
| 35 | +- **Testing approach**: Regular (implement, then tests, within each task) |
| 36 | +- Complete each task fully before moving to the next |
| 37 | +- Make small, focused changes |
| 38 | +- **CRITICAL: every task MUST include new/updated tests** for code changes in that task |
| 39 | + - Tests are not optional — they are a required part of the checklist |
| 40 | + - Unit tests for new and modified functions; success and error/edge scenarios |
| 41 | +- **CRITICAL: all tests must pass before starting next task** — `make test-unit` |
| 42 | +- **CRITICAL: update this plan file when scope changes during implementation** |
| 43 | +- Maintain backward compatibility — P2/TP1 behavior must not change except via the shared zero-width definition (which is character-for-character identical to today's sets) |
| 44 | + |
| 45 | +## Testing Strategy |
| 46 | + |
| 47 | +- **Unit tests**: required for every task (see above). Pattern tests live in `tests/nodes/analyzers/test_static_patterns.py` (analyzer-level) and `tests/unit/test_patterns_new.py`; MCP tests in `tests/test_mcp_tool_poisoning.py`. |
| 48 | +- No UI, no e2e suite — `make test-unit` is the gate; `make test` (incl. integration) at the end. |
| 49 | + |
| 50 | +## Progress Tracking |
| 51 | + |
| 52 | +- Mark completed items with `[x]` immediately when done |
| 53 | +- Add newly discovered tasks with ➕ prefix |
| 54 | +- Document issues/blockers with ⚠️ prefix |
| 55 | +- Update plan if implementation deviates from original scope |
| 56 | + |
| 57 | +## Solution Overview |
| 58 | + |
| 59 | +A new pure-function helper module `whitespace_padding.py` owns the Unicode whitespace character sets and a `detect_whitespace_padding()` scanner that returns structured "padding run" records. Two consumers build findings from those records: |
| 60 | + |
| 61 | +1. `static_patterns_prompt_injection.analyze()` — emits `AnalyzerFinding(rule_id="P9", ...)` for file bodies (all text files the runner feeds it). |
| 62 | +2. `mcp_tool_poisoning.node()` — a new `_check_p9_padding(text, source_field)` emits `Finding(rule_id="P9", ...)` for tool/parameter description fields (horizontal + ratio signals only; vertical gaps are meaningless in single-field descriptions at manifest granularity but blank-line runs inside a description still count as a contiguous-block signal). |
| 63 | + |
| 64 | +`P2_PATTERNS`' zero-width character class is rebuilt from the shared `ZERO_WIDTH_CHARS` constant so the two patterns cannot drift (issue requirement). `mcp_tool_poisoning._ZERO_WIDTH_RE` likewise. |
| 65 | + |
| 66 | +### The three signals (rule id P9 for all) |
| 67 | + |
| 68 | +| # | Signal | Trigger | Severity | Confidence | |
| 69 | +|---|---|---|---|---| |
| 70 | +| 1 | Vertical blank-line run | ≥ 20 consecutive blank/whitespace-only lines | MEDIUM; **HIGH** when non-blank content follows a gap ≥ 40 lines | 0.8 when non-blank content follows the gap; 0.6 when the gap trails the file | |
| 71 | +| 2 | Horizontal whitespace run | ≥ 80 consecutive whitespace chars within a line (incl. leading indentation); fires on the run itself regardless of what follows on the line | MEDIUM | 0.7 | |
| 72 | +| 3 | Oversized whitespace ratio | a single contiguous whitespace block > 2 KB, **or** whitespace > 90% of a file that is > 4 KB | LOW | 0.4 | |
| 73 | + |
| 74 | +Thresholds are module-level named constants (`VERTICAL_BLANK_LINES = 20`, `VERTICAL_HIGH_SEVERITY_LINES = 40`, `HORIZONTAL_RUN_CHARS = 80`, `BLOCK_BYTE_BUDGET = 2048`, `RATIO_THRESHOLD = 0.90`, `RATIO_MIN_FILE_BYTES = 4096`) so tuning is a one-line change. |
| 75 | + |
| 76 | +### Whitespace classification (shared definitions) |
| 77 | + |
| 78 | +```python |
| 79 | +# whitespace_padding.py |
| 80 | +ZERO_WIDTH_CHARS = frozenset("") # shared with P2 and mcp_tool_poisoning |
| 81 | + |
| 82 | +def is_padding_char(ch: str) -> bool: |
| 83 | + # True for: ASCII controls \t \n \r \v \f; Unicode categories Zs, Zl, Zp |
| 84 | + # (covers U+00A0, U+2028, U+2029, U+3000, etc.); and ZERO_WIDTH_CHARS. |
| 85 | +``` |
| 86 | + |
| 87 | +Covers every evasion candidate enumerated in the issue: U+00A0, U+2028, U+2029, U+000C, U+000B, U+3000, U+200B/C/D, U+2060, U+FEFF. |
| 88 | + |
| 89 | +### Detector output |
| 90 | + |
| 91 | +```python |
| 92 | +@dataclass |
| 93 | +class PaddingRun: |
| 94 | + kind: str # "vertical" | "horizontal" | "block" | "ratio" |
| 95 | + start_offset: int # char offset where the run starts |
| 96 | + start_line: int # 1-based |
| 97 | + length: int # chars (or lines for "vertical") |
| 98 | + followed_by_content: bool |
| 99 | + summary: str # visible-ized snippet, e.g. "U+00A0 x82" or "\\n x82" |
| 100 | +``` |
| 101 | + |
| 102 | +`summarize_run()` renders the run as counts of `U+XXXX xN` segments (collapsing mixed runs to the top few char codes) so the reviewer can *see* what was hidden — per the issue's reporting requirement. |
| 103 | + |
| 104 | +### False-positive guards |
| 105 | + |
| 106 | +- **Fenced code blocks**: the horizontal signal skips runs whose line falls inside a Markdown ``` fence region (line-based fence toggle scan; only applied when `file_type == "markdown"`). Vertical and ratio signals are unaffected (20 blank lines inside a fence is still suspicious). |
| 107 | +- **Generated/vendored files**: skip detection entirely for filenames matching `*.min.js`, `*.min.css`, `*.lock`, `package-lock.json`, `yarn.lock`, `*.svg`, `*.map`. |
| 108 | +- **Binary-ish content**: skip when content contains U+FFFD (the `errors="replace"` marker from `build_context`) — the repo has no other binary classification. |
| 109 | +- **Ratio signal stays LOW/0.4** so it informs rather than dominates the score. |
| 110 | +- Eval-dataset prose and > 1 MB files are already skipped upstream by `static_runner`. |
| 111 | + |
| 112 | +## Technical Details |
| 113 | + |
| 114 | +- **Processing flow (file bodies):** `static_runner` → `static_patterns_prompt_injection.analyze()` → after the P4 loop, call `detect_whitespace_padding(content, file_path=..., file_type=...)` → map each `PaddingRun` to an `AnalyzerFinding` with `rule_id="P9"`, `message="Whitespace Padding"`, `tags=[PatternCategory.PROMPT_INJECTION.value]`, `matched_text=run.summary`, `context=get_context(content, run.start_offset)`. P9 runs for **all** file types fed by the runner (unlike P2's markdown/other restriction), minus the vendored/binary guards above — padding in a `.py` or `.txt` body is the same attack. |
| 115 | +- **Processing flow (MCP manifests):** in `mcp_tool_poisoning.node()`, alongside the `_check_tp1/_check_tp2` dispatch over `_extract_metadata_texts()`, call `_check_p9_padding(text, source_field)` for non-identifier fields. Manifest fields use the same `HORIZONTAL_RUN_CHARS` and `BLOCK_BYTE_BUDGET` thresholds for the first cut; the per-file ratio signal is skipped (fields are too short for a 4 KB floor to ever apply). Findings use the module's existing `Finding` construction style with `rule_id="P9"` and the MCP `_FRAMEWORK_TAGS`. |
| 116 | +- **Dedup within a file:** signals 1/2 report each distinct run; signal 3 ("block"/"ratio") reports at most one finding per file. A vertical run that also exceeds the 2 KB block budget reports only the vertical finding (higher-signal id wins; suppress the block record when its span equals a vertical run's span). |
| 117 | +- **Line/offset reporting:** finding points at the line where the padding **starts** (`get_line_number(content, run.start_offset)`); for a newline-free horizontal run the same line number plus the summary's char codes locate it. |
| 118 | +- **Shared zero-width definition:** `P2_PATTERNS[2]` regex is built as `"[" + "".join(ZERO_WIDTH_CHARS) + "]"` (import from `whitespace_padding`); `mcp_tool_poisoning._ZERO_WIDTH_RE` rebuilt from the same constant (note: its current set lacks U+2060/U+FEFF — converging is a strict coverage improvement, and U+2060 is also independently flagged by TP2's invisible-chars check, which is fine: different rule, different meaning). |
| 119 | + |
| 120 | +## What Goes Where |
| 121 | + |
| 122 | +- **Implementation Steps** (`[ ]` checkboxes): code, tests, README/docs — all in this repo. |
| 123 | +- **Post-Completion** (no checkboxes): threshold tuning against real-world skill corpora, issue/PR follow-ups. |
| 124 | + |
| 125 | +## Implementation Steps |
| 126 | + |
| 127 | +### Task 1: Whitespace padding detector helper module |
| 128 | + |
| 129 | +**Files:** |
| 130 | +- Create: `src/skillspector/nodes/analyzers/whitespace_padding.py` |
| 131 | +- Create: `tests/nodes/analyzers/test_whitespace_padding.py` |
| 132 | + |
| 133 | +- [ ] create `whitespace_padding.py` with `ZERO_WIDTH_CHARS`, `is_padding_char()` (unicodedata category `Z*` + controls + zero-width), threshold constants, `PaddingRun` dataclass, and `summarize_run()` visible-izer (`U+00A0 x82` / `\n x82` rendering) |
| 134 | +- [ ] implement `detect_whitespace_padding(content, *, file_type="other") -> list[PaddingRun]` covering all three signals: vertical blank-line runs (with `followed_by_content`), horizontal in-line runs, contiguous block > 2 KB and > 90%-of-file ratio |
| 135 | +- [ ] implement false-positive guards: Markdown fence-region skip for the horizontal signal, U+FFFD (binary-ish) bail-out, vertical-run/block dedup |
| 136 | +- [ ] write tests: each signal fires at its threshold and not below it (19 blank lines no, 20 yes; 79 ws chars no, 80 yes; 2 KB block boundary) |
| 137 | +- [ ] write tests: Unicode evasion cases — padding made of U+00A0, U+2028/U+2029, U+000B/U+000C, U+3000, and zero-width chars all detected; `summarize_run` renders `U+00A0 x82`-style output |
| 138 | +- [ ] write tests: guards — horizontal run inside a ``` fence not reported for markdown, reported for non-markdown; content with U+FFFD returns no runs; `followed_by_content` true/false distinguished |
| 139 | +- [ ] run `make test-unit` — must pass before task 2 |
| 140 | + |
| 141 | +### Task 2: P9 findings in the prompt-injection analyzer + shared zero-width definition |
| 142 | + |
| 143 | +**Files:** |
| 144 | +- Modify: `src/skillspector/nodes/analyzers/static_patterns_prompt_injection.py` |
| 145 | +- Modify: `src/skillspector/nodes/analyzers/pattern_defaults.py` |
| 146 | +- Modify: `tests/nodes/analyzers/test_static_patterns.py` |
| 147 | + |
| 148 | +- [ ] in `static_patterns_prompt_injection.py`: import the detector, add a P9 block in `analyze()` mapping `PaddingRun` → `AnalyzerFinding` (severity/confidence per the signal table above; `matched_text=run.summary`); add the vendored-filename skip; update module docstring "(P1–P4)" → "(P1–P4, P9)" |
| 149 | +- [ ] rebuild `P2_PATTERNS`' zero-width regex from the shared `ZERO_WIDTH_CHARS` constant (no behavior change — same five chars) |
| 150 | +- [ ] add P9 to `pattern_defaults.py`: `DEFAULT_EXPLANATIONS`, `RULE_ID_TO_CATEGORY` (→ `PatternCategory.PROMPT_INJECTION`), `PATTERN_NAMES` ("Whitespace Padding"), `DEFAULT_REMEDIATIONS` |
| 151 | +- [ ] write tests in `test_static_patterns.py`: a SKILL.md body with 80 blank lines then an injected instruction yields a P9 finding with HIGH severity, correct `start_line` (start of the gap), and a visible-ized `matched_text`; trailing-gap variant yields MEDIUM/0.6; horizontal and ratio variants yield their severities; `*.min.js` path yields no P9 |
| 152 | +- [ ] write test: existing P2 zero-width detection still fires identically after the shared-constant refactor |
| 153 | +- [ ] run `make test-unit` — must pass before task 3 |
| 154 | + |
| 155 | +### Task 3: P9 over MCP manifest description fields |
| 156 | + |
| 157 | +**Files:** |
| 158 | +- Modify: `src/skillspector/nodes/analyzers/mcp_tool_poisoning.py` |
| 159 | +- Modify: `tests/test_mcp_tool_poisoning.py` |
| 160 | + |
| 161 | +- [ ] add `_check_p9_padding(text, source_field) -> list[Finding]` using the shared detector (horizontal + contiguous-block signals; skip per-file ratio); wire it into `node()`'s metadata-text loop for non-identifier fields |
| 162 | +- [ ] rebuild `_ZERO_WIDTH_RE` from the shared `ZERO_WIDTH_CHARS` constant (adds U+2060/U+FEFF coverage to TP1's hidden-text check — strict improvement; note it in the docstring) |
| 163 | +- [ ] write tests: a tool description padded with 100 spaces before an instruction yields a P9 finding naming the source field; a normal multi-sentence description yields none; identifier fields are not scanned |
| 164 | +- [ ] write test: TP1 zero-width behavior unchanged for the original three chars, and now also fires on U+2060/U+FEFF |
| 165 | +- [ ] run `make test-unit` — must pass before task 4 |
| 166 | + |
| 167 | +### Task 4: Verify acceptance criteria |
| 168 | + |
| 169 | +- [ ] verify all issue #20 requirements: three signals implemented, Unicode-category-based classification, shared zero-width definition with P2, visible-ized snippets, line/offset reporting, fenced-code + vendored-file + ratio-confidence FP guards, MCP manifest coverage |
| 170 | +- [ ] adversarial self-check: craft a SKILL.md using each padding char from the issue's evasion list (U+00A0, U+2028, U+2029, U+000C, U+000B, U+3000, zero-width family) and confirm P9 fires on every one via the test suite |
| 171 | +- [ ] run full suite: `make test` (unit + integration) |
| 172 | +- [ ] run a real scan over a fixture skill (`uv run skillspector scan <fixture>` or project equivalent) and eyeball the P9 finding rendering in the report output |
| 173 | + |
| 174 | +### Task 5: [Final] Update documentation |
| 175 | + |
| 176 | +- [ ] add P9 row to the README Prompt Injection table and bump its "(5 patterns)" count and the total pattern count |
| 177 | +- [ ] update CLAUDE.md if a new pattern-authoring convention emerged (shared detector helper pattern) |
| 178 | +- [ ] comment on issue #20 summarizing the implemented signals/thresholds and the P9 id choice (P6–P8 were taken) |
| 179 | +- [ ] move this plan to `docs/plans/completed/` |
| 180 | + |
| 181 | +## Post-Completion |
| 182 | + |
| 183 | +**Manual verification:** |
| 184 | +- Tune thresholds against a corpus of real, benign skills (templates with spacer lines, ASCII-art-heavy READMEs) before relying on the ratio signal at anything above LOW confidence; issue #20 explicitly expects tuning. |
| 185 | +- Security review of the regex/scan performance on pathological inputs (the detector is a linear scan, no backtracking regex, but confirm on a 1 MB all-whitespace file). |
| 186 | + |
| 187 | +**External follow-ups:** |
| 188 | +- Open PR against `NVIDIA/SkillSpector` referencing issue #20; the open questions answered here (single P9 id, MEDIUM/MEDIUM/LOW severities, MCP manifests in scope) should be re-stated in the PR description for maintainer sign-off, since the issue author offered to align on signals/thresholds before a PR. |
0 commit comments