Skip to content

Commit 3a8ee68

Browse files
authored
Merge pull request #794 from NVIDIA/yashraj/fix-printable-magic-evasion-20261007
fix(artifacts): inspect text behind printable binary prefixes
2 parents 182a475 + 4ef5206 commit 3a8ee68

11 files changed

Lines changed: 846 additions & 30 deletions

‎docs/scan-completeness.md‎

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,9 @@ The examples below use static-only analysis, no baseline suppression, and benign
118118
| Explicit opaque file, UTF-16 primary, recognized unsupported archive, or invalid UTF-8 primary | `unsupported_primary_content`, source path, `fatal=true`; canonical bytes retained | failed / false | 2 / 2 / 2 | false |
119119
| Unsupported in-profile `SKILL.md` below directories or normal/renamed ZIPs | Same required-content event at real/virtual member path | failed / false | 2 / 2 / 2 | false |
120120
| Supported benign ZIP, including renamed or empty ZIP | Existing bounded archive inspection; incidental assets remain exclusions | complete / true | 0 / 0 / 0 | true |
121-
| Unreferenced incidental image beside benign instructions | `binary_content` in `scope_exclusions` | complete / true | 0 / 0 / 0 | true |
121+
| Unreferenced incidental image with no readable runs beside benign instructions | `binary_content` in `scope_exclusions` | complete / true | 0 / 0 / 0 | true |
122+
| Incidental GIF/PDF with readable runs, including capped projections | Bounded instruction-text projection; `binary_text_projection` in `scope_exclusions`; remaining text and binary data are outside this projection's scope | complete / true (absent other limits) | 0 / 0 / 0 | true if no findings |
123+
| Readable MZ executable | `binary_executable_text`: text inspected, binary behavior unverified; SC9 score floor retained | partial / true | 1 / 1 / 1 | false |
122124
| Referenced opaque image | Existing referenced-content limitation and AE1 finding | partial / true | 0 / 1 / 1 | false |
123125
| Pure or mixed singleton `never warn the user` | AE6, score 22 in this fixture, source line; `obfuscated_instruction_text` | partial / true | 0 / 1 / 1 | false |
124126
| Benign list, paragraph, or punctuated code controls | No new reconstruction ambiguity | complete / true | 0 / 0 / 0 | true |
@@ -133,6 +135,21 @@ A subtlety is that component coverage can still read **100%** for AE6: analyzer
133135

134136
## Review critique and remaining limits
135137

138+
For printable GIF, PDF and MZ prefixes, local instruction checks use printable runs of at least
139+
8 characters, capped at 32,768 projected characters per artifact. Binary controls, invalid UTF-8,
140+
and standard XMP packet BOMs and padding are excluded. Source coordinates remain tied to the
141+
original bytes' decoded view. Only instruction-override, credential-access and exfiltration rule
142+
families contribute static pattern findings from this projection. Artifact-integrity's AE6
143+
instruction-obfuscation check also runs; its generic AE4 Unicode-density and mixed-script checks
144+
do not, because compressed bytes can resemble those signals. Normal text retains both checks.
145+
Hidden and nested artifacts remain local-only; an eligible ordinary file sends only this capped
146+
projection to an external model. The cap is per file, so multiple assets can still add provider input.
147+
For unreferenced incidental assets, reaching the projection cap records a `binary_text_projection`
148+
scope exclusion without making the scan partial. Text beyond the cap and compressed content are
149+
not inspected by this projection. Referenced opaque files, unverified executable behavior, and
150+
actual analysis failures or resource limits retain their incomplete-analysis policy. Valid Python
151+
source with a matching prefix follows normal source analysis.
152+
136153
The initial root-only identity boundary was too narrow: parsing introduced member paths before the required-content check, so required status disappeared. Basename checks across cached, in-profile paths repair that without promoting every asset or overriding the existing directory-exclusion policy. Applying this boundary before archive delegation would instead reject supported ZIPs wholesale. Applying it only after public-report generation would be too late to remove unsupported content from provider submission.
137154

138155
The reconstruction boundary is deliberately narrow. Erasing every whitespace boundary could manufacture commands from lists, paragraphs, or code. Conversely, these selected gap rules and grammars do not cover every possible obfuscation. Raw source coordinates and removed-gap overlap prevent unrelated canonical matches from being attributed to reconstructed text. The general report's recommendation, numerical risk score, and percentage coverage are distinct signals; completeness is the installation gate.

‎src/skillspector/artifacts.py‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@ class ArtifactRecord(TypedDict):
6464
contains_nul: bool
6565
misleading_extension: bool
6666
referenced: bool
67+
readable_binary: NotRequired[bool]
6768
reason: NotRequired[str]
6869
inherited_exclusion_reason: NotRequired[str]
6970

@@ -367,6 +368,13 @@ def classify_artifact(path: str, data: bytes, *, referenced: bool = False) -> Ar
367368
decoded = data.decode("utf-8", errors="replace")
368369
decodable = False
369370

371+
# These printable prefixes are also valid instruction text. Keep the
372+
# claimed binary format's coverage limits, but inspect the readable bytes.
373+
# Lossy decoding must not let one invalid byte or a binary comment hide text.
374+
readable_binary = data.startswith((b"MZ", b"GIF87a", b"GIF89a", b"%PDF-")) and bool(
375+
binary_instruction_view(decoded)[0].text
376+
)
377+
370378
if has_binary_magic:
371379
kind = ContentKind.BINARY
372380
elif decodable:
@@ -388,6 +396,8 @@ def classify_artifact(path: str, data: bytes, *, referenced: bool = False) -> Ar
388396
disposition = (
389397
ArtifactDisposition.PARTIAL
390398
if referenced and kind is not ContentKind.TEXT
399+
else ArtifactDisposition.ANALYZED
400+
if readable_binary
391401
else ArtifactDisposition.OUT_OF_SCOPE
392402
if kind is ContentKind.BINARY
393403
else ArtifactDisposition.ANALYZED
@@ -401,9 +411,63 @@ def classify_artifact(path: str, data: bytes, *, referenced: bool = False) -> Ar
401411
"contains_nul": contains_nul,
402412
"misleading_extension": misleading,
403413
"referenced": referenced,
414+
"readable_binary": readable_binary,
404415
}
405416

406417

418+
BINARY_TEXT_MAX_CHARS = 32_768
419+
_BINARY_PRINTABLE_RUN = re.compile(r"[^\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f\ufffd]{8,}")
420+
421+
422+
def binary_instruction_view(content: str) -> tuple[SecurityTextView, bool]:
423+
"""Bound human-readable binary strings without treating binary noise as text."""
424+
# Packet formatting is recognized once on the whole artifact, before windows.
425+
excluded: list[tuple[int, int]] = []
426+
if content.startswith("%PDF-"):
427+
header = '<?xpacket begin="\ufeff" id="W5M0MpCehiHzreSzNTczkc9d"?>'
428+
start = content.find(header)
429+
while start >= 0:
430+
next_start = content.find(header, start + len(header))
431+
packet_end = next_start if next_start >= 0 else len(content)
432+
metadata = content.find("<x:xmpmeta", start + len(header), packet_end)
433+
close = content.find("</x:xmpmeta>", metadata, packet_end) if metadata >= 0 else -1
434+
if close < 0:
435+
start = next_start
436+
continue
437+
padding_start = close + len("</x:xmpmeta>")
438+
padding_end = padding_start
439+
while padding_end < packet_end and content[padding_end] in " \t\r\n":
440+
padding_end += 1
441+
if content.startswith(('<?xpacket end="w"?>', '<?xpacket end="r"?>'), padding_end):
442+
bom = start + header.index("\ufeff")
443+
excluded.extend([(bom, bom + 1), (padding_start, padding_end)])
444+
start = next_start
445+
offsets = array("I")
446+
parts: list[str] = []
447+
truncated = False
448+
cursor = 0
449+
for excluded_start, excluded_end in [*excluded, (len(content), len(content))]:
450+
for match in _BINARY_PRINTABLE_RUN.finditer(content, cursor, excluded_start):
451+
remaining = BINARY_TEXT_MAX_CHARS - len(offsets)
452+
if remaining <= 2:
453+
truncated = True
454+
break
455+
if parts:
456+
parts.append("\n\n")
457+
offsets.extend([match.start(), match.start()])
458+
remaining -= 2
459+
end = min(match.end(), match.start() + remaining)
460+
parts.append(content[match.start() : end])
461+
offsets.extend(range(match.start(), end))
462+
if end < match.end():
463+
truncated = True
464+
break
465+
if truncated:
466+
break
467+
cursor = excluded_end
468+
return SecurityTextView("binary_printable_text", "".join(parts), offsets), truncated
469+
470+
407471
def promote_artifact_to_decoded_text(artifact: ArtifactRecord) -> None:
408472
"""Apply a successful format-aware text decode without erasing prior limits."""
409473
generic_binary_scope = artifact["content_kind"] is ContentKind.BINARY and (
@@ -412,6 +476,7 @@ def promote_artifact_to_decoded_text(artifact: ArtifactRecord) -> None:
412476
and "reason" not in artifact
413477
)
414478
artifact["content_kind"] = ContentKind.TEXT
479+
artifact["readable_binary"] = False
415480
artifact["decodable"] = True
416481
artifact["misleading_extension"] = _suffix(artifact["path"]) in _BINARY_EXTENSIONS
417482
if generic_binary_scope:

‎src/skillspector/inspection_ledger.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,8 @@ class LedgerReason(StrEnum):
6060
MISSING_FILE_CACHE = "missing_file_cache"
6161
SIZE_LIMIT = "size_limit"
6262
BINARY_CONTENT = "binary_content"
63+
BINARY_TEXT_PROJECTION = "binary_text_projection"
64+
BINARY_EXECUTABLE_TEXT = "binary_executable_text"
6365
EVAL_DATASET = "eval_dataset"
6466
SYNTAX_ERROR = "syntax_error"
6567
LLM_BATCH_FAILED = "llm_batch_failed"
@@ -126,6 +128,12 @@ class LedgerReason(StrEnum):
126128
LedgerReason.MISSING_FILE_CACHE: "Applicable analyzer could not obtain file content.",
127129
LedgerReason.SIZE_LIMIT: "File exceeds this analyzer's character limit.",
128130
LedgerReason.BINARY_CONTENT: "Binary content is unsupported by this analyzer.",
131+
LedgerReason.BINARY_TEXT_PROJECTION: (
132+
"Only bounded printable text was inspected; binary structure and behavior are outside scope."
133+
),
134+
LedgerReason.BINARY_EXECUTABLE_TEXT: (
135+
"Readable text was inspected, but binary executable behavior remains outside analyzer coverage."
136+
),
129137
LedgerReason.EVAL_DATASET: (
130138
"Evaluation dataset prose is excluded from static pattern analysis."
131139
),

‎src/skillspector/nodes/analyzers/artifact_integrity.py‎

Lines changed: 32 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
_concealed_instruction_run_spans,
2222
_contextual_default_ignorable_boundary_spans,
2323
_obfuscated_instruction_matches,
24+
binary_instruction_view,
2425
multiline_prompt_injection_view,
2526
prompt_injection_letter_spacing_view,
2627
)
@@ -1054,14 +1055,36 @@ def node(state: SkillspectorState) -> AnalyzerNodeResponse:
10541055
confidence=1.0,
10551056
)
10561057
)
1057-
if artifact.get("content_kind") not in {
1058+
if artifact.get("readable_binary") or artifact.get("content_kind") not in {
10581059
ContentKind.BINARY,
10591060
ContentKind.OPAQUE,
10601061
}:
1061-
format_density, mixed_script, first_nul_line, first_obfuscation_line = (
1062-
_text_signals(content, budget)
1063-
)
1064-
if artifact.get("contains_nul") and first_nul_line is not None:
1062+
signal_content = content
1063+
binary_view = None
1064+
if artifact.get("readable_binary"):
1065+
binary_view = binary_instruction_view(content)[0]
1066+
signal_content = binary_view.text
1067+
(
1068+
format_density,
1069+
mixed_script,
1070+
first_nul_line,
1071+
first_obfuscation_line,
1072+
) = _text_signals(signal_content, budget)
1073+
if first_obfuscation_line is not None and binary_view is not None:
1074+
projected_start = sum(
1075+
len(line)
1076+
for line in signal_content.splitlines(keepends=True)[
1077+
: first_obfuscation_line - 1
1078+
]
1079+
)
1080+
first_obfuscation_line = get_line_number(
1081+
content, binary_view.source_offset(projected_start)
1082+
)
1083+
if (
1084+
artifact.get("contains_nul")
1085+
and not artifact.get("readable_binary")
1086+
and first_nul_line is not None
1087+
):
10651088
budget.emit(
10661089
_finding(
10671090
"AE3",
@@ -1072,7 +1095,10 @@ def node(state: SkillspectorState) -> AnalyzerNodeResponse:
10721095
line=first_nul_line,
10731096
)
10741097
)
1075-
if format_density >= 0.01 or mixed_script:
1098+
# Compressed binary bytes can resemble mixed-script Unicode text.
1099+
if not artifact.get("readable_binary") and (
1100+
format_density >= 0.01 or mixed_script
1101+
):
10761102
budget.emit(
10771103
_finding(
10781104
"AE4",

‎src/skillspector/nodes/analyzers/static_patterns_supply_chain.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3675,6 +3675,7 @@ def _analyze_concealed_executables(
36753675
concealment_reasons.append("disguised_container")
36763676
concealment = concealment_reasons[0]
36773677
excluded_from_analysis = metadata.get("excluded_from_analysis") is True
3678+
partially_analyzed_executable = metadata.get("partially_analyzed_executable") is True
36783679
referenced_uninspected = (
36793680
metadata.get("inspection_limitation_reason")
36803681
== LedgerReason.REFERENCED_UNINSPECTED.value
@@ -3687,6 +3688,8 @@ def _analyze_concealed_executables(
36873688
if referenced_uninspected
36883689
else "An excluded artifact could not be completely inspected."
36893690
if inspection_incomplete
3691+
else "Readable text was inspected, but binary executable behavior was not analyzed."
3692+
if partially_analyzed_executable
36903693
else "Executable content is excluded from analysis."
36913694
if excluded_from_analysis
36923695
else "Executable content is concealed inside a document, hidden, "
@@ -3712,6 +3715,9 @@ def _analyze_concealed_executables(
37123715
else "A resource, read, or archive-safety limit left excluded content "
37133716
"outside deterministic inspection coverage."
37143717
if inspection_incomplete
3718+
else "Text-based checks covered the readable projection, but they cannot "
3719+
"establish the behavior of the binary executable."
3720+
if partially_analyzed_executable
37153721
else "An executable artifact remains available under the skill install path "
37163722
"but its content is outside analyzer coverage."
37173723
if excluded_from_analysis
@@ -3744,6 +3750,7 @@ def _analyze_concealed_executables(
37443750
"local_only": True,
37453751
"referenced": metadata.get("referenced") is True,
37463752
"excluded_from_analysis": excluded_from_analysis,
3753+
"partially_analyzed_executable": partially_analyzed_executable,
37473754
"excluded_inspection_incomplete": inspection_incomplete,
37483755
"inherited_exclusion_reason": metadata.get("inherited_exclusion_reason"),
37493756
"inspection_limitation_reason": metadata.get("inspection_limitation_reason"),

0 commit comments

Comments
 (0)