Skip to content

feat(macho): report per-slice encryption status in inspect_macho - #5359

Merged
gensokyo-dev merged 31 commits into
morluto:mainfrom
chordpli:feat/macho-encryption-info
Oct 29, 2026
Merged

gensokyo-dev merged 31 commits into
morluto:mainfrom
chordpli:feat/macho-encryption-info

Conversation

@chordpli

@chordpli chordpli commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #4306. inspect_macho now reports the declared LC_ENCRYPTION_INFO(_64) state of every slice, so analysts can distinguish a recorded ciphertext range from program bytes before interpreting disassembly. Output includes observed command fields, CPU identity, slice/input offsets, segment/section overlaps and VM mappings. CLI and MCP share the same provider/application facts; inspect_analysis_view exposes the paged macho encryption facet with retained Evidence.

This is static metadata inspection within the existing native provider. Status derives only from load commands: no entropy guess, payload-encryption test, target execution, decryption or mutation. Artifact inventory's separate container-encryption meaning remains intact. absent requires a complete walk; malformed/unknown states preserve their reason and earlier commands. Older retained Evidence remains readable.

Implementation and continuation

The contributor's history is preserved. Common container/slice framing owns identity and load-command parsing; the strict existing reader retains its defect precedence. Only encryption/segment commands are retained. Mapping uses the existing shared record allowance and checks the framing bound before retaining segment commands.

Review found that a failed early mapping still spent its entire reservation, dropping a healthy later slice's VM/section facts. The producing boundary now commits the reservation only after mapping succeeds. Four tiny double-slice regressions reproduce the original failure for missing section bytes, VM-size mismatch, address-space overflow and command-width mismatch; all pass after the fix. The existing successful-mapping test still proves that retained results consume the shared budget. No downstream repair or parallel interpretation was added.

The bound descriptor and target SHA remain checked around native tools. Cleanup retains the exact owner and primary failure. Native provenance verification now requires complete per-slice absence through real CLI/MCP, and the owned encryptable-image CLI test is included in the macOS CI lane. This tests declared cryptid=1 metadata; it is not a genuine FairPlay sample.

Executed validation

Continuation on Linux, Node 24.22.1 /npm 11.16.0. Runtime commit 26280c746; head 0eba2e7fa additionally merges main d3ecf6800 (website-only changes) and clarifies documentation/comments.

  • 199 tests pass /2 macOS-only skips across 10 selected files, including reader characterization, mapping/status semantics, retained Evidence/view/schema behavior, target replacement, cancellation and cleanup ownership. Four new budget cases failed first; the five source files now pass 155 tests.
  • check:fast, formatting of the changed paths, docs (9 README tests), MCP catalog and discovery (2,834 test files exactly once) pass. Docs reran after the website-only main merge and final documentation changes. git diff --check passes.
  • Real files: official ripgrep 14.1.1 arm64 and x86_64 macOS binaries, downloaded archives checked against published SHA-256 files. LLVM lipo 14.0.6 creates an owned universal combination. Sizes: 4,375,216 /5,162,504 /9,552,560 bytes. Independent LLVM otool/lipo header/command walks confirm all four represented slices have no encryption command; both revisions return identical complete facts.
  • This continuation cannot run real macOS CLI/MCP locally. The original contributor reported macOS 26.5.2 native-tool verification, including CLI/MCP, at c2c2e93fc; it is attributed rather than claimed as rerun here. Real Apple CI now passes: 402 tests pass /5 skip in 29 files, plus all artifact-format/metadata verifiers. Its real clang/lipo/otool/codesign encryption CLI test passes (5.980 s including fixture preparation); the actual stdio CLI/MCP provenance verifier confirms complete slice absence and unchanged target bytes. The separate macOS-only summary fixture is not selected in this lane. No genuine FairPlay-encrypted input has been verified.

Final CI 40851338708, merge d60f8bc /main d3ecf6800: source tests 21,846 pass /16 skip in 1,176 files. Static checks, build and docs pass; the first run found an additional CI integration omission: the new Apple test path was in the workflow but absent from its classifier owner. Final head 5ffb40d26 adds that Apple scope entry; all 310 scope/workflow tests and type/lint pass. Runtime code is unchanged. This keeps future test-only edits from silently skipping real-host verification.

Two other failures in that run are baseline text assertions: bundleComparisonCli.test.ts:223 and flutterMcp.test.ts:259 still expect legacy colon-delimited errors. A clean worktree on pure main d3ecf6800 reproduces both identical failures (17 pass /2 fail, npm run test:focused -- tests/boundary/cli/bundleComparisonCli.test.ts tests/boundary/mcp/flutterMcp.test.ts). These unrelated assertions were not changed. Matrix fail-fast and the superseding push cancel unfinished jobs; those jobs are not counted as passing. The final run passes shard 7; shard 3 fails only the unchanged bundleComparisonMcp.test.ts:168 legacy message assertion. Pure main d3ecf6800 reproduces this too (8 pass /1 identical failure). The other six shards are matrix-cancelled and unverified in this run, not reported as passing. Final-head Greptile gives 5/5 with no unresolved threads. Platform results are available in the linked run. The operator explicitly permits bypass for demonstrated main failures; the PR-owned scope omission has been fixed and locally validated separately.

Completed parser-task measurements

Before 9bd7dddcd is the original contribution integrated on the same main base; after is runtime 26280c746. Two alternating/reversed repetitions, fresh Node processes, warm OS caches. Each first-run task answers for all three real files; the repeated task answers the same three questions six times in one process, including the first pass. Every read uses a fresh descriptor, verifies unchanged file metadata and closes it. No new cache is retained.

Completed task Estimated request +result tokens Elapsed CPU Peak process-family RSS
Before, first three files 123 +242 82.7–83.6 ms 60–70 ms 52.7–66.1 MiB
After, first three files 123 +242 83.1 ms 60 ms 50.7–50.9 MiB
Before, 18 repeated answers 738 +1,452 111–128 ms 110–120 ms 70.1–107.1 MiB
After, 18 repeated answers 738 +1,452 104–112 ms 100 ms 75.0–112.5 MiB

All answer hashes and read counts match: 15 reads /9,560,612 bytes per three-file task. The shared JVM discriminator reads small CAFEBABE containers whole; thin images read their structural metadata. Framing/read costs remain bounded; no speedup is claimed from these short runs.

These measure the shared reader task with a 4,096-record allowance, not whole CLI/MCP agent tasks. Native commands, MCP discovery/schemas/Evidence wrappers and true cold filesystem caches are unmeasured locally. Tokenx 1.3.0 estimates serialized requests/results, not billed tokens. RSS is a 10 ms process-group sample and may miss brief peaks; CPU includes the complete Node task and excludes the sampler. A 256 MiB Node heap /512 MiB family /60 s guard bounds each run. All eight runs complete without a guard stop or survivor; downloaded inputs and measurement artifacts remain outside Git.

Author readiness

Human owner: @chordpli

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

…d readers

Split readMachoImage into readMachoContainer, which classifies a file as
thin, FAT or not Mach-O and keeps FAT range issues per slice, and
readSliceLoadCommands, which reads one slice header and frames its load
commands, returning every command framed before a failure.

readMachoImage keeps its strict policy on top of them: commands framed
before a framing defect still decode first, an unsupported FAT slice
still yields to a later malformed one, and the first FAT range issue is
still reported. A characterization test pins that precedence for inputs
with several defects; it passes against both the previous and the new
implementation.

Part of morluto#4306.
inspect_macho now reports the LC_ENCRYPTION_INFO(_64) state of every
slice in a new `encryption` collection, read in process from load
commands without reading code bytes.

- Each slice is encrypted, not-encrypted, absent, malformed or unknown.
  Absent requires a completed load-command walk; malformed and unknown
  carry a reason and keep commands read before the failure.
- The named range is end-exclusive in slice and input-file coordinates
  and maps to file-backed segment fragments, VM addresses and the
  overlapping nonempty sections.
- A ciphertext range adds a limitation naming its input-file bytes.
- The in-process read and the native tools are bracketed by one regular
  file descriptor and stat identity checks; a change reports
  artifact_changed. A missing or inaccessible target keeps the tool
  facts and reports encryption as non-exhaustive, never absent.
- Retained Evidence recorded before this field is parsed by the owning
  schema and reports encryption as unknown in the macho view and in
  managed/native verification.

Fixes morluto#4306
@chordpli
chordpli marked this pull request as ready for review October 20, 2026 05:53
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-20T05:53:38.898349Z 2b41ca6 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

@greptile-apps

greptile-apps Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5 Tier: apex

[Medium impact] The PR appears safe to merge; the latest change correctly selects the existing macOS test job.

Summary

Adds per-slice encryption facts to inspect_macho, including declared ranges, memory addresses, and paged retained Evidence.

  • Mach-O inspection reports each slice’s declared encryption state and byte range.
  • MCP pages expose Mach-O encryption facts and keep older Evidence readable.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A["Bound target file"] --> B["Shared container and command reader"]
  B --> C["Per-slice encryption facts"]
  C --> D["inspect_macho report"]
  D --> E["CLI and MCP Evidence"]
  E --> F["Summary and paged encryption view"]
Loading

Reviews (24) · Last reviewed commit: "fix(ci): own the Mach-O encryption test ..." · Reviewed by Greptile

Comment thread src/native/NativeMacOSProvider.ts Resolved
…e errors

- Frame load commands through a visitor instead of retaining every
  command, restoring the pre-refactor memory profile (2M eight-byte
  commands: peak heap 370 MiB -> 0.9 MiB).
- Confirm a large FAT container from a valid slice's Mach-O header so a
  sibling's range fault stays per slice instead of hiding healthy slices.
- Name only sections whose bytes the range and their segment both cover.
- Throw inspection failures inside the owned descriptor callback so a
  close failure keeps them as its primary error.
…-info

# Conflicts:
#	src/artifacts/apple/MachoLoadCommandReader.ts
…-info

# Conflicts:
#	src/artifacts/apple/MachoLoadCommandReader.ts
#	src/domain/apple/machoContainer.ts
#	src/native/NativeMacOSProvider.ts
#	src/native/NativeMachoInspection.ts
@chordpli

Copy link
Copy Markdown
Contributor Author

@morluto I merged current main into this branch again (c128c725b) and re-ran the E2E against the merged code. Could you take a look when you have time?

Conflict resolution

  • MachoLoadCommandReader.ts: fix(apple): read LC_LAZY_LOAD_DYLIB_INFO dependency payloads #5650 added LC_LAZY_LOAD_DYLIB_INFO handling inside the load-command loop that this PR moved into frameLoadCommands. I kept the shared framing and moved the lazy-info steps into its visitor: segment spans, the 1024-dependency limit, the unreadable-segment check, and the payload reads in appendDependencies. readSliceCommandTable now takes CPU identity from readMachoHeaderPrefixFacts (44521d3). readExact is kept because the lazy payload reader still uses it.
  • machoContainer.ts: kept the 497037e rule, where a truncated slice header needs a read and a non-Mach-O slice header makes the container not-mach-o. It uses isMachoHeaderMagic.
  • NativeMacOSProvider.ts: the encryption read wrapper now also forwards acceptNonZeroIf.
  • NativeMachoInspection.ts: the conflicting-LC_UUID check runs before the encryption read, so a rejected inspection does not read the file.

Test changes

  • Added readEncryption: unreadEncryption to the new lazy-info test in NativeMachoInspection.test.ts.
  • One assertion in this PR's defect-precedence test used an all-zero slice as the malformed FAT slice. After 497037e a FAT that mixes in a non-Mach-O slice is not-mach-o, so that input no longer reaches the reader. The test now uses a slice with a valid header whose load commands extend past the slice. It still checks that the malformed slice wins over the big-endian one in either order.

Validation at c128c725b

  • npm run check:fast, npm run docs:check: pass
  • npm run test:local -- 12 Mach-O source files: 267 passed
  • npm run test:focused -- 14 Mach-O CLI/MCP/provider boundary files, including the new missing-UUID and UUID-less tests: 84 passed

E2E: built CLI, isolated environment (macOS 26.5.2 arm64, Apple clang 21.0.0)

Fixture Command Result
/bin/ls (x86_64 + arm64e) inspect-macho exit 0; both slices absent
ios_enc0 (-Wl,-encryptable) inspect-macho exit 0; not-encrypted, plaintext 16384-32768
ios_enc1 (cryptid patched to 1, re-signed) inspect-macho exit 0; encrypted, ciphertext 16384-32768, ciphertext limitation
ios_fat (arm64 cryptid 1 + arm64e cryptid 0) inspect-macho exit 0; arm64 encrypted at file 32768-49152, arm64e not-encrypted
fat3 (x86_64 + arm64_32 + arm64) inspect-macho exit 0; x86_64 absent, arm64_32 (LC_ENCRYPTION_INFO, 20) and arm64 encrypted
nouuid (-Wl,-no_uuid) inspect-macho exit 0; absent; missing-LC_UUID limitations from dyld_info
ios_nouuid_enc1 (no UUID, cryptid 1) inspect-macho exit 0; encrypted, ciphertext 16384-32768; missing-LC_UUID limitations
lazy (-Wl,-lazy_library, macOS 27.0 target) inspect-macho exit 0; LC_LAZY_LOAD_DYLIB_INFO listed; absent
lazy trace-dylib-resolution exit 0; @rpath/libl.dylib via LC_LAZY_LOAD_DYLIB_INFO; coverage complete
lazy with the dylib absent trace-dylib-resolution exit 0; lazy-load-unresolved finding
ios_lazy_enc1 (lazy info + cryptid 1) inspect-macho exit 0; encrypted, ciphertext 16384-32768
ios_lazy_enc1 trace-dylib-resolution exit 0; @rpath/libl.dylib via LC_LAZY_LOAD_DYLIB_INFO
fat.o (arm64 + arm64_32 objects) inspect-macho exit 0; both slices absent
arm64_32.o (thin object) inspect-macho exit 1; unsupported Mach-O architecture (#5713, unchanged by this PR)

otool -l agrees with the reported encryption commands and lazy-info payload offsets. The lazy payload at dataoff 49304 holds @rpath/libl.dylib.

Not rerun at this commit: full npm run check:pr and the stdio MCP E2E. The MCP summary boundary test passed. No real FairPlay-encrypted binary was available.

…-info

# Conflicts:
#	src/native/NativeMacOSProvider.ts
…-info

# Conflicts:
#	src/domain/managed/managedNativeVerificationMatch.ts
Comment thread docs/native-investigation.md Resolved
chordpli and others added 3 commits October 25, 2026 21:05
mapRange gave every file-backed segment fragment VM addresses, even for a
segment with no VM size, whose file bytes dyld never maps, and past vmsize
when a segment declared more file bytes than memory.

Read vmsize with the segment. A segment with no VM size keeps its fragment
with null VM addresses. A file size beyond the VM size, or a VM range past
the address space, makes the mapping unknown with a reason while the slice
keeps its encryption status.

The docs also claimed the encryption read never reads code bytes. Like
every Mach-O container read, a 0xcafebabe input of at most 16 MiB is read
whole to rule out a JVM class file; say so.
Main's withArtifactRegularFile now hands callers the admitted handle and
its initial stats. Use those facts instead of re-stating the handle, and
pass the legacy Evidence payload straight to createEvidence.
@chordpli

Copy link
Copy Markdown
Contributor Author

I merged current main (1173c0a9d) into this branch again (3bd154187). The merge applied cleanly but stopped compiling, so I added a fix commit (611ae1e02).

What broke

  • a891bb80a changed the withArtifactRegularFile callback. It now receives the admitted handle and its initial stats as { handle, initial }. NativeMacOSProvider still called file.stat({ bigint: true }) on the old descriptor shape (TS2339/TS2740).
  • The fix reads encryption with initial.size and compares initial against the post-read handle.stat() and lstat(path). All three are number stats, so sameRegularFileState compares like with like. The extra stat before the read is gone.
  • The new rea/no-evidence-json-preadmission rule flagged jsonValueSchema.parse(legacy) in managedNativeVerificationSource.test.ts. The legacy payload now goes straight to createEvidence.

Validation at 611ae1e02 (macOS 26.5.2 arm64, Node 24.22.1)

  • npm run check:fast: pass
  • npm run test:local -- 8 Mach-O/managed source files: 136 passed
  • npm run test:focused -- encryption binding, CLI and MCP summary boundary files: 11 passed

E2E: built CLI rea inspect-macho --json

Fixture Result
/bin/ls exit 0; x86_64 and arm64e absent
ios_enc0 (-Wl,-encryptable) exit 0; not-encrypted, plaintext 16384-32768
ios_enc1 (cryptid patched to 1, re-signed) exit 0; encrypted, ciphertext 16384-32768, ciphertext limitation
ios_fat (arm64 cryptid 1 + arm64e cryptid 0) exit 0; arm64 encrypted at file 49152-65536, arm64e not-encrypted at 114688-131072

otool -l/otool -fv agree: arm64 slice at offset 32768 with cryptoff 16384, cryptsize 16384, cryptid 1.

E2E: real stdio MCP server on ios_fat

open_binary ok
full encryption: [["arm64","encrypted","ciphertext",49152,65536],["arm64e","not-encrypted","plaintext",114688,131072]] total 2 exhaustive true
ciphertext limitations: 1
summary encryption collection: {"returned":2,"total":2,"exhaustive":true} bytes full/summary 12282 3802
facet page rows: 2 next_offset null equal to full: true

Not rerun at this commit: full npm run check:pr. No real FairPlay-encrypted binary was available.

Comment thread src/native/NativeMacOSProvider.ts Resolved
The file owner keeps the completed callback value when the descriptor close
fails or cancellation arrives during close. inspect_macho now validates that
value and retains the report as the error's partial observation, as
inspect_signature does, instead of returning only the cleanup or
cancellation error.
# Conflicts:
#	src/artifacts/apple/MachoLoadCommandReader.ts
#	src/domain/apple/machoContainer.ts
# Conflicts:
#	src/artifacts/apple/MachoLoadCommandReader.ts
#	src/domain/analysisView/machoView.ts
#	src/domain/apple/machoContainer.ts
#	src/domain/managed/managedNativeVerificationSource.ts
#	src/native/NativeMachoInspection.ts
# Conflicts:
#	src/artifacts/apple/MachoLoadCommandReader.ts
main's new dyld refusal cases build NativeMachoInspectionContext without
the encryption reader this branch requires.
main's facet-ownership case builds inspect_macho Evidence without the
encryption coverage this branch requires; the fixture now carries one
slice row and the loop checks the encryption page like the other facets.
A slice retains at most two encryption load commands, since a second one
already makes it malformed; the reason keeps the full count. Range mapping
in every slice shares one budget of segment and section records, charged
from each segment command's capacity before it is decoded. A slice past
the budget keeps its encryption status and reports an unknown mapping.
Segment commands are collected during framing only while their mapping
records still fit in the budget shared by all slices, and the budget is
charged only when the slice maps its range.
@gensokyo-dev

Copy link
Copy Markdown
Collaborator

Final CI audit for 5ffb40d26d0bb7e9afbe50570bfc25106b11b65f:

  • Real Apple CLI/MCP E2E passes: real metadata verifiers plus 402 pass /5 skip in 29 files. The owned clang/lipo/otool/codesign encryption CLI case passes; the actual stdio MCP/native provenance workflow verifies complete per-slice absence and unchanged bytes. This is declared metadata verification, not a genuine FairPlay sample.
  • Final CI source tests: 21,846 pass /16 skip in 1,176 files. Static checks, build, docs, APK signature, Windows native-control build and the selected Linux x64 package baseline pass. Greptile gives 5/5 on the exact head, with all ten threads resolved.
  • Shard 3's only failure is unchanged bundleComparisonMcp.test.ts:168, expecting the old left_bundle_path: ... prose instead of structured field issues. A clean worktree at pure main d3ecf6800015f2183a298d78ffb30a266a9efd2a reproduces the identical failure: npm run test:focused -- tests/boundary/mcp/bundleComparisonMcp.test.ts, 8 pass /1 fail. See the baseline assertion and failed shard.
  • Shard 7 passes; the other six shards are fail-fast cancellations, not passing verification. This limit is retained in the PR body.
  • The PR-owned CI omission was fixed at the classifier owner in this final head. All 310 scope/workflow tests and type/lint pass locally, including the invariant that rejected the missing Apple ownership. Runtime code did not change in that follow-up. Earlier bundle-CLI/Flutter failures also reproduce on pure main (17 pass /2 fail) and were left outside this change.

The operator explicitly authorized bypass for demonstrated main failures. The bypass covers that baseline assertion and its dependent/cancelled CI outcomes, with no known PR bug left unresolved. Original contributor readiness declarations are preserved unchanged; no new human-review attestation was made.

@gensokyo-dev
gensokyo-dev merged commit 860fdd8 into morluto:main Oct 29, 2026
34 of 43 checks passed
@chordpli
chordpli deleted the feat/macho-encryption-info branch October 30, 2026 01:41
@morluto morluto mentioned this pull request Nov 11, 2026
3 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Report per-slice LC_ENCRYPTION_INFO status and ciphertext ranges in inspect_macho

2 participants