Skip to content

chore: spell control characters as escapes in tracked sources - #1784

Merged
openwong2kim merged 2 commits into
openwong2kim:mainfrom
snowyukitty:publish/source-control-bytes-20261004
Oct 4, 2026
Merged

openwong2kim merged 2 commits into
openwong2kim:mainfrom
snowyukitty:publish/source-control-bytes-20261004

Conversation

@snowyukitty

@snowyukitty snowyukitty commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Ten source files and docs/phone-client-contract.md contain raw NUL bytes, written straight into string literals, one comment and one piece of inline code. With default tool settings that has three effects:

  • Code search misses these files. ripgrep treats a file containing a NUL as binary and stops searching it. On current main, a default directory search (rg "export class WebTerminalServer" src/daemon/web) returns nothing. The same is true for private advertisedHosts( (line 9020) and for UNKNOWN_SUBJECT_KEY in deviceAudit.ts. grep -r prints "Binary file matches" instead of the line. A person or agent who searches with plain rg or grep -r will not find the 9,819-line WebTerminalServer.ts, the phone client contract, or the rest of the files below. Tools that pass --text are not affected.
  • Some changes cannot be reviewed. In deviceAudit.ts the NUL is within git's first 8000 bytes, so git's binary heuristic applies. git diff, git log -p and GitHub's PR view all show "Binary files differ". For example, the GitHub API returns no patch for that file in 74b876c.
  • Line endings drift. The repository has no .gitattributes, so binary detection kept parseEntry.ts and its test out of line-ending normalization. They are stored with CRLF; the rest of the repository is LF.

This PR spells those characters as escapes and adds a test that keeps raw control bytes out.

Changes

  • Raw NUL becomes \u0000, the spelling hostSearch.ts and turnEvents.ts already use for the same code-block marker. This covers 10 sources: cache keys, separators, test fixtures and a comment. In the phone client contract, the inline-code marker now reads \u0000code:<n>\u0000, with a note that \u0000 is a NUL character.
  • Raw ESC, BEL and US bytes in 9 more files become \x1b, \x07 and \x1f. Most are test fixtures for terminal sanitizers; one is a NIC key in SettingsPanel.tsx.
  • New scripts/__tests__/sourceControlBytes.test.mjs. Every tracked regular file on an extension/name allowlist must be free of C0 control bytes other than tab, LF and CR. The allowlist covers source, docs, config, .patch/.diff, and dotfiles such as .gitignore.
    • patches/ is exempt, because package patches carry upstream bytes verbatim.
    • Symlinks and submodules recorded in the Git index are skipped by mode. Paths are decoded as strict UTF-8, and a path that is not valid UTF-8 fails the test instead of being skipped.
    • A file missing from the working tree is skipped. Any other read error fails the test, so a partial scan can't pass.
    • A failure names file:line:column and the byte, for example src/daemon/web/deviceAudit.ts:80:30 raw \x00.
    • It follows the git ls-files pattern of internalDocGuard.test.mjs.
  • A second commit stores parseEntry.ts and parseEntry.test.ts with LF, like the rest of the repository. It changes line endings only (git diff --ignore-cr-at-eol is empty), so you can review the first commit alone or use "Hide whitespace". Left as CRLF, both files would fail git diff --check with default whitespace settings on every future edit, because git now treats them as text.

Behavior

Code behavior is unchanged: every rewrite in a source file respells a string literal, a regex or a comment. To check, I ran each of the 19 changed sources through esbuild.transformSync (comments dropped, charset: 'ascii') before and after:

  • 18 produce byte-identical output.
  • scripts/hooks-daemon-dynamic.mjs differs only because esbuild keeps regex source verbatim. With ESC spelled \x1b, its output is identical too, and the old and new regexes give the same result on four sample strings.

The documented wire value is also unchanged; only how the doc writes the NUL is different.

CHANGELOG entry

n/a: repository hygiene with no user-facing change.

Stability tier impact

  • Change to an internal surface (no contract impact).

docs/phone-client-contract.md changes only in how one marker is written. The marker on the wire is unchanged.

Substrate contract impact (Substrate 3.0)

n/a.

Test plan

Verified on Windows with Node 22.12.0, on upstream f31eb506, then rebased onto 94c49222 without conflicts.

  • Guard: 9/9 pass (the tracked-tree check plus unit cases for positions, UTF-16 columns, allowlist, Git modes, non-UTF-8 and BOM-prefixed paths, malformed entries and read errors). With the upstream bytes of deviceAudit.ts restored, it fails and names deviceAudit.ts:80:30.
  • Search: on f31eb506 a default directory search finds none of the three symbols above; on this branch it finds all three.
  • Rebase: upstream has since touched three of these files (WebTerminalServer.ts, the phone contract, ClaudePtyBrainAdapter.test.ts). After rebasing, the guard passes and the esbuild comparison against 94c49222 gives the same result.
  • Tests on every changed file: 35 files / 1,171 tests. Ten failures also fail on untouched main on this machine: symlink EPERM without Developer Mode, the Node 22.12 file-changed preview case, and a skill-discovery test that reads the developer's real ~/.claude. Two more were fetch failed socket timeouts under load and passed 3 of 3 times when rerun alone.
  • Static checks: node scripts/typecheck.mjs passes. ESLint on the new test is clean. git diff --check passes.

Related issues / PRs

n/a.

Reviewer checklist

  • CHANGELOG entry above is filled in or explicitly marked n/a.
  • Stability tier impact is correctly classified.
  • Substrate contract docs (PROTOCOL.md, inventory.md, stability.md) updated if the surface changed (n/a: no contract change).
  • Tests cover the new behavior and the regression case (if a bug fix).
  • No accidental commit of secrets, tokens, .env, or unrelated large binaries.

Summary by CodeRabbit

  • Documentation
    • The phone-client contract now displays the code-block marker using escaped notation.
  • Tests
    • Added checks that detect unexpected control characters in tracked text files.
    • Updated test examples to represent control characters with escaped notation; expected results remain unchanged.

Ten source files and docs/phone-client-contract.md held raw NUL bytes inside
string literals, a comment and inline code. With default settings ripgrep
treats such a file as binary and stops searching it: on main, a directory
search does not find even `export class WebTerminalServer` (line 1317). In
deviceAudit.ts the NUL sits within git's first 8000 bytes, so `git diff` and
GitHub's PR view show "Binary files differ" for it.

Raw ESC, BEL and US bytes in nine more files are respelled the same way. In
source files every rewrite respells a literal, regex or comment: esbuild
output is byte-identical for 18 of the 19 changed sources, and the remaining
one differs only in how a regex spells ESC. The contract doc now writes its
NUL marker as \u0000 and says what it denotes.

A tracked-tree test rejects raw C0 control bytes (other than tab, LF and CR)
in allowlisted text files. It skips symlinks and submodules recorded in the
index, exempts patches/, decodes paths as strict UTF-8, and fails on any
read error other than a missing file.
Git treated both files as binary while they held a raw NUL, so their CRLF
endings were never normalized like the rest of the repository. This commit
changes line endings only.
@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1f5ba1af-8f53-4152-985f-a8dbf694da60
📥 Commits

Reviewing files that changed from the base of the PR and between 94c4922 and a31d712.

📒 Files selected for processing (21)
  • docs/phone-client-contract.md
  • scripts/__tests__/sourceControlBytes.test.mjs
  • scripts/hooks-daemon-dynamic.mjs
  • scripts/lanlink-pr5-cdp-dogfood.mjs
  • scripts/m3-csp-device-stream-dynamic.mjs
  • src/cli/commands/__tests__/mcpClients.test.ts
  • src/daemon/approvals/__tests__/nativeDecision.test.ts
  • src/daemon/channels/__tests__/channelWakeWorker.test.ts
  • src/daemon/transcript/TranscriptProjector.ts
  • src/daemon/transcript/__tests__/parseEntry.test.ts
  • src/daemon/transcript/parseEntry.ts
  • src/daemon/web/WebTerminalServer.ts
  • src/daemon/web/__tests__/deviceStore.runtime.test.ts
  • src/daemon/web/__tests__/phoneGitRead.test.ts
  • src/daemon/web/deviceAudit.ts
  • src/main/deck/__tests__/ClaudePtyBrainAdapter.test.ts
  • src/main/metadata/__tests__/PrReviewRouter.test.ts
  • src/main/pipe/handlers/worktask.rpc.ts
  • src/mcp/browser-replay/__tests__/tool.siteMemory.test.ts
  • src/renderer/components/FleetView/__tests__/RemoteInboxList.test.tsx
  • src/renderer/components/Settings/SettingsPanel.tsx

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The changes replace literal control characters with escaped notation in code, tests, and documentation. A new test scans eligible tracked files for raw C0 control bytes and checks file classification, Git index parsing, reported positions, and read errors.

Changes

Control-byte hygiene

Layer / File(s) Summary
Replace literal control characters with escapes
docs/phone-client-contract.md, scripts/hooks-daemon-dynamic.mjs, scripts/lanlink-pr5-cdp-dogfood.mjs, scripts/m3-csp-device-stream-dynamic.mjs, src/cli/commands/__tests__/mcpClients.test.ts, src/daemon/approvals/__tests__/nativeDecision.test.ts, src/daemon/channels/__tests__/channelWakeWorker.test.ts, src/daemon/transcript/TranscriptProjector.ts, src/daemon/web/WebTerminalServer.ts, src/daemon/web/__tests__/deviceStore.runtime.test.ts, src/daemon/web/__tests__/phoneGitRead.test.ts, src/main/deck/__tests__/ClaudePtyBrainAdapter.test.ts, src/main/metadata/__tests__/PrReviewRouter.test.ts, src/main/pipe/handlers/worktask.rpc.ts, src/renderer/components/FleetView/__tests__/RemoteInboxList.test.tsx, src/renderer/components/Settings/SettingsPanel.tsx
Control characters in strings, keys, a regex, and documentation now use escape notation. The summaries report no change to the resulting runtime values or test expectations.
Add tracked-file control-byte scanner tests
scripts/__tests__/sourceControlBytes.test.mjs
The tests cover tracked-file selection, excluded paths, C0-byte detection and positions, Git index path parsing, missing files, and other read errors.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other

Suggested reviewers: openwong2kim

Merge Risk: ⚪ Minimal · up to a31d7

The control-character cleanup appears ready to merge after normal checks; no behavior-changing issue was identified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 70.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 16 files. (1 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: replacing raw control characters in tracked sources with escaped spellings.
Full details: Docstring Coverage

Explanation

Docstring coverage is 70.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 16 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openwong2kim

Copy link
Copy Markdown
Owner

Thanks, this is a great find. Three reviewers checked it token by token: every runtime string, template and regex value is unchanged, and the CRLF→LF files are value-identical too. Merging. One optional nit for whenever: sourceControlBytes.test.mjs (~L162/164) has a raw U+FEFF before 'probe.md', which could be spelled '\uFEFF' in the spirit of this PR.

@openwong2kim
openwong2kim merged commit 212c5be into openwong2kim:main Oct 4, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants