Skip to content

Commit 2dc20bf

Browse files
author
LASK
committed
fix(jsonl): broaden CSI final-byte range; correct cleanProtocolLine JSDoc
Both addresses Copilot inline review on PR #311. 1. The CSI regex previously matched final bytes only in `[a-zA-Z]`, which misses valid CSI sequences that end on a non-letter. The most relevant missing case in this codebase is `\x1b[200~` / `\x1b[201~` — bracketed-paste content markers — which a TTY-attached subprocess emits *around* pasted content, not just the on/off `\x1b[?2004h` / `\x1b[?2004l` toggles. With the old regex, a JSON record sandwiched between `200~` and `201~` would survive `replace`, then fail the first-char `{`/`[` guard, then be silently dropped — the worst kind of regression for this fix's stated goal. Broaden to the ECMA-48 CSI grammar: parameter bytes in 0x30..0x3F, optional intermediate bytes in 0x20..0x2F, final byte in 0x40..0x7E. Adds a regression test for `\x1b[200~{...}\x1b[201~` and the boundary finals `@` (0x40) and `` ` `` (0x60). 2. The JSDoc declared `@param {string} rawLine` even though the function explicitly handles non-string input by short-circuiting to `null` (and a unit test asserts that). Update to `@param {unknown}` so editors and tsc-via-checkJs don't mislead callers into thinking they must type-narrow first.
1 parent 5b94144 commit 2dc20bf

2 files changed

Lines changed: 27 additions & 4 deletions

File tree

‎plugins/codex/scripts/lib/jsonl.mjs‎

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -32,16 +32,25 @@
3232
* that fails the caller surfaces a real protocol error as before.
3333
*/
3434

35-
// CSI: ESC [ … final-byte (e.g. \x1b[?2004h, \x1b[31m)
36-
// OSC: ESC ] … (BEL | ESC \) (e.g. window-title sets)
35+
// CSI: ESC [ <params> <intermediates> <final>
36+
// params = 0x30..0x3F (digits, ?, etc.)
37+
// intermediates= 0x20..0x2F (space, !, ", $, %, &, ', (, ), *, +, ,, -, ., /)
38+
// final = 0x40..0x7E (any byte in @, A..Z, [, \, ], ^, _, `, a..z, {, |, }, ~)
39+
// This grammar matches the original ECMA-48 spec and covers cases like
40+
// bracketed-paste *content* markers `\x1b[200~` / `\x1b[201~` (final
41+
// byte `~`) which a letter-only `[a-zA-Z]` final misses.
42+
// OSC: ESC ] <text> (BEL | ESC \) e.g. window-title sets
3743
// eslint-disable-next-line no-control-regex
38-
const ANSI_ESCAPE_RE = /\x1b\[[0-9;?]*[a-zA-Z]|\x1b\][^\x07]*(?:\x07|\x1b\\)/g;
44+
const ANSI_ESCAPE_RE = /\x1b\[[\x30-\x3F]*[\x20-\x2F]*[\x40-\x7E]|\x1b\][^\x07]*(?:\x07|\x1b\\)/g;
3945

4046
/**
4147
* Returns a JSON-shaped candidate string for the given raw line, or
4248
* `null` if the line should be skipped because it cannot be valid JSONL.
4349
*
44-
* @param {string} rawLine
50+
* Accepts arbitrary input — non-string values short-circuit to `null` so
51+
* callers don't have to type-check before invoking.
52+
*
53+
* @param {unknown} rawLine
4554
* @returns {string | null}
4655
*/
4756
export function cleanProtocolLine(rawLine) {

‎tests/jsonl.test.mjs‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,20 @@ test("cleanProtocolLine strips bracketed-paste-mode ANSI prefix (issue #23)", ()
3535
);
3636
});
3737

38+
test("cleanProtocolLine strips CSI sequences whose final byte is not a letter", () => {
39+
// Bracketed-paste content markers use `~` as the final byte. A
40+
// letter-only CSI final pattern (`[a-zA-Z]`) misses them and the
41+
// surrounding JSON would be incorrectly dropped by the first-char
42+
// guard. ECMA-48's CSI grammar allows any final byte in 0x40..0x7E.
43+
assert.equal(
44+
cleanProtocolLine('\x1b[200~{"id":1}\x1b[201~'),
45+
'{"id":1}'
46+
);
47+
// Other non-letter finals at the boundaries of the 0x40..0x7E range.
48+
assert.equal(cleanProtocolLine('\x1b[@{"id":1}'), '{"id":1}'); // 0x40
49+
assert.equal(cleanProtocolLine('\x1b[`{"id":1}'), '{"id":1}'); // 0x60
50+
});
51+
3852
test("cleanProtocolLine strips OSC window-title sequences", () => {
3953
assert.equal(
4054
cleanProtocolLine('\x1b]0;some title\x07{"id":1}'),

0 commit comments

Comments
 (0)