Skip to content

Commit ae216b7

Browse files
committed
Read both apostrophe-escaping conventions instead of guessing
Consuming every '' pair as a YAML escape broke the backslash convention when an escaped apostrophe ends the value: in password='...Q1\'' the first apostrophe is content and the second is the delimiter, but both were eaten as a pair, no delimiter was found, and the field produced no finding. Nothing local distinguishes the two readings — '' is either one YAML-escaped apostrophe inside the scalar or a backslash-escaped apostrophe followed by the closing quote — so deciding from adjacency gets one of them wrong every time. The convention now joins serialization depth as an enumerated reading rather than an inference, and the detector's existing rule applies: if any admissible reading holds a credential, the event is rejected. A field still reports at most once, since the readings are interpretations of one value rather than separate credentials. Refs #199
1 parent 033e555 commit ae216b7

2 files changed

Lines changed: 122 additions & 17 deletions

File tree

‎packages/core/src/secrets/rules.ts‎

Lines changed: 33 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -281,24 +281,26 @@ function closesValue(mark: string, depth: number, backslashes: number): boolean
281281
return backslashes % 2 ** (depth + 1) === delimiterEscapes(mark, depth);
282282
}
283283

284-
/** Read a quoted value on the assumption that the text is at `depth`. */
284+
/**
285+
* Read a quoted value under one reading of the text.
286+
*
287+
* `depth` is the assumed serialization depth. `yamlDoubling` picks which
288+
* convention a `''` pair is read under: one escaped apostrophe inside a YAML
289+
* scalar, or two characters the backslash rule judges separately.
290+
*/
285291
function readAtDepth(
286292
content: string,
287293
bodyStart: number,
288294
mark: string,
289295
depth: number,
296+
yamlDoubling: boolean,
290297
): { value: string; at: number } | undefined {
291298
for (let at = bodyStart; at < content.length; at++) {
292299
if (content[at] !== mark) {
293300
continue;
294301
}
295302

296-
// YAML escapes an apostrophe inside a single-quoted scalar by doubling
297-
// it, not with a backslash — and JSON leaves both the doubling and the
298-
// delimiters alone, so this reads the same at every depth. Without it the
299-
// first apostrophe of a pair closes the value, and a frontmatter password
300-
// truncates to whatever preceded it.
301-
if (mark === "'" && content[at + 1] === "'") {
303+
if (yamlDoubling && mark === "'" && content[at + 1] === "'") {
302304
at++;
303305
continue;
304306
}
@@ -323,16 +325,23 @@ function readAtDepth(
323325
/**
324326
* Every reading of a quoted value that the text admits.
325327
*
326-
* A double-quote delimiter announces its own depth: the opening run of
327-
* backslashes is `2 ** depth - 1`, so the depth is recoverable. An apostrophe
328-
* does not, because JSON never escapes it — `'ab\'cd'` looks identical
329-
* whether it came off disk or out of a serialized event, while the escape
330-
* inside it doubled. Both readings are returned and the caller judges each,
331-
* which is the right bias for a detector: if any reading of the text is a
332-
* credential, it must not be persisted.
328+
* Two things are unknowable from the text alone, and both are enumerated
329+
* rather than guessed.
333330
*
334-
* The readings coincide whenever the value contains no backslashes, so this
335-
* costs nothing on ordinary content.
331+
* A double-quote delimiter announces its serialization depth: the opening run
332+
* of backslashes is `2 ** depth - 1`, so the depth is recoverable. An
333+
* apostrophe does not, because JSON never escapes it.
334+
*
335+
* A `''` pair is likewise ambiguous — one YAML-escaped apostrophe inside the
336+
* scalar, or a backslash-escaped apostrophe followed by the closing
337+
* delimiter. Nothing local distinguishes them, so deciding from adjacency
338+
* alone gets one of the two wrong every time.
339+
*
340+
* Every admissible reading is therefore returned and the caller judges each.
341+
* If any reading of the text is a credential it must not be persisted, which
342+
* is the conservative bias a detector should have. The readings coincide on
343+
* ordinary content, where they cost nothing, and a field still reports at
344+
* most once.
336345
*/
337346
function quotedValues(content: string, at: number): Array<{ value: string; at: number }> {
338347
let open = at;
@@ -353,8 +362,15 @@ function quotedValues(content: string, at: number): Array<{ value: string; at: n
353362
: []
354363
: [0, 1];
355364

365+
// YAML doubling applies to single-quoted scalars only, and at most one of
366+
// the two conventions is right for any given text. Which one is not
367+
// knowable here, so both are read.
368+
const conventions = mark === "'" ? [true, false] : [false];
369+
356370
const readings = depths
357-
.map((depth) => readAtDepth(content, open + 1, mark, depth))
371+
.flatMap((depth) =>
372+
conventions.map((yamlDoubling) => readAtDepth(content, open + 1, mark, depth, yamlDoubling)),
373+
)
358374
.filter((reading) => reading !== undefined);
359375

360376
return readings.filter(

‎packages/core/tests/secret-rules.test.ts‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -679,3 +679,92 @@ describe("YAML doubled apostrophes in single-quoted fields", () => {
679679
expect(yield* scanner.scan(serializeDurableEvent(asImport(PLACEHOLDER_SOURCE)))).toEqual([]);
680680
});
681681
});
682+
683+
/**
684+
* `''` is ambiguous, and the scanner does not get to pick.
685+
*
686+
* It is either one YAML-escaped apostrophe inside the scalar, or a
687+
* backslash-escaped apostrophe followed by the closing delimiter. Nothing
688+
* local distinguishes them: `'ab''cd'` is the first and `'abc\''` the second,
689+
* and a reader that decides from adjacency gets one of the two wrong every
690+
* time — silently, because the wrong reading either truncates the value or
691+
* never finds a delimiter at all.
692+
*
693+
* Both readings are produced and each is judged, so a credential under either
694+
* convention is rejected.
695+
*/
696+
describe("both apostrophe-escaping conventions", () => {
697+
const TAIL = A.slice(0, 20);
698+
699+
/** YAML: the pair is one escaped apostrophe inside the value. */
700+
const YAML_CREDENTIAL = `ab''cd${TAIL}`;
701+
/** Backslash: the pair is an escaped apostrophe then the closing quote. */
702+
const BACKSLASH_CREDENTIAL = `${TAIL}Q1\\'`;
703+
704+
const CASES: Record<string, { source: string; credential: string; trailer: string }> = {
705+
"a YAML doubled apostrophe": {
706+
source: `password: '${YAML_CREDENTIAL}'`,
707+
credential: YAML_CREDENTIAL,
708+
trailer: " owner: someoneelseentirely",
709+
},
710+
"a trailing backslash-escaped apostrophe": {
711+
source: `password='${BACKSLASH_CREDENTIAL}'`,
712+
credential: BACKSLASH_CREDENTIAL,
713+
trailer: " owner=someoneelseentirely",
714+
},
715+
};
716+
717+
const asImport = (source: string): DurableEvent => ({
718+
type: "yield",
719+
coroutineId: "root",
720+
description: { type: "import_component", name: "__root__" },
721+
result: { status: "ok", value: { path: "README.md", content: source } },
722+
});
723+
724+
for (const [name, { source, credential, trailer }] of Object.entries(CASES)) {
725+
it(`detects a credential ending in ${name}, directly`, function* () {
726+
expect((yield* createSecretScanner().scan(source)).length).toBeGreaterThan(0);
727+
});
728+
729+
it(`detects a credential ending in ${name}, inside a serialized root import`, function* () {
730+
expect(
731+
(yield* createSecretScanner().scan(serializeDurableEvent(asImport(source)))).length,
732+
).toBeGreaterThan(0);
733+
});
734+
735+
it(`terminates ${name} at the right delimiter`, function* () {
736+
const document = source + trailer;
737+
738+
const findings = (yield* createSecretScanner().scan(document)).filter(
739+
(finding) => finding.ruleId === "@executablemd/secretlint-rule-credentials",
740+
);
741+
742+
// One field, one finding — the readings are interpretations of a single
743+
// value, not separate credentials.
744+
expect(findings).toHaveLength(1);
745+
const { column } = findings[0]!.location;
746+
expect(column).toBe(document.indexOf(credential));
747+
expect(document.slice(column, column + credential.length)).toBe(credential);
748+
});
749+
750+
it(`keeps ${name} and its fragments out of the findings`, function* () {
751+
const scanner = createSecretScanner();
752+
753+
for (const content of [source, serializeDurableEvent(asImport(source))]) {
754+
const serialized = JSON.stringify(yield* scanner.scan(content));
755+
expect(serialized).not.toContain(credential);
756+
expect(serialized).not.toContain(TAIL);
757+
expect(serialized).not.toContain(TAIL.slice(0, 8));
758+
}
759+
});
760+
}
761+
762+
it("leaves single-quoted placeholders alone under both conventions", function* () {
763+
const scanner = createSecretScanner();
764+
765+
for (const source of [`password: 'your-password-here'`, `password='your-api-key-here'`]) {
766+
expect(yield* scanner.scan(source)).toEqual([]);
767+
expect(yield* scanner.scan(serializeDurableEvent(asImport(source)))).toEqual([]);
768+
}
769+
});
770+
});

0 commit comments

Comments
 (0)