Skip to content

Commit 4d0e2bf

Browse files
committed
Close three trust-boundary gaps in candidate scanning
Reading through @effectionx/fs was not a trust boundary. Its readTextFile and walk are FsApi.operations.*, a Context Api, so FsApi.around() replaces them exactly as API.Fs.around() replaces the runtime's — and both are reachable from inside an execution. A scan reading through either lets the thing being inspected choose what the inspector sees. Enumeration and reads now go to node:fs/promises, which has no interception point, with the departure from the repository's fs rule documented where it happens. Paths are scanned before content, because Git persists tree-entry names as surely as blobs, and only the portion below the staging root is scanned or reported. A path finding carries no path: the offending value is the path, so naming it in a report would republish the credential. The result type makes that unrepresentable rather than merely discouraged. Symbolic links are refused instead of followed. A link can reach outside the candidate entirely, so what it points at says nothing about what Git would store, and its target can itself carry a credential. The rejection carries no path, no target, and no cause. Refs #199
1 parent e9689f0 commit 4d0e2bf

2 files changed

Lines changed: 296 additions & 58 deletions

File tree

‎packages/core/src/secrets/files.ts‎

Lines changed: 109 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -6,42 +6,132 @@
66
* exists. It reuses the execution's scanner so a snapshot is held to exactly
77
* the rules the journal was.
88
*
9-
* Reads go straight to `@effectionx/fs` rather than through the runtime's
10-
* `Fs` Context Api. That Api exists so a document can be given a stubbed or
11-
* sandboxed filesystem, and its middleware is reachable from inside an
12-
* execution — routing a security scan through it would let the thing being
13-
* inspected choose what the inspector sees. A trusted scan reads the real
14-
* files.
9+
* ## Why this module does not use `@effectionx/fs`
10+
*
11+
* The repository rule is to reach the filesystem through `@effectionx/fs`,
12+
* and this module deliberately does not. That package's `readTextFile` and
13+
* `walk` are `FsApi.operations.*` — a Context Api — so `FsApi.around()`
14+
* replaces them for the current scope, exactly as `API.Fs.around()` replaces
15+
* the runtime's. Both are reachable from inside an execution, and eval blocks
16+
* can import anything.
17+
*
18+
* A scan that reads through either one lets the thing being inspected choose
19+
* what the inspector sees: middleware returning clean text for a file that
20+
* holds a credential defeats the check completely. So the reads here go to
21+
* `node:fs/promises`, which has no interception point, adapted with `until`.
22+
* The rule is about consistent I/O plumbing; this is a trust boundary, and
23+
* the two goals genuinely conflict here.
1524
*/
1625

17-
import { each, type Operation } from "effection";
18-
import { readTextFile, walk } from "@effectionx/fs";
26+
import { until } from "effection";
27+
import type { Operation } from "effection";
28+
import { readdir, readFile } from "node:fs/promises";
29+
import { join, relative, sep } from "node:path";
1930
import type { SecretFinding } from "./findings.ts";
2031
import type { SecretScanner } from "./scanner.ts";
2132

22-
/** A finding together with the file it came from. */
23-
export interface FileSecretFinding extends SecretFinding {
24-
/** Path of the file the finding was found in. */
25-
path: string;
33+
/**
34+
* A finding from a candidate file, and what it was found in.
35+
*
36+
* A content finding names the file it came from — its path has already been
37+
* scanned and cleared by then, so reporting it leaks nothing. A path finding
38+
* deliberately carries no path at all: the offending value *is* the path, and
39+
* Git persists tree-entry names, so naming it in a report would republish the
40+
* credential the scan exists to stop.
41+
*/
42+
export type FileSecretFinding =
43+
| (SecretFinding & {
44+
subject: "content";
45+
/** Root-relative path of the file. Scanned and clean. */
46+
path: string;
47+
})
48+
| (SecretFinding & {
49+
subject: "path";
50+
});
51+
52+
/**
53+
* A candidate that cannot be scanned safely, so it is not published.
54+
*
55+
* Carries no path, no link target, and no cause. The name of an offending
56+
* entry can itself be the credential, and a symlink target can point at one,
57+
* so the diagnostic says what happened and nothing about where.
58+
*/
59+
export class CandidateRejectedError extends Error {
60+
constructor(reason: string) {
61+
super(
62+
`secret detection refused a snapshot candidate: ${reason}. ` +
63+
`Details are withheld because a path or link target can itself carry a credential.`,
64+
);
65+
this.name = "CandidateRejectedError";
66+
}
2667
}
2768

2869
/**
2970
* Scan every file under `root`, in memory, and return all findings.
3071
*
72+
* Each entry's root-relative path is scanned before its content, because Git
73+
* persists tree-entry names as surely as blobs. Only the portion below `root`
74+
* is ever scanned or reported — the temporary directory prefix is an artifact
75+
* of how the candidate was staged and has nothing to do with the snapshot.
76+
*
3177
* Returning findings rather than throwing on the first one lets the caller
3278
* report everything a snapshot would have leaked instead of one file at a
3379
* time. An empty result means the candidate is clear.
80+
*
81+
* Symbolic links are refused outright rather than followed. A link can reach
82+
* outside the candidate entirely, so scanning what it points at would say
83+
* nothing about what Git would store. Supporting them is a separate design.
3484
*/
3585
export function* scanFiles(root: string, scanner: SecretScanner): Operation<FileSecretFinding[]> {
3686
const findings: FileSecretFinding[] = [];
87+
yield* scanDirectory(root, root, scanner, findings);
88+
return findings;
89+
}
90+
91+
function* scanDirectory(
92+
root: string,
93+
directory: string,
94+
scanner: SecretScanner,
95+
findings: FileSecretFinding[],
96+
): Operation<void> {
97+
const entries = yield* until(readdir(directory, { withFileTypes: true }));
3798

38-
for (const entry of yield* each(walk(root, { includeFiles: true, includeDirs: false }))) {
39-
const content = yield* readTextFile(entry.path);
40-
for (const finding of yield* scanner.scan(content)) {
41-
findings.push({ ...finding, path: entry.path });
99+
for (const entry of entries) {
100+
const absolute = join(directory, entry.name);
101+
// Dirent types come from lstat, so a link is seen as a link rather than
102+
// as whatever it points at.
103+
if (entry.isSymbolicLink()) {
104+
throw new CandidateRejectedError("it contains a symbolic link");
42105
}
43-
yield* each.next();
44-
}
45106

46-
return findings;
107+
// POSIX separators keep a path scanning the same on every platform.
108+
const path = relative(root, absolute).split(sep).join("/");
109+
const inPath = yield* scanner.scan(path);
110+
111+
if (inPath.length > 0) {
112+
// Deliberately no `path` — reporting it would republish the credential.
113+
findings.push(...inPath.map((finding) => ({ ...finding, subject: "path" as const })));
114+
continue;
115+
}
116+
117+
if (entry.isDirectory()) {
118+
yield* scanDirectory(root, absolute, scanner, findings);
119+
continue;
120+
}
121+
122+
if (!entry.isFile()) {
123+
throw new CandidateRejectedError(
124+
"it contains an entry that is neither a file nor a directory",
125+
);
126+
}
127+
128+
const content = yield* until(readFile(absolute, "utf8"));
129+
findings.push(
130+
...(yield* scanner.scan(content)).map((finding) => ({
131+
...finding,
132+
subject: "content" as const,
133+
path,
134+
})),
135+
);
136+
}
47137
}

0 commit comments

Comments
 (0)