Skip to content

Commit 042e240

Browse files
authored
📄 Say that XMD scans for credentials by default, and prove it over each backend (#575)
The journal-gates page still described secret detection as something a host brings: a `containsCredential` placeholder inside `guardDurableStream`. Every execution has scanned by default since #199, so the page taught the wrong common path, and the CLI reference did not mention the opt-out at all. The page now leads with the built-in policy — default-on offline scanning installed before the first live event, what a finding does to the offending append and to the run, output withholding, the diagnostic that names a rule and never a value, and replay that does not rescan. `secretDetection: false` and `--no-secret-detection` appear as a dangerous diagnostic escape hatch with the warning the CLI actually writes, not as remediation. `guardDurableStream` keeps its gate-ordering, per-event and replay explanation, now as the generic decorator it is, and its example refuses an oversized event rather than deciding what a credential is. The reference lists the option with the same invocation-wide semantics. Two regressions cover the real default scanner over the two backends the memory cases could not reach. Each puts a synthetic, run-time-assembled credential in the root source, passes the backend straight to `execute()` with default options, and asserts `Err(SecretDetectedError)`, the absence of the rejected import, an independently safe `Close(err)`, and the canary nowhere in the persisted events or the error's readable fields — through the JSONL bytes on disk for the file backend, and through a read-back for HTTP. Co-authored-by: Taras Mankovski <[email protected]>
1 parent 2c18a18 commit 042e240

3 files changed

Lines changed: 215 additions & 24 deletions

File tree

‎packages/core/tests/guarded-journal.test.ts‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,23 @@ import type { DurableEvent, DurableStream } from "@executablemd/durable-streams"
2626
import { API } from "@executablemd/runtime";
2727
import { useStubFs } from "@executablemd/runtime/test";
2828
import { execute } from "../src/execute.ts";
29+
import { inlineSource } from "../src/root-source.ts";
30+
31+
const ALPHABET = "abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
32+
33+
/**
34+
* A synthetic GitHub token, format-realistic and assembled here at run time so
35+
* no usable-looking literal enters the repository and scanning this file finds
36+
* nothing. Nothing here reads an environment variable, a Git credential, or
37+
* user configuration.
38+
*/
39+
const CANARY = ["ghp", "_", ALPHABET.slice(0, 36)].join("");
40+
41+
/** A root whose own source carries the canary, so its import event does. */
42+
const TAINTED = `# Tainted
43+
44+
The token is ${CANARY} and must never reach a backend.
45+
`;
2946

3047
const DOCUMENT = `# Guarded
3148
@@ -270,3 +287,78 @@ describe("a guarded journal", () => {
270287
expect(persisted.map(label)).toEqual(["yield(import_component)", "yield(exec)", "close(root)"]);
271288
});
272289
});
290+
291+
/**
292+
* The default scanner over each persistence backend (#199).
293+
*
294+
* These runs install no gate of their own. `execute()` selects its own
295+
* credential policy before the first live event, so what they exercise is the
296+
* real default scanner reaching a file and an HTTP backend — the memory case
297+
* is covered in `secret-detection.test.ts` and is not repeated here.
298+
*/
299+
describe("the default scanner over a persistence backend", () => {
300+
it("keeps a credential out of a file backend", function* () {
301+
const dir = yield* useTempDirectory("xmd-default-scanner-");
302+
const journalPath = path.join(dir, "journal.jsonl");
303+
const backend = fileStream(journalPath);
304+
305+
const result = yield* yield* execute({ ...inlineSource(TAINTED), stream: backend });
306+
307+
expect(result.ok).toBe(false);
308+
expect(result.ok === false && result.error.name).toBe("SecretDetectedError");
309+
310+
// The diagnostic names the rule that fired and never the matched value.
311+
const error = result.ok === false ? result.error : new Error("unreachable");
312+
expect(error.message).not.toContain(CANARY);
313+
expect(error.stack ?? "").not.toContain(CANARY);
314+
315+
// The offending import never reached the backend, and the close the
316+
// failure produced crossed the policy on its own.
317+
const persisted = yield* backend.readAll();
318+
expect(persisted.some((event) => event.type === "yield")).toBe(false);
319+
expect(persisted.map(label)).toEqual(["close(root)"]);
320+
const close = persisted[0]!;
321+
expect(close.type === "close" && close.result.status).toBe("err");
322+
expect(persisted.map(serializeDurableEvent).join("")).not.toContain(CANARY);
323+
324+
// What is on disk is exactly those events, canary-free byte for byte.
325+
const bytes = yield* readTextFile(journalPath);
326+
expect(bytes).toBe(persisted.map(serializeDurableEvent).join(""));
327+
expect(bytes).not.toContain(CANARY);
328+
});
329+
330+
it("keeps a credential out of an HTTP backend", function* () {
331+
// port 0: the corpus runs under three runtimes concurrently, and a fixed
332+
// port lets those servers collide — macOS shares the listen port between
333+
// processes instead of refusing the second bind.
334+
const server = new DurableStreamTestServer({ port: 0 });
335+
const baseUrl = yield* until(server.start());
336+
yield* ensure(() => until(server.stop()));
337+
338+
const backend = yield* useHttpDurableStream({
339+
baseUrl,
340+
streamId: "default-scanner",
341+
producerId: "default-scanner-test",
342+
epoch: 1,
343+
fetch: closingFetch,
344+
});
345+
346+
const result = yield* yield* execute({ ...inlineSource(TAINTED), stream: backend });
347+
348+
expect(result.ok).toBe(false);
349+
expect(result.ok === false && result.error.name).toBe("SecretDetectedError");
350+
351+
// The diagnostic names the rule that fired and never the matched value.
352+
const error = result.ok === false ? result.error : new Error("unreachable");
353+
expect(error.message).not.toContain(CANARY);
354+
expect(error.stack ?? "").not.toContain(CANARY);
355+
356+
// Read back through the stream rather than from anything the run retained.
357+
const persisted = yield* backend.readAll();
358+
expect(persisted.some((event) => event.type === "yield")).toBe(false);
359+
expect(persisted.map(label)).toEqual(["close(root)"]);
360+
const close = persisted[0]!;
361+
expect(close.type === "close" && close.result.status).toBe("err");
362+
expect(persisted.map(serializeDurableEvent).join("")).not.toContain(CANARY);
363+
});
364+
});

‎site/routes/docs/journal.tsx‎

Lines changed: 114 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,19 @@ import { define } from "../../utils.ts";
22
import { CodeBlock } from "../../components/Code.tsx";
33
import { NextCard } from "../../components/NextCard.tsx";
44

5+
const DEFAULT = `import { execute } from "@executablemd/core";
6+
7+
const execution = yield* execute({ path: "./README.md", stream: backend });`;
8+
9+
const DISABLE = `const execution = yield* execute({
10+
path: "./README.md",
11+
stream: backend,
12+
secretDetection: false,
13+
});`;
14+
15+
const CLI = `$ xmd run workflow.md --no-secret-detection
16+
WARNING: secret detection is disabled; credentials may be persisted.`;
17+
518
const OBSERVE = `import { execute } from "@executablemd/core";
619
import { guardDurableStream, serializeDurableEvent } from "@executablemd/durable-streams";
720
@@ -12,8 +25,8 @@ const stream = guardDurableStream(backend, function* (event) {
1225
const execution = yield* execute({ path: "./README.md", stream });`;
1326

1427
const REJECT = `const stream = guardDurableStream(backend, function* (event) {
15-
if (yield* containsCredential(event)) {
16-
throw new Error("refusing to journal a credential");
28+
if (serializeDurableEvent(event).length > 64_000) {
29+
throw new Error("refusing to journal an oversized event");
1730
}
1831
});`;
1932

@@ -23,12 +36,101 @@ export default define.page(function Journal() {
2336
<h1>Journal gates</h1>
2437
<p class="muted">
2538
A gate is a check that runs before a durable event reaches its
26-
persistence backend. Wrap the stream you already pass to{" "}
27-
<code>execute</code>, and every live event — the root component import,
28-
each block's yield, each close — passes through it first.
39+
persistence backend. XMD installs one for you: every execution refuses
40+
to persist an event that carries a credential.
41+
</p>
42+
43+
<h2>Secret detection is on by default</h2>
44+
<p>
45+
There is nothing to install and no service to reach — the scan is local.
46+
{" "}
47+
<code>execute</code>{" "}
48+
selects its journal before the durable run starts, so the root component
49+
import is already covered, and the same policy holds for every later
50+
yield and close.
51+
</p>
52+
<CodeBlock>{DEFAULT}</CodeBlock>
53+
54+
<h2>What a finding does</h2>
55+
<p>
56+
A finding rejects that one append before the backend is invoked, and the
57+
failure reaches the durable effect that produced the event. What happens
58+
next is ordinary error handling: a <code>&lt;PrintErrors&gt;</code>{" "}
59+
region prints the rejection where it stands and the run continues, while
60+
an effect at a plain root — and a rejection during the root import —
61+
fails the run.
62+
</p>
63+
<p>
64+
Either way the offending event is absent, and the closing event the
65+
failure produces is a separate append that crosses the policy on its
66+
own. A rejection never means an empty journal, so assert that the
67+
offending event is missing rather than that nothing was written.
68+
</p>
69+
<p>
70+
There is no allowlist, sanitization, repair, or approval step. A finding
71+
is a defect in the code or the data flowing through it, and fixing that
72+
is the response.
2973
</p>
3074

31-
<h2>Observing the journal</h2>
75+
<h2>Output is held to the same policy</h2>
76+
<p>
77+
A live output chunk the execution's scanner cannot clear — a finding, or
78+
a scan that fails — is withheld from the output stream in full. Chunks
79+
already cleared stay observable and become the stream's partial close
80+
value. Withholding decides nothing about the run: the journal gate
81+
remains the rejection authority, and the rejected event stays absent
82+
either way.
83+
</p>
84+
85+
<h2>What a finding reports</h2>
86+
<p>
87+
Findings and scanner failures carry the rule identity and the position
88+
that matched — never the matched value, the scanned content, or the
89+
detector's own error. A diagnostic you can paste into an issue is the
90+
point.
91+
</p>
92+
93+
<h2>Replay does not rescan</h2>
94+
<p>
95+
A replayed execution restores its retained output and history without
96+
scanning them again. The journal it replays already crossed the gate
97+
when it was written.
98+
</p>
99+
100+
<h2>Turning it off</h2>
101+
<p>
102+
A host can disable the policy with{" "}
103+
<code>secretDetection: false</code>. This is a dangerous diagnostic
104+
escape hatch, not a remediation: with it off, a credential in a document
105+
or in a command's output is written to the journal and stays there.
106+
</p>
107+
<CodeBlock>{DISABLE}</CodeBlock>
108+
<p>
109+
At the command line the only spelling is{" "}
110+
<code>--no-secret-detection</code>. It is a switch, it applies to the
111+
whole invocation rather than to one document, and a disabled invocation
112+
says so once on standard error before the first document runs.
113+
</p>
114+
<CodeBlock command>{CLI}</CodeBlock>
115+
<p class="muted">
116+
The request belongs to the host alone. Root props, frontmatter,
117+
component props and eval bindings named <code>secretDetection</code>
118+
{" "}
119+
mean nothing to it — a document has no way to reach the value the host
120+
passed, or to turn the policy off.
121+
</p>
122+
123+
<h2>Adding a gate of your own</h2>
124+
<p>
125+
<code>guardDurableStream</code> wraps the stream you pass to{" "}
126+
<code>execute</code>{" "}
127+
so every live event passes your check first. The decorator is generic:
128+
it knows nothing about credentials and runs whatever gate it is given.
129+
XMD's credential policy is one such gate, installed for you; yours runs
130+
beside it.
131+
</p>
132+
133+
<h3>Observing the journal</h3>
32134
<p>
33135
The gate receives the structured event.{" "}
34136
<code>serializeDurableEvent</code>{" "}
@@ -37,21 +139,18 @@ export default define.page(function Journal() {
37139
</p>
38140
<CodeBlock>{OBSERVE}</CodeBlock>
39141

40-
<h2>Failing closed</h2>
142+
<h3>Failing closed</h3>
41143
<p>
42144
A gate that raises stops the append. The event never reaches the
43145
backend, and the failure travels back to the durable effect that
44146
produced it — the block does not resume with a result that was never
45-
journaled.
147+
journaled. Rejection is per event here too: events already accepted
148+
stay, and the closing event a rejection causes crosses the gate on its
149+
own.
46150
</p>
47151
<CodeBlock>{REJECT}</CodeBlock>
48-
<p class="muted">
49-
<code>containsCredential</code>{" "}
50-
stands in for whatever policy you bring. Deciding what counts as a
51-
secret is your call, not this library's.
52-
</p>
53152

54-
<h2>What a gate cannot do</h2>
153+
<h3>What a gate cannot do</h3>
55154
<p>
56155
A gate returns nothing, and the event it receives is a copy — writing to
57156
that copy changes nothing about what gets journaled. It cannot rewrite,
@@ -60,16 +159,7 @@ export default define.page(function Journal() {
60159
journaled twice.
61160
</p>
62161

63-
<h2>Rejection is per event</h2>
64-
<p>
65-
Refusing one event does not empty the journal. Events already accepted
66-
stay, and the failure a rejection causes may itself produce a closing
67-
event that crosses the gate on its own and is admitted. When you assert
68-
on a rejection, assert that the offending event is absent — not that
69-
nothing was written.
70-
</p>
71-
72-
<h2>Install before execution</h2>
162+
<h3>Install before execution</h3>
73163
<p>
74164
Wrap the stream before calling{" "}
75165
<code>execute</code>. A gate added later cannot see what was already

‎site/routes/docs/reference.tsx‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,15 @@ export default define.page(function Reference() {
2727
— write a diagnostic JSONL trace of the run to a new file. The path
2828
must not exist and is never replayed.
2929
</li>
30+
<li>
31+
<code>--no-secret-detection</code>{" "}
32+
— turn off credential detection for the whole invocation. Detection is
33+
on by default, and refuses to persist a durable event carrying a
34+
credential. This is the only spelling that disables it, and a disabled
35+
run warns once on standard error that credentials may be persisted. A
36+
dangerous diagnostic escape hatch: the normal response to a finding is
37+
to fix the code or data flow that produced it.
38+
</li>
3039
<li>
3140
<code>--verbose</code>, <code>-V</code>{" "}
3241
— print journal entries to stderr.

0 commit comments

Comments
 (0)