Skip to content

Commit 3da44ae

Browse files
committed
✨ Make <Dir> create the directory it names (#643)
`<Dir path>` installed a working directory and created nothing, so a document that named a directory which was not there ran its content somewhere that did not exist. It now performs one provider-neutral directory act before it installs cwd and before any content expands. `API.Files` gains a mandatory `ensureDirectory`. It recursively creates a missing target, succeeds for an existing directory without replacing it or clearing its contents, and refuses a file or another non-directory at the target or on the way to it. Success carries nothing at all — no path, no handle, and no word on whether this call is what created it — for the same reason `deleteFile` carries nothing: a document that asked for a directory to exist has been answered by its existence, and a caller that could tell the two apart would be branching on a race. The host provider creates directly and persistently. There is no rollback and no teardown removal: a later failure of the content running inside a directory says nothing about whether the directory should exist. It is also the one operation here that accepts an absolute path, which is `<Dir>`'s established exception, so it resolves its own destination rather than inheriting the rule that refuses one. The workflow provider publishes one `workspace_file` effect. The recursive creation runs inside its savepoint, so a refusal after a partial creation leaves neither the parent nor a rolled-back transaction — the effect that follows still commits. An absolute path there is a logical Workspace path used as written, and names nothing on the host. Shared `<Dir>` calls only `API.Files.ensureDirectory`, through core's door that turns an illegal provider throw into a fenced failure. It reaches no `API.Fs`, no `@effectionx/fs`, no runtime API and no Repository composition, and it changes no Repository selection or identity member. `allow={["write"]}` now authorizes persistent recursive creation, so the pinned paired identity is versioned to `@executablemd/workflow/composition/dir-v2#Dir`. A continuation retaining the former identity — which authorized placement that created nothing — refuses before generated execution and before any mutation. The thirteen composition components take their accepted descriptions. Evidence: HF15b, HF24–HF28, FF2, the compiled contract probe, WF23–WF26, WGAC17, ORC6–ORC6i, runtime parity under Deno, Node and Bun, and ORC20.
1 parent 762ab2c commit 3da44ae

19 files changed

Lines changed: 1529 additions & 99 deletions

‎packages/cli/tests/run-composition.test.ts‎

Lines changed: 46 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,9 @@ import { InMemoryStream } from "@executablemd/durable-streams";
2323
import { collect, execute, inlineSource, registerComponents } from "@executablemd/core";
2424
import { API, Service, useHostFiles } from "@executablemd/runtime";
2525
import type { RuntimeFetchResponse } from "@executablemd/runtime";
26-
import { exists, readdir } from "@effectionx/fs";
26+
import { exists, readdir, readTextFile, writeTextFile } from "@effectionx/fs";
2727
import { useTempDirectory } from "@executablemd/test-support/temp";
28+
import { join } from "node:path";
2829
import { COMPOSITION_REGISTRATIONS } from "@executablemd/workflow";
2930
import { syntaxCatalog, useRunProfileRegistry } from "../src/syntax.ts";
3031
import { DEFAULT_REPOSITORY_ROOT, unsupportedRepositories } from "../src/run-repositories.ts";
@@ -215,6 +216,50 @@ describe("ORC2 — one language, described everywhere and operated somewhere", (
215216
});
216217
});
217218

219+
describe("ORC2 — Dir is operational where the repository provider is not", () => {
220+
// The one composition element that needs no repository provider. Every other
221+
// name here reports an absent provider on this runtime; `<Dir>` reaches the
222+
// host `API.Files` provider instead, which all four entrypoints install — so
223+
// it creates its directory and runs its content on Deno, Node and Bun alike.
224+
//
225+
// This runs in the portable suite deliberately: the operation is the same one
226+
// the tier suites drive under Deno, and the claim being made here is that
227+
// nothing about it is Deno's.
228+
it("creates the directory and runs its content with no repository provider", function* () {
229+
const workspace = yield* useTempDirectory("xmd-orc2-dir-");
230+
231+
const rendered = String(
232+
yield* ordinaryWithoutProvider(
233+
'<Dir path="made/deep">\n\n<File path="inside.md">landed</File>\n\nINSIDE\n\n</Dir>\n',
234+
workspace,
235+
),
236+
);
237+
238+
expect(rendered).toContain("INSIDE");
239+
expect(yield* exists(join(workspace, "made", "deep"))).toBe(true);
240+
expect(yield* readTextFile(join(workspace, "made", "deep", "inside.md"))).toBe("landed");
241+
});
242+
243+
// And the refusal half, on the same runtime: a non-directory is refused
244+
// rather than answered, so `<Dir>` being operational is not `<Dir>` being
245+
// permissive.
246+
it("refuses a non-directory target without a repository provider", function* () {
247+
const workspace = yield* useTempDirectory("xmd-orc2-dir-refusal-");
248+
yield* writeTextFile(join(workspace, "occupied"), "a file");
249+
250+
const rendered = String(
251+
yield* ordinaryWithoutProvider(
252+
'<PrintErrors>\n<Dir path="occupied">\n\nINSIDE\n\n</Dir>\n</PrintErrors>\n',
253+
workspace,
254+
),
255+
);
256+
257+
expect(rendered).toContain("not a directory");
258+
expect(rendered).not.toContain("INSIDE");
259+
expect(yield* readTextFile(join(workspace, "occupied"))).toBe("a file");
260+
});
261+
});
262+
218263
describe("ORC2 — where the managed root is", () => {
219264
it("names ~/.xmd/repositories and nothing a document can influence", function* () {
220265
expect(DEFAULT_REPOSITORY_ROOT.endsWith("/.xmd/repositories")).toBe(true);

‎packages/cli/tests/workflow-agent.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -719,7 +719,7 @@ describe("Tier WAL — the workflow Agent observation loop", () => {
719719
expect(recorded).toHaveLength(1);
720720
const policy = JSON.stringify(recorded[0]);
721721
expect(policy).toContain("File:write");
722-
expect(policy).toContain("@executablemd/workflow/composition#Dir");
722+
expect(policy).toContain("@executablemd/workflow/composition/dir-v2#Dir");
723723
expect(policy).toContain('"allow":["write"]');
724724
// And the change is in the run's own Workspace, where an ordinary read
725725
// beneath the fragment's own directory finds it. Anchored on the

‎packages/cli/tests/workflow-suspension.test.ts‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,47 @@ interface Retained {
192192
readonly rootCloses: number;
193193
}
194194

195+
/** Every journal record, as stored. */
196+
function readRecords(path: string): string[] {
197+
const database = new DatabaseSync(path, { readOnly: true });
198+
try {
199+
return database
200+
.prepare("SELECT record FROM journal_events ORDER BY sequence")
201+
.all()
202+
.map((row) => String(row["record"]));
203+
} finally {
204+
database.close();
205+
}
206+
}
207+
208+
/**
209+
* Rewrite every retained record through `edit`.
210+
*
211+
* A continuation admitted under a previous release cannot be produced by
212+
* running this one, so the retained history is edited into the shape that
213+
* release left behind. Nothing else about the run changes — the same source,
214+
* roots and selection — so the identity is the only thing a resume can
215+
* disagree about.
216+
*/
217+
function rewriteRecords(path: string, edit: (record: string) => string): void {
218+
const database = new DatabaseSync(path);
219+
try {
220+
const rows = database
221+
.prepare("SELECT event_id AS id, record FROM journal_events")
222+
.all()
223+
.map((row) => ({ id: String(row["id"]), record: String(row["record"]) }));
224+
const update = database.prepare("UPDATE journal_events SET record = ? WHERE event_id = ?");
225+
for (const row of rows) {
226+
const edited = edit(row.record);
227+
if (edited !== row.record) {
228+
update.run(edited, row.id);
229+
}
230+
}
231+
} finally {
232+
database.close();
233+
}
234+
}
235+
195236
function retained(path: string): Retained {
196237
const database = new DatabaseSync(path, { readOnly: true });
197238
try {
@@ -1467,6 +1508,80 @@ the retained note
14671508
expect(answers(path)[0]?.state).toBe("consumed");
14681509
});
14691510
}
1511+
1512+
// WGAC17: `allow={["write"]}` admits the versioned paired `<Dir>`, and that
1513+
// entry now authorizes persistent recursive creation. The former identity
1514+
// authorized placement that created nothing, so a run admitted under it must
1515+
// not silently receive the wider authority.
1516+
//
1517+
// The continuation is constructed the only way it can be: the retained
1518+
// admission is rewritten to carry the former identity, which is exactly what
1519+
// a run admitted before this change holds. Nothing else about the run is
1520+
// touched — same source, same roots, same selection — so the identity is the
1521+
// single thing the resume disagrees about.
1522+
it("WGAC17: a continuation retaining the former Dir identity refuses before generated execution and mutation", function* () {
1523+
const store = yield* useRunStore();
1524+
const fixture = yield* useCheckpointFixture(bundledRoot("", "Propose"), {
1525+
Propose: `<Evaluate source={"<Dir path=\\"generated\\">\\n\\n<File path=\\"inside.md\\">from the fragment</File>\\n\\n</Dir>"} allow={["write"]} as="observed" />\n`,
1526+
});
1527+
yield* useRepositoryGit(fixture.repository);
1528+
1529+
const calls: AgentCalls = { prompts: [] };
1530+
const rendered: string[] = [];
1531+
const started = yield* invoke(
1532+
{ ...REQUEST, action: "start" },
1533+
yield* startFor(fixture),
1534+
productionHost(store, calls),
1535+
pinnedBody(rendered),
1536+
);
1537+
expect(started.exitCode).toBe(2);
1538+
const runId = String(started.written.err.find((line) => line.startsWith("workflow run: ")))
1539+
.slice("workflow run: ".length)
1540+
.trim();
1541+
const path = workflowRunPath(store, runId);
1542+
1543+
// The admission carries the versioned identity, and the generated Dir made
1544+
// its directory. This is the baseline the continuation is held to.
1545+
const suspended = retained(path);
1546+
expect(suspended.status).toBe("suspended");
1547+
const atSuspension = counts(path);
1548+
const rootsAtSuspension = workspaceRootState(path);
1549+
expect(readRecords(path).some((record) => record.includes("dir-v2#Dir"))).toBe(true);
1550+
1551+
const suspensionId = suspended.requests[0] ?? "";
1552+
yield* manage(
1553+
{ action: "answer", runId, suspensionId, value: { proceed: true }, secretDetection: true },
1554+
productionHost(store, calls),
1555+
);
1556+
1557+
// The run becomes one admitted under the former identity.
1558+
rewriteRecords(path, (record) =>
1559+
record.replaceAll(
1560+
"@executablemd/workflow/composition/dir-v2#Dir",
1561+
"@executablemd/workflow/composition#Dir",
1562+
),
1563+
);
1564+
const beforeResume = counts(path);
1565+
const rootsBeforeResume = workspaceRootState(path);
1566+
1567+
const resumed = yield* invoke(
1568+
{ ...REQUEST, action: "resume", target: runId },
1569+
undefined,
1570+
productionHost(store, calls),
1571+
pinnedBody(rendered),
1572+
);
1573+
1574+
// Refused, and before anything generated ran: no further effect of any
1575+
// kind was committed and no Workspace root was published.
1576+
expect(resumed.exitCode).not.toBe(0);
1577+
expect(counts(path)).toEqual(beforeResume);
1578+
expect(workspaceRootState(path)).toEqual(rootsBeforeResume);
1579+
// Nothing after the wait reached the document either.
1580+
expect(rendered).toEqual([]);
1581+
// And the refusal does not publish which identity moved.
1582+
const reported = resumed.written.err.join("\n");
1583+
expect(reported).not.toContain("dir-v2");
1584+
});
14701585
});
14711586

14721587
/**

‎packages/core/mod.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -348,6 +348,17 @@ export { createSecretScanner } from "./src/secrets/scanner.ts";
348348
export type { SecretScanner } from "./src/secrets/scanner.ts";
349349
export { SecretDetectedError, SecretScannerError } from "./src/secrets/findings.ts";
350350
export type { SecretFinding } from "./src/secrets/findings.ts";
351+
/**
352+
* The engine's `API.Files` door, for a component that does not live in core.
353+
*
354+
* `<Dir>` is the composition package's, and it performs a document filesystem
355+
* act like any other component. It must reach the Api the same way core's own
356+
* components do — through the call that converts an illegal throw into a
357+
* failure the engine can fence — rather than by holding the Api itself, which
358+
* is what "a provider may fail and may not throw" means in practice.
359+
*/
360+
export { ensureDirectory } from "./src/files.ts";
361+
351362
export { scanFiles } from "./src/secrets/files.ts";
352363
export type { FileSecretFinding } from "./src/secrets/files.ts";
353364

‎packages/core/src/files.ts‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -321,6 +321,26 @@ export function deleteFile(input: FilePathInput): Operation<Result<void>> {
321321
return unit(Files.operations.deleteFile(input), { operation: "delete", phase: "access" });
322322
}
323323

324+
/**
325+
* Make this authored path name a directory, with nothing coming back.
326+
*
327+
* Unit for the same reason `deleteFile` is: what the caller asked for is the
328+
* directory's existence, and existence is the whole answer. A receipt saying
329+
* whether this call created it or found it would be a difference the operation
330+
* deliberately does not offer, and a caller that could branch on it would be
331+
* branching on a race.
332+
*
333+
* `access` is the fallback phase, because reaching the directory is the step
334+
* this operation exists for — a refusal the vocabulary does not recognize
335+
* happened while trying to make it, not while reading the path.
336+
*/
337+
export function ensureDirectory(input: FilePathInput): Operation<Result<void>> {
338+
return unit(Files.operations.ensureDirectory(input), {
339+
operation: "ensure-directory",
340+
phase: "access",
341+
});
342+
}
343+
324344
export function readFileText(input: FilePathInput): Operation<Result<string>> {
325345
return outcome(Files.operations.readTextFile(input), {
326346
operation: "read",

‎packages/core/tests/files-fatal.test.ts‎

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ import { expect } from "@executablemd/test-support/expect";
2020
import { ensure, Err, Ok, resource, scoped, spawn, suspend, until, withResolvers } from "effection";
2121
import type { Operation, Result } from "effection";
2222
import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs";
23+
import { ensureDirectory } from "../src/files.ts";
24+
import { registerComponents } from "../src/components/registration.ts";
2325
import {
2426
API,
2527
FILES_ERROR,
@@ -258,9 +260,41 @@ describe("Tier FF — Files infrastructure failure", () => {
258260
});
259261

260262
// FF2: the same for every other form. Each stops at its first Files call.
261-
it("FF2: read, delete, Glob and TempDir all stop at their first provider call", function* () {
263+
//
264+
// `<Dir>` belongs to `@executablemd/workflow`, which depends on core rather
265+
// than the other way round, so the element itself cannot be expanded here.
266+
// What can be — and what FF2 is actually about — is the one call it makes:
267+
// `ensureDirectory` through the engine's door, with no provider installed.
268+
// `<Ensures>` performs exactly that and nothing else, so an absent provider
269+
// reaches this suite by the same path the real component would take. That the
270+
// shipped `<Dir>` makes this call and no other is asserted where `<Dir>`
271+
// lives.
272+
it("FF2: read, delete, ensureDirectory, Glob and TempDir all stop at their first provider call", function* () {
262273
const dir = yield* useFixture();
263274

275+
const ensures = function* (): Operation<void> {
276+
yield* registerComponents([
277+
{
278+
name: "Ensures",
279+
origin: "test://ensures",
280+
props: { type: "object", properties: {}, additionalProperties: false },
281+
*fn(): Operation<string> {
282+
yield* ensureDirectory({ cwd: dir, path: "made" });
283+
return "REACHED";
284+
},
285+
},
286+
]);
287+
};
288+
289+
const absent = yield* run(dir, "<Ensures />\n\nAFTER", ensures);
290+
expect(absent.ok).toBe(false);
291+
expect(parseFilesFatal(fatalCause(absent.error))?.kind).toBe("provider-unavailable");
292+
expect(absent.output).not.toContain("AFTER");
293+
// The operation is what failed, not the element around it: nothing after the
294+
// call inside the component ran either.
295+
expect(absent.output).not.toContain("REACHED");
296+
expect(yield* exists(join(dir, "made"))).toBe(false);
297+
264298
for (const source of [
265299
'<File path="notes.md" />\n\nAFTER',
266300
// Deletion has no preliminary check to fail at, so its first and only

‎packages/runtime/files.ts‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,12 +116,19 @@ const REASONS: readonly FilesReason[] = [
116116
];
117117

118118
/** The operations whose failure carries no commit outcome. */
119-
export type FilesOperation = "check-file-path" | "read" | "delete" | "glob" | "temporary-directory";
119+
export type FilesOperation =
120+
| "check-file-path"
121+
| "read"
122+
| "delete"
123+
| "ensure-directory"
124+
| "glob"
125+
| "temporary-directory";
120126

121127
const OPERATIONS: readonly FilesOperation[] = [
122128
"check-file-path",
123129
"read",
124130
"delete",
131+
"ensure-directory",
125132
"glob",
126133
"temporary-directory",
127134
];
@@ -244,6 +251,20 @@ export interface FilesHandler {
244251
* that same success.
245252
*/
246253
deleteFile(input: FilePathInput): Operation<Result<void>>;
254+
/**
255+
* Make this path name a directory, and answer with nothing.
256+
*
257+
* Recursively creates the target and any missing parent. An existing
258+
* directory is already the answer, so it succeeds without replacing it,
259+
* clearing it or touching what is in it; a file or another non-directory, at
260+
* the target or anywhere on the way to it, is a refusal.
261+
*
262+
* Mandatory like the rest, and Unit for the same reason `deleteFile` is: a
263+
* document that asked for a directory to exist has been answered by its
264+
* existence. Nothing comes back to branch on — no path, no handle, and no
265+
* word on whether this call is what created it.
266+
*/
267+
ensureDirectory(input: FilePathInput): Operation<Result<void>>;
247268
/** Sorted, deduplicated, POSIX-separated paths of the regular files that match. */
248269
globFiles(input: GlobInput): Operation<Result<string[]>>;
249270
/**
@@ -827,6 +848,10 @@ export const Files: Api<FilesHandler> = createApi<FilesHandler>("executablemd.ru
827848
throw new FilesProviderUnavailableError();
828849
},
829850
// deno-lint-ignore require-yield
851+
*ensureDirectory(_input: FilePathInput): Operation<Result<void>> {
852+
throw new FilesProviderUnavailableError();
853+
},
854+
// deno-lint-ignore require-yield
830855
*globFiles(_input: GlobInput): Operation<Result<string[]>> {
831856
throw new FilesProviderUnavailableError();
832857
},

0 commit comments

Comments
 (0)