Skip to content

Commit aa47442

Browse files
authored
💥 Contain document filesystem access behind API.Files (#227) (#403)
* 💥 Contain document filesystem access behind API.Files (#227) `<File>`, `<Glob>` and `<TempDir>` reached the host filesystem directly, so `xmd run` and a workflow run could not mean the same thing for one document. Document filesystem access now goes through `API.Files`, a contextual Api of whole semantic operations with no host default. The four CLI entrypoints install the host provider explicitly; a run with none installed fails rather than reaching the host. The three components make no filesystem call of their own and import no host path or fs module. What they keep is order: a write's lexical check runs before its children, and the semantic write that follows repeats admission and owns every later phase, so the earlier check authorizes nothing. Ordinary failures cross the boundary as frozen structural data — a reason from a fixed vocabulary and the phase it came from — and every printed message is byte-identical to before. A provider that is absent, refuses an operation, or breaks its own contract throws instead, and core's fatal traversal ranks it between a durability failure and a documentation failure, by identity and by structural tag. * 🐛 Keep core's error module out of the runtime's host graph `errors.ts` is in the graph a separately loaded copy of `printErrors` bundles, and importing the runtime's package root pulled the host Apis in with it — including a native addon no bundler can inline. The Files recognizer needs none of that, so it comes from the leaf module through a new `./files` subpath. The five-target job prepares with `deno install` rather than `deno task deps`: the task caches graphs this job does not need and reaches them by spawning a child, which does not survive the Windows runner's path handling. Its compile now carries the repository's isolation flags, and the rule that enforces them reads every compile in a workflow rather than letting the first invocation's flags answer for the rest. * 🐛 Sort the search results with the lib the Node typecheck targets `toSorted` needs es2023, which `tsconfig.node.json` does not select. The array is built from a Set on the line above, so nothing shared is being mutated. * 🔒 Make Files recognition total and strict, and keep cancellation cleanup fatal Structural parsing runs from `fatalCause`, which every generic catch in expansion consults. A provider is free to hand back a Proxy that refuses to be inspected, and one of these parsers throwing would replace the failure being classified with a failure about classifying it. Every read is now total, and so is the cause traversal: a hostile wrapper narrows what discovery finds instead. Recognizing a Files fatal is a decision to let that exact object travel onward by identity, so it now requires the whole public contract — frozen data with no extra fields, the fixed diagnostic for its kind, and no cause. A candidate carrying a raw message or an errno chain is replaced by a fresh invariant rather than preserved. Durability recognition is unchanged: it stays #394's class-based mechanism, and only the Files boundary is crossed by a second loaded copy. A host cleanup that fails while cancellation is unwinding has no outcome to report beside it. It leaves the scope as a fixed teardown invariant instead of manufacturing a write result, carrying neither the platform's error nor the generated temporary's name. The loaded-copy claim is now proved by a real second copy: the Files module is bundled, imported as its own module, and its failures are recognized in both directions. * 🔒 Rebuild every Files outcome from validated parts Recognizing a Files fatal hands that exact object onward, so the contract now covers the whole Error: the fixed name and diagnostic for its kind, frozen data with exactly the kind's fields, no cause, and no other enumerable member — string or symbol. A path riding on `name`, on an extra property, or under a symbol key fails the contract, and `invokeFiles` replaces the candidate rather than preserving it. A `Result` is only conventionally a Result. The TypeScript signature is a claim about the provider, not a guarantee, so a component that read `ok`, `value`, or `error` first would be the thing that ran a hostile accessor — outside anything that sanitizes. The core wrappers now inspect the container totally and rebuild every outcome from validated parts: no provider-originated container, error object, or payload reaches `<File>`, `<Glob>` or `<TempDir>`, and a search result is copied rather than passed along. A container that will not say how it settled, and a success it cannot describe, are provider-contract failures. A malformed non-write failure is not: the vocabulary already has a sentence for it, so a fresh generic failure is substituted and the document carries on. FA24 now runs all six orderings of the three fatal kinds, through both wrappers. * 🔒 Tell an unreadable Result member apart from an absent one `undefined` was standing for three different answers: a member read fine and held undefined, a member was absent, and reading a member threw. Collapsing them let two containers through that never described their outcome — `checkFilePath` accepted a success whose `value` refused to be read, and a non-write failure whose `error` refused was downgraded to the printable generic. Presence and readability are now asked separately, and every operation says which it requires. A search array is copied by index through the same reader, so length and element traps are covered too and the iterator is never consulted. The one place the contract bends is `checkFilePath`: Effection spells a payload-free success as its shared `Unit`, `{ ok: true }` with no `value` member at all, so absence there is the ordinary success rather than a failure. What is refused is a `value` that cannot be read, and one that is present but is something other than undefined. The seam now passes live Proxies around real `Ok`/`Err` values through `API.Files`, so the boundary is the first thing to run a hostile trap — the previous JSON round-trip invoked the getter inside the provider handler and proved only the already-covered handler-throw path. * 🔒 Recognize a search result's array brand totally `Array.isArray` is itself an operation on provider-controlled data: it throws on a revoked Proxy. Running it outside the total readers let a raw TypeError leave the boundary untagged, so a search whose result was revoked before it was returned surfaced the platform's message and let the document carry on. The brand check now answers instead of throwing, and a value whose array identity cannot be inspected is malformed success data like any other. The guard sits inside the payload contract rather than around the call site, so it stays the thing under test: restoring the unguarded call reds the regression. The specification catalog gains the rows for behavior already implemented — HF12b and FF11 through FF15 — and the provider-failure prose now separates an outcome that will not say what it is, which is a contract violation, from one that reads fine and reports a failure this version does not recognize, which is the generic sentence. * 📝 State the non-write failure rule for every non-write operation The provider-failure passages named only a read and a search, which left path admission and TempDir unstated even though they take the same path: readable data that does not validate selects the generic sentence there too. The FF14 row described a stricter rule than the boundary implements. It now matches FF14c and FF15: an unreadable settlement is a contract failure, so is an unreadable selected member, and an absent success value only where the operation carries one — path admission succeeds without one. A readable but unrecognized non-write failure stays printable. FF11 is renamed to what it asserts. Declining to recognize a hostile shape is not recognizing it as a valid structural failure. --------- Co-authored-by: Taras Mankovski <[email protected]>
1 parent 19bda00 commit aa47442

35 files changed

Lines changed: 5546 additions & 611 deletions

‎.github/workflows/ci.yml‎

Lines changed: 109 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -255,6 +255,103 @@ jobs:
255255
--component-dir smoke-test/agent/components \
256256
--raw
257257
258+
# The host document-filesystem contract, on every target a release ships.
259+
#
260+
# `test-deno`, `test-node`, and `test-bun` run the whole corpus on Linux x64
261+
# and prove the contract holds there. What they cannot prove is that it holds
262+
# on the other four triples: the host adapter's containment is path arithmetic
263+
# plus `realpath`, and both are the platform's — Windows has drive letters,
264+
# UNC paths, junctions, and reparse points that POSIX does not, and the two
265+
# macOS rows resolve `/var` through a symlink that a naive comparison reads as
266+
# an escape.
267+
#
268+
# So this row is focused rather than exhaustive: one suite, plus a compiled
269+
# probe, on each of the five. The compiled probe is the second half of the
270+
# claim — the shipped artifact is a binary, and the adapter reaches
271+
# `node:path`, `node:fs`, and `node:os` through whatever `deno compile` put in
272+
# its graph.
273+
filesystem-contract:
274+
strategy:
275+
fail-fast: false
276+
matrix:
277+
include:
278+
- runner: macos-15
279+
target: aarch64-apple-darwin
280+
- runner: macos-15-intel
281+
target: x86_64-apple-darwin
282+
- runner: ubuntu-24.04
283+
target: x86_64-unknown-linux-gnu
284+
- runner: ubuntu-24.04-arm
285+
target: aarch64-unknown-linux-gnu
286+
- runner: windows-2025
287+
target: x86_64-pc-windows-msvc
288+
runs-on: ${{ matrix.runner }}
289+
defaults:
290+
run:
291+
shell: bash
292+
steps:
293+
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
294+
295+
- uses: denoland/setup-deno@e95548e56dfa95d4e1a28d6f422fafe75c4c26fb # v2.0.3
296+
with:
297+
deno-version: v2.9.5
298+
299+
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
300+
with:
301+
node-version: "22"
302+
303+
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
304+
305+
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
306+
with:
307+
bun-version: 1.3.14
308+
309+
# `deno install` rather than `deno task deps`: the task also caches the
310+
# graphs a browser build and a release compile walk, and it reaches them
311+
# by spawning a child — which does not survive the Windows runner's path
312+
# handling. Nothing here builds the bundle or compiles the CLI, so the
313+
# plain frozen install is the whole preparation this job needs.
314+
- name: Install dependencies
315+
run: deno install --frozen
316+
317+
# The compile below runs under `--node-modules-dir=none`, which resolves
318+
# npm packages from the Deno cache rather than from `node_modules`. This
319+
# caches the probe's graph in that mode without touching the layout the
320+
# step above just created.
321+
- name: Cache the probe's graph for a compile
322+
run: >
323+
deno install --entrypoint --node-modules-dir=none --frozen
324+
scripts/files-contract-probe.ts
325+
326+
# Deno first, and the compile with it: `pnpm install` and `bun install`
327+
# each rewrite `node_modules` into their own layout, so a Deno step after
328+
# one of them resolves through links the other pruned (#279).
329+
- name: Host contract under Deno
330+
run: deno test --allow-all --frozen packages/runtime/tests/host-files.test.ts
331+
332+
- name: Host contract as a compiled binary
333+
run: |
334+
set -eu
335+
deno compile --node-modules-dir=none --cached-only --frozen --allow-all \
336+
--output dist/files-contract-probe scripts/files-contract-probe.ts
337+
if [ -f dist/files-contract-probe.exe ]; then
338+
./dist/files-contract-probe.exe
339+
else
340+
./dist/files-contract-probe
341+
fi
342+
343+
- name: Install the Node layout
344+
run: pnpm install
345+
346+
- name: Host contract under Node
347+
run: pnpm exec tsx --tsconfig tsconfig.node.json --test packages/runtime/tests/host-files.test.ts
348+
349+
- name: Install the Bun layout
350+
run: bun install
351+
352+
- name: Host contract under Bun
353+
run: bun test --timeout=300000 packages/runtime/tests/host-files.test.ts
354+
258355
# The same chain `deno task verify:clean` runs locally. It is the regression
259356
# for #279's ownership claim: a build that installs anything moves the
260357
# prepared-state fingerprint, prunes pnpm's links, and fails the resolution
@@ -384,7 +481,18 @@ jobs:
384481
run: bun run packages/cli/src/bun.ts test smoke-test/test-agent/README.md --raw
385482

386483
green:
387-
needs: [lint, test-deno, jsr, smoke, composability, site, test-node, test-bun]
484+
needs:
485+
[
486+
lint,
487+
test-deno,
488+
jsr,
489+
smoke,
490+
filesystem-contract,
491+
composability,
492+
site,
493+
test-node,
494+
test-bun,
495+
]
388496
if: always()
389497
runs-on: ubuntu-latest
390498
steps:

‎architecture.md‎

Lines changed: 71 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,10 @@ Existing documents and code get aligned to this section retroactively.
4242
| blocker | execution needs input (auth, a human answer); not a failure |
4343
| suspension | a durable wait: a crash restarts into the same wait |
4444
| Workspace | the provider-neutral, run-owned environment that supplies retained filesystem, repository, process and working-directory capabilities to a workflow |
45+
| document filesystem | the files a document names in its own text, reached only through `API.Files`; distinct from the host paths the engine's own control plane reads |
46+
| Files provider | the installed implementation of `API.Files` for a document execution: the host provider under `xmd run`, a transaction-bound provider under a workflow run. There is no default, and no provider falls back to another |
47+
| stable host namespace | the condition the host Files provider's containment is stated under: no other process replaces a directory, symlink, junction or reparse point between the moment a path is observed and the moment it is used |
48+
| Files infrastructure failure | a Files provider that is absent, that refuses an operation, or that broke its own contract; fatal like a durability failure, and never a printed error |
4549
| ephemeral | a replay classification for an operation, context or attachment that runs again to reconstruct live execution; its result is not substituted from the journal and it owns no durable workflow state |
4650
| live binding | an execution-owned value reconstructed ephemerally for the current document execution; it is visible only to constructs that explicitly consume the live binding overlay and never enters interpolation or the journal |
4751
| attached service | a scoped host process that publishes its authenticated loopback endpoint through the XMD service handshake protocol and remains supervised for the lifetime of its service attachment |
@@ -688,7 +692,7 @@ the process may terminate, and a later document execution arrives back at the
688692
same wait. An error that reveals a blocker (an expired login) reaches suspension
689693
through middleware; waiting itself is never raised.
690694

691-
### 8. Durability failures are outside the model
695+
### 8. Durability and Files infrastructure failures are outside the model
692696

693697
A durability failure (§6.11) says the journal no longer describes the document
694698
execution. No middleware sees it; it is never the document's own outcome. A
@@ -705,6 +709,69 @@ throws. A pre-persistence policy rejection remains the policy's ordinary
705709
document failure; the guarded stream marks that boundary before the backing
706710
append and does not activate the fail-stop state.
707711

712+
A Files infrastructure failure is outside the model on the same terms. A
713+
missing provider, a refused operation, and a provider that broke its own
714+
contract are none of them things the document did or can act on, and printing
715+
one would let every step after the file work run as though the file work had
716+
happened. No middleware converts one, and no printing boundary prints one.
717+
718+
Both are discovered through one cycle-safe traversal of the whole cause graph,
719+
and precedence is decided by kind rather than by position: a durability failure
720+
first, then a Files infrastructure failure, then a documentation failure. The
721+
selected failure comes back by identity, because a fail-stop that records "the
722+
first error" has to record the one that happened.
723+
724+
The two kinds are *recognized* by different mechanisms, and the difference is
725+
deliberate. A durability failure is recognized exactly as it always has been —
726+
by class, against the failure types durable-streams exports — and that is
727+
unchanged. A **Files** infrastructure failure is recognized structurally: a
728+
stable tag on frozen data, the fixed diagnostic for its kind, and no cause,
729+
never `instanceof`. It has to be, because the Files boundary is the one a
730+
separately loaded copy of a package reaches across, where `instanceof` answers
731+
false. Recognizing one is also a decision to let that exact object travel
732+
onward, so a candidate carrying anything else — a raw message, a cause chain,
733+
an extra field — fails the contract and is replaced by a fresh invariant rather
734+
than preserved.
735+
736+
The traversal reads values it did not create, so every read in it is total: a
737+
thrown Proxy or a failing accessor narrows what discovery finds instead of
738+
replacing the failure being classified.
739+
740+
## The document filesystem boundary
741+
742+
Contextual routing is not authority. `API.Files` decides *which* provider
743+
answers a document's file operations; what a provider is allowed to do is
744+
decided by the provider, from identities the contextual layer cannot supply.
745+
746+
The Api's operations are whole semantic acts — read this path, replace this
747+
path, list what these patterns select — rather than steps a caller sequences.
748+
The one preliminary operation, `checkFilePath`, is deliberately weak: pure path
749+
arithmetic, no filesystem access, and it returns nothing usable. `<File>` calls
750+
it to decide whether a write's children may expand, and the later write repeats
751+
the same admission from the same authored path. Nothing is handed between them,
752+
so a check that was skipped or answered elsewhere authorizes nothing.
753+
754+
The two providers make different containment claims, and both are stated rather
755+
than implied:
756+
757+
- **`xmd run`** resolves document paths in the caller's own filesystem. It
758+
refuses empty, absolute, and lexically escaping paths without touching the
759+
filesystem, and refuses an observed outward symlink once resolution can see
760+
one. That holds while the host namespace is stable; it is not a sandbox, and
761+
closing the replacement window would require a native dependency this project
762+
does not take.
763+
- **A workflow run** resolves document paths in the run-owned Workspace's
764+
logical filesystem, inside the caller-owned transaction. A document path
765+
never becomes a host path, so there is no host namespace to replace.
766+
767+
Neither claim covers a native command a document runs.
768+
769+
Failure data crosses the boundary as a plain frozen object under a stable tag,
770+
carrying a reason from a fixed vocabulary and the phase it came from. No
771+
message, errno code, resolved path, temporary name, or symlink target crosses.
772+
Consumers parse that data before reading a field; a write whose data does not
773+
validate is a provider-contract failure rather than a commit state to invent.
774+
708775
## Attempts
709776

710777
- The journal records every attempt in full; replay restores the outcome
@@ -831,6 +898,9 @@ Status is measured against main.
831898
| Workspace coordination API | fails closed by default and lets a Workspace operation explicitly select provider coordination | built on the #365 stack; the atomic Deno Workspace handler is unbuilt |
832899
| explicit WorkflowRun journal route | binds one already-filtered publication to one exact active transaction and otherwise uses ordinary serialized journal storage | built on the #365 stack |
833900
| `API.Service` / `startService()` | creates an authenticated, supervised loopback service attachment through a provider-neutral operation | built on main |
901+
| `API.Files` | routes every document filesystem operation to the installed provider, with no host default and structural failure data | built on the #227 stack |
902+
| host Files provider / `useHostFiles()` | resolves document paths in the caller's filesystem, containing them while the host namespace is stable; installed by all four CLI entrypoints | built on the #227 stack |
903+
| transaction-bound Files provider | resolves document paths in the run-owned logical Workspace inside the caller-owned transaction | unbuilt; the adapter is #227's second layer, and workflow effect coordination and CLI reachability remain unbuilt |
834904
| `service=<binding>` | publishes the attachment's endpoint into the live binding overlay for its invocation | built on main |
835905
| `ephemeral eval` | reconstructs live middleware and bindings without a journal entry | built on main |
836906
| `useWorkflowServiceDenial()` | provides and tests a non-delegating workflow service denial provider; #366 will install it in future start and resume scopes | built on main; no workflow CLI execution branch exists yet |

‎packages/cli/src/bun.ts‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
import { main } from "effection";
99
import { fileURLToPath } from "node:url";
1010
import process from "node:process";
11-
import { API } from "@executablemd/runtime";
11+
import { API, useHostFiles } from "@executablemd/runtime";
1212
import { compileDataUri } from "@executablemd/core";
1313
import { runXmd } from "./cli.ts";
1414
import { useBunService } from "./bun-service.ts";
@@ -30,5 +30,10 @@ await main(function* (args) {
3030
},
3131
{ at: "min" },
3232
);
33+
// Document filesystem access resolves in the caller's own filesystem here.
34+
// It is installed explicitly, and at the same depth, because `API.Files` has
35+
// no host default: a run with no provider must fail rather than reach the
36+
// host by accident.
37+
yield* useHostFiles();
3338
yield* runXmd(args, useBunService);
3439
});

‎packages/cli/src/compiled.ts‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
import { main } from "effection";
99
import process from "node:process";
10-
import { API } from "@executablemd/runtime";
10+
import { API, useHostFiles } from "@executablemd/runtime";
1111
import { compileDataUri } from "@executablemd/core";
1212
import { runXmd } from "./cli.ts";
1313
import { useCompiledService } from "./compiled-service.ts";
@@ -27,5 +27,10 @@ await main(function* (args) {
2727
},
2828
{ at: "min" },
2929
);
30+
// Document filesystem access resolves in the caller's own filesystem here.
31+
// It is installed explicitly, and at the same depth, because `API.Files` has
32+
// no host default: a run with no provider must fail rather than reach the
33+
// host by accident.
34+
yield* useHostFiles();
3035
yield* runXmd(args, useCompiledService);
3136
});

‎packages/cli/src/deno.ts‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
import { main } from "effection";
1111
import { fileURLToPath } from "node:url";
1212
import process from "node:process";
13-
import { API } from "@executablemd/runtime";
13+
import { API, useHostFiles } from "@executablemd/runtime";
1414
import { compileDataUri } from "@executablemd/core";
1515
import { runXmd } from "./cli.ts";
1616
import { useDenoService } from "./deno-service.ts";
@@ -32,5 +32,10 @@ await main(function* (args) {
3232
},
3333
{ at: "min" },
3434
);
35+
// Document filesystem access resolves in the caller's own filesystem here.
36+
// It is installed explicitly, and at the same depth, because `API.Files` has
37+
// no host default: a run with no provider must fail rather than reach the
38+
// host by accident.
39+
yield* useHostFiles();
3540
yield* runXmd(args, useDenoService);
3641
});

‎packages/cli/src/node.ts‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
import { main } from "effection";
1515
import { fileURLToPath } from "node:url";
1616
import process from "node:process";
17-
import { API } from "@executablemd/runtime";
17+
import { API, useHostFiles } from "@executablemd/runtime";
1818
import { compileTempFile } from "@executablemd/core";
1919
import { runXmd } from "./cli.ts";
2020
import { useNodeService } from "./node-service.ts";
@@ -37,5 +37,10 @@ await main(function* (args) {
3737
},
3838
{ at: "min" },
3939
);
40+
// Document filesystem access resolves in the caller's own filesystem here.
41+
// It is installed explicitly, and at the same depth, because `API.Files` has
42+
// no host default: a run with no provider must fail rather than reach the
43+
// host by accident.
44+
yield* useHostFiles();
4045
yield* runXmd(args, useNodeService);
4146
});

0 commit comments

Comments
 (0)