Repository navigation
Commit 0962ea2
authored
📤 Seal one workflow run's evidence into a file, and put nothing there unless it worked (#615)
* 🧱 Read one committed frontier as the values an artifact is made of
The first half of export: a provider-neutral request and result, and a reader
that turns one run into the detached snapshot #602's writer accepts.
`readForkSource()` slices the prefix a checkpoint selects, because a fork
inherits history up to a point and writes its own beginning. An export inherits
nothing — it records what the run committed — so this reads every journal row,
every retained root, and the families a fork has no use for: document
executions, prior lineage, suspension answers and Agent session mappings. Two
readers over the same tables rather than one with a mode, because "up to here"
and "everything" differ in what each is allowed to leave out.
Nothing repairs. Every statement is a SELECT on a connection the caller opened
read-only, so a run whose host died mid-execution exports as the running run it
is; inventing a tidier frontier would record something that never happened.
The definition source closure arrives already authenticated, because turning
retrieval metadata back into bytes means opening a repository and repository
access belongs to the host that has it. What is checked here is the other half:
that the closure describes the definition this frontier actually retains, so a
caller cannot seal one run's evidence around another run's Markdown.
The artifact value model moves to src/artifact/, above the provider. The
lifecycle API is provider-neutral and exported from the package root, so naming
a closure in its signature would have made `@executablemd/workflow`
type-depend on the Deno adapter — the one boundary that entrypoint exists to
keep. The SQLite encoding stays private where it was.
Three live readers gained whole-table forms beside their by-id ones, and the
fork's checkout filter became optional rather than duplicated. An export reads
these rows through the same parsers a continuation does.
Refs #600
* 🔐 Choose a frontier under the lock, and seal it once the lock is gone
`WorkflowLifecycle.export()` on the Deno provider. It takes the run's executor
lock, reads one committed frontier through the ordinary inspection path, and
lets the lock go before a single byte is written.
The lock buys one thing: that the frontier is a consistent committed view
rather than a moving one. Once the rows are values in memory nothing can
invalidate them, so the container is built with the lock already released and a
large export does not keep a run unrunnable for as long as it takes to write.
Reading through the inspection path is what gives a crashed source the existing
recovery-copy discipline for free — a private copy is recovered and read, and
the retained database is left exactly as it was found. Export settles nothing,
relabels nothing and appends nothing.
A run somebody is executing right now has no settled frontier to choose, so it
is refused as its own condition rather than queued. Waiting would turn "this
run is busy" into a hang.
The suite proves the refusal leaves no file, that a settled run round-trips
through the container and reads back as the run it was, that the run is
unchanged afterwards, and that a closure describing another definition is
refused before anything is written. It reaches node:sqlite, so it joins the
Bun exclusion list in this commit rather than in the one that discovers CI.
Refs #600
* 📤 Give a run's evidence a file, and put nothing there unless it worked
`xmd workflow export <run-id> --output=<path.xmd>`.
Three steps that each refuse on their own terms. The run's snapshot says what
definition it retains and where that definition could be fetched from; the
repository turns that back into the exact Markdown and proves it against the
identities the definition holds; the provider chooses a frontier under the
executor lock and writes the container.
The destination is checked by the grammar, because a missing --output, a name
that does not say what the file is, and an attempt to stream a container down a
pipe are all mistakes a caller fixes by typing something else. Whether the run
can be exported at all is a different question, asked after that one passes.
The artifact is built in a temporary directory beside its destination and moved
onto it. A rename within one directory is atomic, so an onlooker sees the
finished file or no file, never a half-written one wearing the name somebody
asked for; building elsewhere and copying would give up exactly that. What a
failure leaves behind is a temporary directory nobody was told about, rather
than a file at the path a user will go looking at.
The two digests are reported under labels that say which question each answers:
one names the evidence and survives a re-encoding, the other names the bytes
that were transferred. `--output`'s own help says the artifact carries the run's
complete retained Workspace and may hold source, generated files and secrets.
A definition pins a commit and a path, never the document's own identity, so the
root blob identity is derived from the bytes the repository handed back — by the
same function the container compares them with, rather than a second spelling
of it.
Refs #600
* 🧾 Prove the export command's own boundary
What the command owns, rather than what the container does with a snapshot.
A completed run seals to the path a caller named, reports both digests under
labels that say which question each answers, and leaves no staging directory
beside the result. Four destinations are refused — none at all, a name that
does not say what the file is, a stream, and somebody's existing file — and the
one that already existed still holds its own bytes afterwards. A run that does
not exist creates nothing at the target.
The report goes to stdout, where every other inspection result goes; the tests
read it there rather than asserting the shape of a log.
Refs #600
* 🔒 Fetch a run's source instead of being handed it
A closure travelling on the export request was source somebody supplied. The
request reaches the provider through the contextual lifecycle name, so anything
holding that name could offer arbitrary Markdown with a correctly recomputed
blob identity, and every check downstream would agree with it: the descriptor
matched, the bytes hashed to what the closure declared, and the container
sealed it as evidence about a run.
So the request carries no source at all. A host installs a reader into the
provider at installation and it is captured in that closure — the only way to
change what an export seals is to be the host that installed it. The provider
calls it with the definition the frontier it just read actually retains, and
still asks whether what came back describes that definition, because a reader
is host code too. A host that installs none cannot export, which is the honest
answer: an export it could perform without one would seal source nobody
fetched.
The lock is unchanged — selection and detachment only. Reading a repository
happens after it is released.
Publication was a rename behind an existence check, which is a promise about a
moment that has already passed by the time the move happens. It is a link now:
the destination is created or the name is already taken, atomically, so a file
that appeared while the artifact was being written survives untouched. Cleanup
runs before success is printed, and an export that cannot clean up takes its
published target back rather than leaving something that looks finished beside
state nobody owns. The filesystem steps are plain injected operations,
defaulting to the real ones.
`readRetrievalMetadata` projected one column into a parser that reads the whole
retrieval record, so every CLI export failed on a run that had retrieval
metadata. It reads the row.
XE4 is the regression: unchanged descriptor fields, arbitrary Markdown, its real
blob id, offered on the request — and the artifact still holds what the host
read. XE5 covers a host with no reader.
Refs #600
* 🧯 Make each publication step fail on purpose, and say which one ran
The evidence that was missing, and the reason the last attempt at it was worth
deleting: an export that refuses early leaves no target and no staging state
either, so a suite that only looked at the directory afterwards passed against a
build where publication was never reached at all.
So every case records which injected operations ran and asserts that first. A
racing target proves `link` was reached and that the racer's bytes are still its
own. A publication failure proves `link` ran, nothing was reported, and staging
is gone. A cleanup failure asserts the order `link -> remove -> unlink`, which
is the whole claim: the artifact really was published, cleanup really did fail,
and the target really was taken back. A cancellation captures the staging
directory while it exists and then finds it removed with no target beside it.
The fixture starts its run through the shared launcher rather than a raw
subprocess. Spreading the parent environment into the child left the run
somewhere neither the CLI nor the test agreed on, which is what made every case
refuse before publication.
Refs #600
* 🧪 Halt a real export, and compare a real run against what it sealed
Three pieces of evidence #600 was missing, and one classification that was wrong.
The cancellation is a cancellation. `exportArtifact` runs in a task of its own,
its injected publication step captures the staging directory and then suspends
without returning, and the test halts the task. What unwinds is Effection tearing
a live scope down, so the staging resource's own backstop is what removes the
directory — not `publish`'s caught-error path, which a thrown Error would have
taken instead. Both output channels are captured and asserted empty, because a
success line and a refusal are the two ways that case could have passed for the
wrong reason. Disabling the backstop fails it.
XE7 is one representative run compared family by family. It executes the document
that is also its definition — a Repository cloned from a remote holding a
committed symbolic link and an executable file, a branch, a written file, a
staged commit and a Worktree — and exports it through `WorkflowLifecycle.export`.
Every comparison has a source outside the artifact: the Workspace against the
run's own DOFS filesystem walked path by path, the definition against the objects
Git holds, the Repository and Worktree rows against the remote that was cloned
and the commit the run's history records making, and the retained history against
the journal the run committed. Dropping the worktrees, or carrying only the
current root, fails it. A snapshot handed to `writeXmdArtifact()` would have
proved the container and said nothing about what an export selects.
WFX4-WFX6 are the command's own refusals, each as a process: a real executor
holding the run's lock, a repository that is no longer one, and a repository that
answers for the pinned commit with another object. Each reports failure, names
why, and leaves neither the destination nor the staging directory beside it.
Both export suites were listed as Bun-only exclusions on the claim that Node runs
them. Node does not: every case takes the run's executor lock, and that lock is
`Deno.openSync` through the `Deno` global. They were red in the Node shard from
the commit that added them, so they move to the Deno-only list with the reason
the source actually gives.
Refs #600
* 🧹 Keep the artifact's record shapes out of the runtime-neutral surface
DLC13 failed on all three runtimes. `src/artifact/types.ts` and
`src/artifact/source.ts` sat in the shared tree while naming what only the Deno
adapter has: the retained shapes they compose are DOFS manifests, DOFS blobs,
SQLite-retained Repository, Worktree, answer and Agent-session rows, their
content kinds spell `dofs-*`, and the source reader hashes with `node:crypto`.
The runtime-neutral surface is not allowed to reach any of that, and the check
that says so is not a style rule — it is what keeps a second adapter possible.
So both modules go back under `src/deno/artifact/`, where the rows they describe
live. What the provider-neutral lifecycle actually needed from them was one type:
`XmdArtifactFrontier`, three strings naming which run, which final committed
event and which Workspace root. That is the lifecycle's own decision rather than
the container's record of it, so it is declared beside the export request and
result and the container imports it. The dependency now runs container →
lifecycle, and nothing is restated in two places.
The published surfaces follow the same line. `mod.ts` keeps the export request,
its result and the frontier; the closure types, `gitBlobIdentity` and the source
reader move to `deno.ts`, and the CLI reads them from there — it only ever
reaches them through its Deno host. The container's own encoding stays private
on both entrypoints, which xmd-artifact.test.ts C9 still enforces.
No behavior changes: the same values travel the same path.
Refs #600
* 🔒 Make publication the point an export stops being reversible
The architect's amendment. Publication keeps both properties it had — the whole
artifact appears at once, and a name something else took is refused rather than
replaced — and gains the one it was missing: nothing fallible happens after it.
Everything private now finishes first. The completed artifact is moved out of
its staging directory onto one hidden sibling entry, and the directory is
removed, so by the time the destination is touched there is no temporary whose
cleanup could fail. A failure, a publication refusal or a cancellation up to
that moment leaves no file where the caller asked for one.
Then the destination is linked, and that link is the commit. It is synchronous,
with the narrow Code Rule 2 exception and the invariant named at the site: an
operation can be halted between the kernel creating the entry and the caller
learning that it did, and an export that published without recording it would
tear down believing it had not. The commit is recorded with nothing suspended in
between, so no cancellation can arrive in the gap.
After that the artifact is never taken back. What is left is one private entry
holding the same inode under a name nobody was told about; removing it is best
effort, and a failure there is a warning naming the leftover beside the success
report, never instead of it. The rollback branch is gone. Reporting that nothing
was published while a complete artifact sits at the path was false, and the
unlink that made it true destroyed the evidence the caller asked for — and could
fail too, which is the defect: a failed export that left the artifact published.
This is the rule `<File>` already follows (executable-mdx-spec §6.13): once the
commit returns, the write is reported and the leftover is named. The artifact
spec and Story 2 criterion 3 now say the same thing.
WXP is drawn around the commit. Before it, WXP1 proves a racing destination
survives byte-identical, WXP2 that a failed link publishes nothing, WXP3 that a
cleanup failure stops the export with `link` never reached, and WXP4 that a real
halt — spawn, suspend in a pre-publication step, `task.halt()` — leaves neither
target nor private state and writes to neither channel. After it, WXP5 makes the
post-commit cleanup fail and proves the artifact is there and is a real
container, that the command exits zero and reports it, that the warning names
the retained entry, and that no removal was ever aimed at the destination. WXP6
proves the same on the ordinary path: `link` is the only step that names the
destination at all. Reinstating the rollback fails WXP5.
Refs #600
---------
Co-authored-by: Taras Mankovski <[email protected]>1 parent 84a8068 commit 0962ea2
26 files changed
Lines changed: 2447 additions & 67 deletions
File tree
- packages
- cli
- src
- tests
- workflow
- src
- deno
- artifact
- workspace
- lifecycle
- storage
- tests
- support
- plans
- scripts
- specs
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3209 | 3209 | | |
3210 | 3210 | | |
3211 | 3211 | | |
3212 | | - | |
| 3212 | + | |
3213 | 3213 | | |
3214 | 3214 | | |
3215 | 3215 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| 35 | + | |
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
| |||
59 | 60 | | |
60 | 61 | | |
61 | 62 | | |
62 | | - | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
63 | 67 | | |
64 | 68 | | |
65 | 69 | | |
| |||
0 commit comments