Repository navigation
Commit b952af6
authored
📦 Seal one workflow run's evidence into a portable .xmd file (#610)
* Specify portable XMD workflow artifacts
* 📝 Plan portable XMD artifact implementation
* 📦 Seal one workflow run's evidence into a portable .xmd file
Version 1 of the XMD artifact container, and the verifier every later
artifact operation reads one through.
A writer takes one complete detached snapshot — the artifact frontier, the
run record, its executions and prior fork lineage, every journal row with
its exact filtered record and provenance, every retained Workspace root,
DOFS manifest and blob, the Repository and Worktree records, the suspension
answers, the Agent session mappings, and the authenticated definition source
closure — and writes it into one new SQLite file in one transaction, under
the `XMDA` marker that tells it apart from a live run store on sight.
Identity is a statement about the evidence rather than about the file. The
canonical manifest enumerates every record and retained byte with its kind,
its complete logical natural key as canonical JSON, its encoding, its length
and its hash; page size, insertion order, free space and host path are
outside it. Two containers holding the same snapshot derive the same
identity and report different final-file digests, which is what makes one a
semantic identity and the other a transport digest.
The reader is total. Path admission, a read-only open, integrity, the family
marker, the container version, the artifact version, the exact declared
schema, every entry's declared length and hash, every parsed record and
every reference between them, and only then the stored manifest and identity
compared against the ones the accepted content produces. Nothing partial is
ever returned, and the stored manifest is comparison evidence rather than
the source of the inventory — a reader that trusted it would be asking a
tampered file to describe its own tampering.
Opening never writes: no journal mode, no migration, no pragma that writes,
no sidecar, no lock. A refusal leaves the file byte- and mode-identical,
including from a directory nobody may write to.
`.xmd` and the versioned semantic contract are the public boundary. The
marker, the two tables and the SQL stay behind a provider-internal barrel,
and the suite asserts that neither published entrypoint names any of them.
Two live Workspace parsers gained a rejection parameter and one pure DOFS
manifest decoder was factored out of the row that holds it, so a detached
copy is validated by the same code a live run uses without borrowing a
sentence that tells an operator to restore a run from a backup.
This story exposes no CLI, takes no executor lock, reads no live run store
and creates no fork. Export, artifact inspection and the artifact-source
fork remain unbuilt.
Refs #602
* 🔒 Read the artifact's version before its shape, and hand back bytes nobody can edit
Two corrections to the version-1 verifier, and the assertions that prove each.
The artifact format version is now read between recognizing the container and
comparing its declared schema, which is the order the specification already
stated. A build that changed the format may have changed the tables carrying
it, so a file declaring a format this one does not implement is an unsupported
version whatever its tables look like — reporting it as a schema disagreeing
with itself would be this build guessing about a version it cannot read. A
container that is version 1 and is not shaped like one is still damage.
Gate 10 now holds for every leaf, bytes included. `Object.freeze` refuses a
typed array holding elements, this runtime offers no immutable `ArrayBuffer` to
build one over, and a write-refusing `Proxy` fails `ArrayBuffer.isView`,
`Buffer.from` and `TextDecoder.decode` — it would hand a caller bytes nothing
can hash or read. So a byte leaf is a sealed original behind a frozen accessor
that answers with a copy: replacing the member is refused, and writing into
what a read returned lands on a copy nobody else holds. The consequence, which
belongs to whoever owns the value contract, is that a byte member is a fresh
array per read rather than a stable reference.
C5 gains the undeclared trigger its frozen row named and had no evidence for,
and the ordering evidence for its existing future-artifact row. C7 attempts
both edits against every byte leaf and re-reads through the member rather than
through the array it was handed. The matrix is still nine criteria.
Both corrections were checked by breaking them: a plain freeze on a blob fails
C7, and the old gate order fails C5 with the schema refusal where the version
refusal is expected.
Production parsing and return paths carry no type assertions. The inventory
hands back identities as values rather than as text a decoder re-parsed and
asserted about, the stored encoding narrows through a predicate, a stored
identity parses through the shared JSON parser, the retained answer state
narrows through a helper, and the content-entry builders let their annotated
return type infer their literal encoding. The suites reach readonly values
through Reflect and parse stored records through the shared object parser.
Refs #602
* 🔑 Authenticate a retained answer against the wait it says it answers
A retained answer named a journal event and nothing checked what that event
was. The fixture proved how little that asks for: it pointed a consumed answer
at a Git effect and a pending one at a coroutine Close, carried no suspension
event at all, and the writer, its own read-back and both semantic tests took
it.
An answer is now held to the history it claims. The event it names has to be a
retained suspension request; that request has to be for this wait, by the
identity it carries twice over; the request is parsed rather than read, because
it reached the journal through a public durable operation; its fingerprint is
recomputed and compared, so a value judged against some other schema is not an
answer to this one; the value is judged again by the schema that wait retained;
and the row's state has to agree with the journal — consumed means the answer
was published, pending means it was not, and a publication may not precede its
request.
Every one of those questions is asked with the live contracts. The request
fingerprint had been written out twice already, once in delivery and once in
the resume that consumes an answer; it is now one function beside the request
parser, and all three callers share it. Judging the value is the same schema
compilation a delivery and an `<Elicit>` use.
Compiling that schema is an operation, so the semantic pass became one, and the
reader now closes its connection after reading content into memory rather than
holding it open through verification. That is what gate 6 always said, and the
gates after it never needed the file again.
The fixture carries two waits a run could really have reached — identifiers
derived from the run and the durable position, a request and its publication
for the answered one, a request alone for the wait the run is suspended at.
C8 gains the refusals that hold each rule: an event that is not a request, a
request for the other wait, a fingerprint from elsewhere, a value the schema
rejects, a consumed row with nothing published, and a pending row whose answer
already was.
Six of those seven pass against the check this replaces. That is what the gap
was.
Refs #602
* 📍 Hold a published answer to the position a publication occupies
A consumed row asked only whether *some* answer for its wait appeared later in
the journal. Later is not a position: `suspendFor()` publishes the request and
then the answer as two durable operations in one coroutine, back to back, so an
answer sits at the request's index plus one in that coroutine and nowhere else.
Anything looser accepts a publication belonging to another wait, in another
coroutine, as evidence that this one ended.
So a publication is now read at its durable position — recovered by counting
each coroutine's settled yields, which is what an index is; a Close settles
nothing and occupies none. Every publication has to name one wait in both the
fields that carry it and stand directly behind that wait's request, whether or
not a row claims it. A consumed row then has to find its publication exactly
there, carrying a successful result, and carrying the value the row retains.
The rich fixture had two faults behind that gap. Its suspension positions were
guesses rather than the coroutine's own count, and it marked the run's own
answered wait as inherited — which cannot happen, because fork creation copies
journal rows and Workspace state and never the retained answer rows behind
them. It now reads as the fork it claims to be: the source run's answered wait
inherited whole with no row of its own, and this run's own answered and pending
waits written by it. That first case is also why a publication no row claims is
accepted when it stands behind its request, and refused when it does not.
C8 gains the publication side: a mismatched identity, a publication moved to
another coroutine, an unsuccessful result, a value that is not the one
retained, a second publication for one wait, and one for a wait no request
opened. The seven answer-row regressions stay.
Removing the four new checks makes five of those six fall through to the
manifest comparison instead of the semantic gate, which is the difference
between recognizing tampered history and noticing the bytes moved.
Refs #602
* 🧷 Ask a publication which wait it stands behind, and whether it carried a value
Two ways a snapshot still reached a canonical identity it had not earned.
A request describes its wait twice, in `name` and in `suspensionId`, and the
structural pairing only compared the first. A pair agreeing about the name and
disagreeing about the identity names two waits, and neither of them is one the
other could have ended. Both fields are now compared, and asked of every
publication rather than only of one some retained row points at — inherited
history legitimately arrives as a request and its publication with no row
behind them, which is exactly where nothing else was looking.
A successful publication was read through `result.value ?? null`, so a result
carrying no value at all read as a publication of `null`. A wait that ended
with nothing and a wait whose retained answer is null are different histories,
and one was standing in for the other. The member is now asked for by name, and
only once it is there is the value compared.
Both regressions are writer-path, because neither can be made by damaging a
sealed file — a sealed file is one this build already accepted. They are
snapshots offered to `writeXmdArtifact()`, which has to refuse them before a
container exists, and the test asserts nothing is left at the destination.
At the previous commit each of them is written and given an identity. Removing
either check alone puts that back.
Refs #602
* 🪪 Hold every wait this run opened to the position that names it
A suspension identity is derived from the run and the exact durable position,
and a caller supplies neither — that is what keeps two waits in one procedure
from receiving each other's answer. The artifact never asked. A forged identity
used consistently across a request, the answer published behind it and the row
pointing at both agreed with itself, so every check that compared those three
to each other passed, and the container was written and given a canonical
identity.
So every request the retained run authored is now re-derived from its own run
id, its coroutine, and the index recovered by counting that coroutine's settled
yields, and refused when the position names some other wait. It is asked of the
request itself rather than of requests something else reaches: a wait nobody
answered, standing at a position that cannot produce its name, is the same
forgery with one fewer record in it.
Only its own. An inherited request was derived in the run it came from, at that
run's position, and re-deriving it against the child would refuse every fork
for having a history — the fixture's inherited request and its publication stay
accepted, and removing that exemption refuses the whole snapshot.
Request shape is now checked where the request is read rather than only where a
publication pairs with one, so a retained request that names its wait one way
in `name` and another in `suspensionId` is refused whether or not anything
stands behind it.
Refs #602
* 🧪 Keep the artifact suite out of the one runtime that cannot load it
Bun 1.3.14 has no `node:sqlite`, so the artifact suite does not fail there —
it never loads. The subject is the Deno provider's container, which imports
that module directly, so there is nothing to run under Bun and nothing a
different test would recover.
Recorded in a list of its own rather than in the shared Deno-only manifest,
because the two state different facts. That one says a subject belongs to
Deno. This one says a runtime is missing a built-in the subject imports — and
Node has since gained `node:sqlite` unflagged and runs this file today, so
folding the entry into the shared list would drop coverage Node is currently
giving.
Refs #602
---------
Co-authored-by: Taras Mankovski <[email protected]>1 parent 37aa8a6 commit b952af6
20 files changed
Lines changed: 6401 additions & 55 deletions
File tree
- packages/workflow
- src
- deno
- artifact
- workspace
- storage
- suspension
- tests
- support
- plans
- scripts
- specs
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
60 | 65 | | |
61 | 66 | | |
62 | 67 | | |
| |||
747 | 752 | | |
748 | 753 | | |
749 | 754 | | |
| 755 | + | |
| 756 | + | |
| 757 | + | |
| 758 | + | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
| 763 | + | |
| 764 | + | |
| 765 | + | |
| 766 | + | |
| 767 | + | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
| 771 | + | |
| 772 | + | |
| 773 | + | |
| 774 | + | |
| 775 | + | |
| 776 | + | |
| 777 | + | |
| 778 | + | |
| 779 | + | |
| 780 | + | |
| 781 | + | |
| 782 | + | |
| 783 | + | |
| 784 | + | |
| 785 | + | |
| 786 | + | |
| 787 | + | |
| 788 | + | |
| 789 | + | |
| 790 | + | |
| 791 | + | |
750 | 792 | | |
751 | 793 | | |
752 | 794 | | |
| |||
3167 | 3209 | | |
3168 | 3210 | | |
3169 | 3211 | | |
| 3212 | + | |
3170 | 3213 | | |
3171 | 3214 | | |
3172 | 3215 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
0 commit comments