Skip to content

Commit 92c17da

Browse files
committed
🔑 Authenticate a retained answer against the wait it says it answers
A retained answer named a journal event and nothing checked what that event was. The fixture proved how little that asks for: it pointed a consumed answer at a Git effect and a pending one at a coroutine Close, carried no suspension event at all, and the writer, its own read-back and both semantic tests took it. An answer is now held to the history it claims. The event it names has to be a retained suspension request; that request has to be for this wait, by the identity it carries twice over; the request is parsed rather than read, because it reached the journal through a public durable operation; its fingerprint is recomputed and compared, so a value judged against some other schema is not an answer to this one; the value is judged again by the schema that wait retained; and the row's state has to agree with the journal — consumed means the answer was published, pending means it was not, and a publication may not precede its request. Every one of those questions is asked with the live contracts. The request fingerprint had been written out twice already, once in delivery and once in the resume that consumes an answer; it is now one function beside the request parser, and all three callers share it. Judging the value is the same schema compilation a delivery and an `<Elicit>` use. Compiling that schema is an operation, so the semantic pass became one, and the reader now closes its connection after reading content into memory rather than holding it open through verification. That is what gate 6 always said, and the gates after it never needed the file again. The fixture carries two waits a run could really have reached — identifiers derived from the run and the durable position, a request and its publication for the answered one, a request alone for the wait the run is suspended at. C8 gains the refusals that hold each rule: an event that is not a request, a request for the other wait, a fingerprint from elsewhere, a value the schema rejects, a consumed row with nothing published, and a pending row whose answer already was. Six of those seven pass against the check this replaces. That is what the gap was. Refs #602
1 parent 64194bc commit 92c17da

8 files changed

Lines changed: 381 additions & 77 deletions

File tree

‎packages/workflow/src/deno/artifact/read.ts‎

Lines changed: 42 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ function isXmdArtifactEncoding(value: unknown): value is XmdArtifactEncoding {
9292
export function* readXmdArtifact(path: string): Operation<Result<VerifiedXmdArtifact>> {
9393
try {
9494
yield* admitPath(path);
95-
return Ok(openArtifact(path));
95+
return Ok(yield* recognized(detach(path), path));
9696
} catch (error) {
9797
if (error instanceof WorkflowStorageError) {
9898
return Err(error);
@@ -101,7 +101,23 @@ export function* readXmdArtifact(path: string): Operation<Result<VerifiedXmdArti
101101
}
102102
}
103103

104-
function openArtifact(path: string): VerifiedXmdArtifact {
104+
/** What one connection hands over before it is closed. */
105+
interface SealedContent {
106+
readonly entries: readonly XmdArtifactContentEntry[];
107+
readonly manifest: Uint8Array;
108+
readonly identity: string;
109+
}
110+
111+
/**
112+
* Gates 2 through 7, and then the connection is gone.
113+
*
114+
* Everything the rest of recognition needs is in memory before this returns, so
115+
* no later gate is reading through a handle and nothing a caller could observe
116+
* outlives the file being open. It is also what lets the semantic pass be an
117+
* operation: judging a retained answer means compiling the schema its wait
118+
* retained, and that cannot happen inside a synchronous SQLite transaction.
119+
*/
120+
function detach(path: string): SealedContent {
105121
let database: DatabaseSync;
106122
try {
107123
database = new DatabaseSync(path, { readOnly: true });
@@ -114,7 +130,7 @@ function openArtifact(path: string): VerifiedXmdArtifact {
114130
// a reader that could change the file would not be reading evidence.
115131
database.exec("PRAGMA foreign_keys = ON");
116132
database.exec("PRAGMA busy_timeout = 5000");
117-
return readTransaction(database, () => verified(database, path));
133+
return readTransaction(database, () => sealedContent(database, path));
118134
} catch (error) {
119135
throw translateArtifactSqliteError(error, path);
120136
} finally {
@@ -153,7 +169,7 @@ function* admitPath(path: string): Operation<void> {
153169
}
154170
}
155171

156-
function verified(database: DatabaseSync, path: string): VerifiedXmdArtifact {
172+
function sealedContent(database: DatabaseSync, path: string): SealedContent {
157173
// Gate 3 and the first half of gate 4: integrity, the family marker, and the
158174
// container schema version — all of which a container answers before any
159175
// question about what its rows mean.
@@ -173,26 +189,40 @@ function verified(database: DatabaseSync, path: string): VerifiedXmdArtifact {
173189
checkXmdArtifactForeignKeys(database, path);
174190
const header = readHeader(database, path, container.containerVersion);
175191

176-
// Gate 6: everything into detached memory, so nothing a caller can observe
177-
// outlives the connection.
178-
const entries = readContent(database, path);
192+
// Gates 6 and 7: every entry held to what its own row declares, and all of it
193+
// into detached memory.
194+
return Object.freeze({
195+
entries: readContent(database, path),
196+
manifest: header.manifest,
197+
identity: header.identity,
198+
});
199+
}
179200

201+
/**
202+
* Gates 8 through 10, with nothing open.
203+
*
204+
* Nothing here can reach the file again, which is the point: what is being
205+
* decided is whether the bytes already in hand describe one snapshot a live
206+
* workflow contract accepts, and then whether the manifest they produce is the
207+
* one stored beside them.
208+
*/
209+
function* recognized(sealed: SealedContent, path: string): Operation<VerifiedXmdArtifact> {
180210
// Gate 8: the records themselves, and every reference between them.
181-
const contents = decodeXmdArtifactInventory(entries, path);
182-
verifyXmdArtifactSemantics(contents, path);
211+
const contents = decodeXmdArtifactInventory(sealed.entries, path);
212+
yield* verifyXmdArtifactSemantics(contents, path);
183213

184214
// Gate 9: the manifest this content produces, against the one stored beside
185215
// it, and the identity that manifest derives against the stored identity.
186-
const rebuilt = buildXmdArtifactManifest(entries, (kind) => {
216+
const rebuilt = buildXmdArtifactManifest(sealed.entries, (kind) => {
187217
throw new XmdArtifactInventoryError(
188218
path,
189219
`it holds more than one ${kind} record under one identity`,
190220
);
191221
});
192-
if (Buffer.compare(Buffer.from(rebuilt.bytes), Buffer.from(header.manifest)) !== 0) {
222+
if (Buffer.compare(Buffer.from(rebuilt.bytes), Buffer.from(sealed.manifest)) !== 0) {
193223
throw new XmdArtifactManifestMismatchError(path);
194224
}
195-
if (rebuilt.identity !== header.identity) {
225+
if (rebuilt.identity !== sealed.identity) {
196226
throw new XmdArtifactIdentityMismatchError(path);
197227
}
198228

‎packages/workflow/src/deno/artifact/records.ts‎

Lines changed: 143 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,17 @@
2727

2828
import { createHash } from "node:crypto";
2929
import { Buffer } from "node:buffer";
30+
import type { Operation } from "effection";
31+
import { prepareElicitation, validateParsed } from "@executablemd/core";
3032
import { parseDurableEvent } from "@executablemd/durable-streams";
31-
import type { Json } from "@executablemd/durable-streams";
33+
import type { DurableEvent, Json } from "@executablemd/durable-streams";
34+
import { SUSPENSION_ANSWER } from "../../suspension/answer.ts";
35+
import {
36+
parseSuspensionRequest,
37+
suspensionRequestFingerprint,
38+
type WorkflowSuspensionRequest,
39+
} from "../../suspension/api.ts";
40+
import { SUSPENSION_REQUEST } from "../../suspension/suspend.ts";
3241
import { readEventSource } from "../../lifecycle/history.ts";
3342
import type { InheritedEventProvenance } from "../../lifecycle/history.ts";
3443
import {
@@ -1193,7 +1202,10 @@ function inventoryFailure(path: string): Reject {
11931202
* to the commit the definition pins — so all of that is asked here, and it is
11941203
* asked before any of it is handed to a caller.
11951204
*/
1196-
export function verifyXmdArtifactSemantics(contents: XmdArtifactContents, path: string): void {
1205+
export function* verifyXmdArtifactSemantics(
1206+
contents: XmdArtifactContents,
1207+
path: string,
1208+
): Operation<void> {
11971209
const reject: Reject = inventoryFailure(path);
11981210

11991211
const eventIds = new Set(contents.journal.map((row) => row.eventId));
@@ -1214,7 +1226,7 @@ export function verifyXmdArtifactSemantics(contents: XmdArtifactContents, path:
12141226
const manifests = verifyContentStore(contents, reject);
12151227
verifyRoots(contents, manifests, path, reject);
12161228
verifyCheckouts(contents, reject);
1217-
verifySuspensions(contents, eventIds, reject);
1229+
yield* verifySuspensions(contents, path, reject);
12181230
verifyAgentSessions(contents, reject);
12191231
verifyDefinitionClosure(contents, reject);
12201232
}
@@ -1396,21 +1408,144 @@ function verifyCheckouts(contents: XmdArtifactContents, reject: Reject): void {
13961408
}
13971409
}
13981410

1399-
function verifySuspensions(
1411+
/**
1412+
* Every retained answer, authenticated against the history it claims to answer.
1413+
*
1414+
* A row naming an event the journal holds proves nothing on its own: what has
1415+
* to be true is that the named event *is* the wait, that it is *this* wait, that
1416+
* the request it retained is the one the value was judged against, that the
1417+
* value could still end that wait, and that the row's state agrees with whether
1418+
* the journal published an answer for it.
1419+
*
1420+
* Every one of those questions is asked with the live contracts rather than
1421+
* with an encoding of this module's own — the same `parseSuspensionRequest`,
1422+
* the same fingerprint, the same schema compilation a delivery and a resume
1423+
* use. A second spelling of any of them would be a second answer to a question
1424+
* a live run has already settled.
1425+
*/
1426+
function* verifySuspensions(
14001427
contents: XmdArtifactContents,
1401-
eventIds: ReadonlySet<string>,
1428+
path: string,
14021429
reject: Reject,
1403-
): void {
1430+
): Operation<void> {
1431+
const journal = contents.journal.map((row) => ({
1432+
eventId: row.eventId,
1433+
event: parseJournalEvent(row.record, path),
1434+
}));
1435+
const requests = new Map<string, { readonly at: number; readonly event: DurableEvent }>();
1436+
const publications = new Map<string, number>();
1437+
1438+
for (const [at, entry] of journal.entries()) {
1439+
const event = entry.event;
1440+
if (event.type !== "yield") {
1441+
continue;
1442+
}
1443+
if (event.description.type === SUSPENSION_REQUEST) {
1444+
requests.set(entry.eventId, { at, event });
1445+
}
1446+
if (event.description.type === SUSPENSION_ANSWER) {
1447+
const named = event.description.name;
1448+
if (typeof named !== "string" || publications.has(named)) {
1449+
reject("it publishes more than one answer for one wait");
1450+
}
1451+
publications.set(named, at);
1452+
}
1453+
}
1454+
14041455
const suspensions = new Set<string>();
14051456
for (const answer of contents.answers) {
14061457
if (suspensions.has(answer.suspensionId)) {
14071458
reject("it holds one suspension answer twice");
14081459
}
14091460
suspensions.add(answer.suspensionId);
1410-
if (!eventIds.has(answer.requestEventId)) {
1411-
reject("a suspension answer names a request event this artifact does not hold");
1461+
1462+
const request = requests.get(answer.requestEventId);
1463+
if (request === undefined) {
1464+
reject("a suspension answer names an event that is not a retained request for a wait");
1465+
}
1466+
const description = request.event.type === "yield" ? request.event.description : undefined;
1467+
if (description === undefined) {
1468+
reject("a suspension answer names an event that is not a retained request for a wait");
1469+
}
1470+
if (
1471+
description["name"] !== answer.suspensionId ||
1472+
description["suspensionId"] !== answer.suspensionId
1473+
) {
1474+
reject("a suspension answer names a request published for a different wait");
1475+
}
1476+
1477+
// Parsed rather than read. The request reached the journal through a public
1478+
// durable operation, so what the description holds is a claim about a
1479+
// request until this walks it — and a schema nothing could validate against
1480+
// must not be the schema an answer is judged by.
1481+
let wait: WorkflowSuspensionRequest;
1482+
try {
1483+
wait = parseSuspensionRequest({
1484+
request: description["request"],
1485+
responseSchema: description["responseSchema"],
1486+
});
1487+
} catch {
1488+
// The live parser's sentence describes a request a document offered; what
1489+
// this artifact holds is a retained one, and the value stays out of it.
1490+
reject("a retained suspension request does not describe a wait");
1491+
}
1492+
1493+
if (suspensionRequestFingerprint(wait) !== answer.requestFingerprint) {
1494+
reject("a suspension answer was delivered against a request this artifact does not hold");
1495+
}
1496+
yield* judgeRetainedAnswer(wait, answer.answer, reject);
1497+
1498+
const publishedAt = publications.get(answer.suspensionId);
1499+
if (answer.state === "consumed" && publishedAt === undefined) {
1500+
reject("a consumed suspension answer has no published answer in this artifact's history");
14121501
}
1502+
if (answer.state === "pending" && publishedAt !== undefined) {
1503+
reject("a pending suspension answer was already published in this artifact's history");
1504+
}
1505+
if (publishedAt !== undefined && publishedAt <= request.at) {
1506+
reject("a suspension answer is published before the request it answers");
1507+
}
1508+
}
1509+
1510+
for (const suspension of publications.keys()) {
1511+
if (!suspensions.has(suspension)) {
1512+
reject("it publishes an answer for a wait it retains no answer for");
1513+
}
1514+
}
1515+
}
1516+
1517+
/**
1518+
* The value, judged by the schema the wait itself retained.
1519+
*
1520+
* The same compilation a delivery and an `<Elicit>` use, so what this artifact
1521+
* accepts as an answer is exactly what a live run would have. The refusal says
1522+
* that it does not satisfy the schema and never which part of it failed: the
1523+
* issues quote the value, and a value is retained content.
1524+
*/
1525+
function* judgeRetainedAnswer(
1526+
wait: WorkflowSuspensionRequest,
1527+
answer: Json,
1528+
reject: Reject,
1529+
): Operation<void> {
1530+
let issues;
1531+
try {
1532+
const prepared = yield* prepareElicitation(wait.responseSchema, "artifact answer");
1533+
issues = validateParsed(prepared.validate, answer);
1534+
} catch {
1535+
reject("a retained response schema cannot judge the answer stored against it");
1536+
}
1537+
if (issues.length > 0) {
1538+
reject("a retained answer does not satisfy the response schema its wait retained");
1539+
}
1540+
}
1541+
1542+
/** One retained record, as the durable event it claims to be. */
1543+
function parseJournalEvent(record: string, path: string): DurableEvent {
1544+
const parsed = parseDurableEvent(record);
1545+
if (!parsed.ok) {
1546+
throw new XmdArtifactRecordError(path, "journal-record", "it is not a durable event");
14131547
}
1548+
return parsed.value;
14141549
}
14151550

14161551
/**

‎packages/workflow/src/deno/artifact/write.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,7 @@ function* seal(path: string, contents: DetachedXmdArtifact): Operation<XmdArtifa
124124
// is refused here, where there is still nothing on disk to remove.
125125
const entries = encodeXmdArtifactInventory(contents);
126126
const detached = decodeXmdArtifactInventory(entries, path);
127-
verifyXmdArtifactSemantics(detached, path);
127+
yield* verifyXmdArtifactSemantics(detached, path);
128128
const built = buildXmdArtifactManifest(entries, (kind) => {
129129
throw new XmdArtifactInventoryError(
130130
path,

‎packages/workflow/src/deno/delivery.ts‎

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,6 @@ import type { DatabaseSync } from "node:sqlite";
2525
import { exists } from "@effectionx/fs";
2626
import { Err, Ok, type Operation, type Result, scoped } from "effection";
2727
import {
28-
canonicalFingerprint,
2928
createSecretScanner,
3029
type Json,
3130
prepareElicitation,
@@ -35,7 +34,11 @@ import {
3534
} from "@executablemd/core";
3635
import { serializeDurableEvent } from "@executablemd/durable-streams";
3736
import type { DurableEvent } from "@executablemd/durable-streams";
38-
import { parseSuspensionRequest, type WorkflowSuspensionRequest } from "../suspension/api.ts";
37+
import {
38+
parseSuspensionRequest,
39+
suspensionRequestFingerprint,
40+
type WorkflowSuspensionRequest,
41+
} from "../suspension/api.ts";
3942
import { SUSPENSION_ANSWER } from "../suspension/answer.ts";
4043
import {
4144
type WorkflowAnswerDelivery,
@@ -239,17 +242,10 @@ function retainedWait(
239242
record,
240243
requestEventId: entry.eventId,
241244
request,
242-
fingerprint: requestFingerprint(request),
245+
fingerprint: suspensionRequestFingerprint(request),
243246
};
244247
}
245248

246-
function requestFingerprint(request: WorkflowSuspensionRequest): string {
247-
return canonicalFingerprint({
248-
request: request.request,
249-
responseSchema: request.responseSchema,
250-
});
251-
}
252-
253249
/**
254250
* The value, judged by the schema the wait retained.
255251
*

‎packages/workflow/src/deno/suspension.ts‎

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ import { canonicalFingerprint } from "@executablemd/core";
5555
import type { EffectDescription } from "@executablemd/durable-streams";
5656
import {
5757
parseSuspensionRequest,
58+
suspensionRequestFingerprint,
5859
WorkflowSuspension,
5960
type WorkflowSuspensionRequest,
6061
} from "../suspension/api.ts";
@@ -341,7 +342,7 @@ function answerProvider(
341342
"execution before this one could publish it.",
342343
);
343344
}
344-
if (retained.requestFingerprint !== fingerprintOf(authority.request)) {
345+
if (retained.requestFingerprint !== suspensionRequestFingerprint(authority.request)) {
345346
throw new WorkflowRequestError(
346347
`the answer retained for ${authority.suspensionId} was delivered against a ` +
347348
"different request, so it is not an answer to the wait this execution reached.",
@@ -376,10 +377,3 @@ function answerProvider(
376377
},
377378
};
378379
}
379-
380-
function fingerprintOf(request: WorkflowSuspensionRequest): string {
381-
return canonicalFingerprint({
382-
request: request.request,
383-
responseSchema: request.responseSchema,
384-
});
385-
}

0 commit comments

Comments
 (0)