Skip to content

Commit 4093071

Browse files
authored
📥 Deliver a typed answer to a suspended workflow run (#479)
* 📥 Deliver a typed answer to a suspended workflow run A suspended run holds a wait: a retained request, a response schema, and no way to end it. This adds the operation that ends one. xmd workflow answer <run-id> <suspension-id> <json> Delivery is not execution. It takes no executor lock, fetches no definition, attaches no Workspace, records no document execution, appends no journal event and changes no status — so a run about to be resumed, or being resumed right now, can still be answered. A value is accepted only when the run is suspended, its stop reason names a retained suspension_request carrying that id, and the value satisfies the schema that request retained. Retained state and the answer event it would become cross the same secret gate durable journal persistence uses. Every refusal leaves the database unchanged, and neither the rejected value nor a secret match reaches a diagnostic. The answer becomes a durable operation of its own, directly behind the request, so replay tells an answered wait from an unanswered one. Its live path hands the publication to whoever keeps the run's delivery state: consuming that state and appending the suspension_answer Yield are one transaction, and the value returns only once that transaction has committed. A failure on either side leaves the answer pending and publishes nothing. Version 1 gains workflow_suspension_answers in place, on the same terms as every earlier amendment: no version 2, no migration, and strict recognition refusing the previous shape as an incomplete pre-release. Scheduling stays blocked on #301. * 🧪 Keep the answer-delivery suite out of Node and Bun The suite opens a real node:sqlite run database and takes the Deno lifecycle adapter's advisory lock, like every other Deno-adapter workflow suite. It was not listed in the exclusion manifest, so test-node ran it and failed eleven ways on the executor lock, and test-bun failed to resolve node:sqlite at all. Its tier is renamed WAD, because Tier WA already names the ACP server lifecycle suite in packages/test-agent. * 🧹 Drop two redundant template wrappers in answer delivery Each wrapped an already-string expression in a template of its own. * 📖 Register delivery in the Terminology table Delivery retains one externally supplied value for an exact durable suspension request without executing the run or advancing its lifecycle. It is not resume, not the answer publication that execution appends, and not the answer consumption that spends the retained value; each of those is registered beside it so the distinction has one place to live. --------- Co-authored-by: Taras Mankovski <[email protected]>
1 parent 4f2baa8 commit 4093071

28 files changed

Lines changed: 2671 additions & 94 deletions

‎architecture.md‎

Lines changed: 39 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,9 @@ Existing documents and code get aligned to this section retroactively.
4343
| blocker | execution needs input (auth, a human answer); not a failure |
4444
| suspension | a durable wait: a crash restarts into the same wait |
4545
| suspension ID | an opaque stable identifier derived by the trusted workflow execution for one logical suspension request; replay preserves it and only equality and input correlation are public |
46+
| delivery | retaining one externally supplied value for an exact durable suspension request, without executing the workflow run or advancing its lifecycle: it takes no executor lock, begins no document execution, appends no journal event and changes no run status. It is distinct from resume, which is the execution that later reaches the wait; from answer publication, which is what that execution appends; and from answer consumption, which is what spends the retained value |
47+
| answer publication | appending the durable answer event that records what ended one durable wait; performed by the resuming document execution, never by a delivery |
48+
| answer consumption | marking a retained delivery spent; it commits in the same transaction as the answer publication it belongs to, so neither can be observed without the other |
4649
| workflow executor | the live host invocation that advances or settles one workflow run |
4750
| executor lock | the scope-owned, non-blocking exclusive host lock that permits at most one workflow executor for a run |
4851
| stale execution | an unfinished retained document execution found after acquiring the executor lock proves its previous workflow executor is gone |
@@ -741,15 +744,34 @@ replayed effect. At the same suspension ID, absent input reports the same
741744
notice, halts again and adds no second request event. The new document-execution
742745
record may settle `suspended`; the request remains singular.
743746

744-
Issue #300 owns input delivery and scheduling. Its boundary is the suspension
745-
ID and retained response schema. A host may offer one value for that exact ID
746-
only after validating it against the retained schema. On a resumed execution,
747-
only authenticated retained answer state may let `suspendFor()` publish a
748-
separate ordinary durable answer Yield and return its value; a contextual
749-
handler's return value is never answer authority. Delivery remains pending until
750-
answer publication commits, so a crash cannot consume input without retaining
751-
the answer; duplicates, mismatched IDs and late delivery are refused. None of
752-
that transport or scheduling behavior is part of #367.
747+
### Typed answer delivery
748+
749+
Issue #300 builds typed answer delivery. Its boundary is the suspension ID and
750+
the retained response schema. Delivery is an operation separate from execution:
751+
`xmd workflow answer <run-id> <suspension-id> <json>` retains one value for that
752+
exact wait and takes no executor lock, fetches no definition, attaches no
753+
Workspace, inserts no document-execution record, appends no journal event and
754+
changes no run status. A host offers a value only after the run is `suspended`,
755+
its stop reason names a retained `suspension_request` event carrying that ID,
756+
and the value satisfies the schema that request retained. The retained state and
757+
the answer event it would become cross the same secret gate durable journal
758+
persistence uses; `--no-secret-detection` disables that gate for one delivery.
759+
Every refusal — duplicate, consumed, wrong run, wrong request, late, invalid,
760+
cancelled-run and deleted-run — leaves the database unchanged, and neither the
761+
rejected value nor a secret match appears in a diagnostic.
762+
763+
On a resumed execution, only authenticated retained answer state lets
764+
`suspendFor()` publish a separate ordinary durable `suspension_answer` Yield and
765+
return its value; a contextual handler's return value is never answer authority.
766+
Consuming the retained answer and appending that Yield are one database
767+
transaction, so a crash before it commits leaves the answer pending and a
768+
transaction that does not commit publishes nothing. Replay after that
769+
transaction commits restores the recorded answer event without reaching the live
770+
controller and without consuming or publishing again.
771+
772+
Scheduling — automatic resume, watchers, unattended iteration and remote host
773+
selection — remains blocked on #301's supervised-workflow contract and is not
774+
part of this behavior.
753775

754776
This contract folds issue #322's suspension effect into #367. Issue #322 no
755777
longer supplies a separate implementation prerequisite; its typed correlated
@@ -758,9 +780,10 @@ answer delivery remains with #300.
758780
Issue #367 changes no durable schema shape. Single-executor enforcement uses only the
759781
provider-owned advisory-lock sidecar, suspension uses existing filtered journal
760782
and run-state records, and history reads the existing event-to-root association.
761-
Complete schema version 1 therefore remains one exact shape. Any later durable
762-
input-delivery record for #300 requires its own schema-version decision and
763-
strict recognition amendment.
783+
Issue #300's retained answer state does add a durable record, and it amends
784+
complete schema version 1 in place on the same terms as every other amendment:
785+
one `workflow_suspension_answers` table, no version 2, no migration, and strict
786+
recognition refusing every earlier shape as an incomplete pre-release.
764787

765788
Issue #293 amended that exact shape in place rather than introducing a second
766789
version. This pre-release carries no persistence compatibility obligation, so
@@ -2116,7 +2139,10 @@ Status is measured against main.
21162139
| Deno-local DOFS provider | owns one authoritative SQLite/DOFS connection per run path, captures arbitrary canonical retained roots, privately restores them, and atomically coordinates one Workspace mutation with its filtered Yield | built on the #365 stack; public document filesystem effects and the CLI lifecycle route to it on the #366 stack |
21172140
| scoped Worker Shell | executes `just-bash` through the Workspace adapter inside a Deno Worker | containment and effect-transaction POCs complete (#351, #357); production integration unbuilt |
21182141
| `<Retry max timeout>` | retry a region until it completes | defined, unbuilt |
2119-
| `suspendFor()` | publishes one filtered suspension request, settles `suspended`, releases the executor lock, and re-enters the same wait without duplicating it | built on the #367 stack; typed answer delivery and scheduling belong to #300 |
2142+
| `suspendFor()` | publishes one filtered suspension request, settles `suspended`, releases the executor lock, and re-enters the same wait without duplicating it | built on the #367 stack |
2143+
| `xmd workflow answer <run-id> <suspension-id> <json>` | retains one schema-validated value for one retained wait, taking no executor lock and changing no run state | built by #300 |
2144+
| `suspension_answer` durable effect | ends a wait from retained delivery state, publishing the answer and consuming that state in one transaction | built by #300 |
2145+
| workflow scheduling (watchers, unattended iteration, remote host selection) | — | blocked on #301 |
21202146
| `<Result as>` | binds `{ok: true, value}` or `{ok: false, error}`; a failure becomes a bound value, not a raise | defined, unbuilt |
21212147
| error middleware (JS api) | retry · suspend · decline | defined, unbuilt |
21222148

‎packages/cli/src/cli.ts‎

Lines changed: 24 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1006,7 +1006,7 @@ interface PropsPhase {
10061006
* positional written after `--` is not in it, and asking the parser again
10071007
* would lose exactly the token the separator was there to protect.
10081008
*/
1009-
workflow?: { action?: string; target?: string };
1009+
workflow?: { action?: string; target?: string; suspension?: string; answer?: string };
10101010
root?: RootDocumentSource;
10111011
bindings: Binding[];
10121012
extraction?: Extraction;
@@ -1168,10 +1168,15 @@ function exactRoot(root: RootDocumentSource, target: string | undefined): RootDo
11681168
function* prepareWorkflowProps(
11691169
rawArgs: string[],
11701170
args: string[],
1171-
config: { action?: string; target?: string },
1171+
config: { action?: string; target?: string; suspension?: string; answer?: string },
11721172
inlineDocument: string | undefined,
11731173
): Operation<PropsPhase> {
1174-
const workflow = { action: config.action, target: config.target };
1174+
const workflow = {
1175+
action: config.action,
1176+
target: config.target,
1177+
suspension: config.suspension,
1178+
answer: config.answer,
1179+
};
11751180
if (inlineDocument !== undefined) {
11761181
return {
11771182
args: rawArgs,
@@ -1184,15 +1189,15 @@ function* prepareWorkflowProps(
11841189
// Read before anything is stripped: `--` ends option parsing, so a token
11851190
// after it is positional however it is spelled, and the count has to be taken
11861191
// while the separator is still there to say where options stopped.
1187-
const extra = extraWorkflowArgument(rawArgs);
1192+
const extra = extraWorkflowArgument(rawArgs, config.action);
11881193
if (extra !== undefined) {
11891194
return {
11901195
args: rawArgs,
11911196
bindings: [],
11921197
workflow,
11931198
error:
1194-
`unrecognized argument for xmd workflow: ${extra} — start names one definition and ` +
1195-
"resume names one run",
1199+
`unrecognized argument for xmd workflow: ${extra} — start names one definition, ` +
1200+
"resume names one run, and answer names a run, a wait and one JSON value",
11961201
};
11971202
}
11981203

@@ -1268,11 +1273,15 @@ function namesWorkflow(args: string[]): boolean {
12681273
* separated value, and `--` ends option parsing: every token after it is
12691274
* positional, including one that begins with `-`.
12701275
*/
1271-
function extraWorkflowArgument(args: string[]): string | undefined {
1276+
function extraWorkflowArgument(args: string[], action?: string): string | undefined {
12721277
const start = args.indexOf("workflow");
12731278
if (start === -1) {
12741279
return undefined;
12751280
}
1281+
// The action is itself the first positional. Every action but `answer` takes
1282+
// one more; `answer` takes three, because a delivery names the run, the wait
1283+
// inside it and the value in that order.
1284+
const allowed = action === "answer" ? 4 : 2;
12761285
let positionals = 0;
12771286
let skip = false;
12781287
let parsingOptions = true;
@@ -1294,7 +1303,7 @@ function extraWorkflowArgument(args: string[]): string | undefined {
12941303
continue;
12951304
}
12961305
positionals += 1;
1297-
if (positionals > 2) {
1306+
if (positionals > allowed) {
12981307
return arg;
12991308
}
13001309
}
@@ -1331,12 +1340,14 @@ function separateArgs(args: string[]): SeparatedArgs {
13311340
* separator it is positional because of where it is.
13321341
*/
13331342
function workflowPositionals(
1334-
config: { action?: string; target?: string },
1343+
config: { action?: string; target?: string; suspension?: string; answer?: string },
13351344
tail: string[],
1336-
): { action?: string; target?: string } {
1337-
const named = [config.action, config.target].filter((value) => value !== undefined);
1338-
const [action, target] = [...named, ...tail];
1339-
return { action, target };
1345+
): { action?: string; target?: string; suspension?: string; answer?: string } {
1346+
const named = [config.action, config.target, config.suspension, config.answer].filter(
1347+
(value) => value !== undefined,
1348+
);
1349+
const [action, target, suspension, answer] = [...named, ...tail];
1350+
return { action, target, suspension, answer };
13401351
}
13411352

13421353
const COMMAND_NAMES = ["run", "test", "test-agent", "workflow"];

‎packages/cli/src/deno-workflow.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ import { join } from "node:path";
1919
import type { Operation } from "effection";
2020
import { env as readEnv } from "@executablemd/runtime";
2121
import {
22+
useWorkflowInputDelivery,
2223
useWorkflowLifecycle,
2324
useWorkflowRunHost,
2425
withWorkflowWorkspace,
@@ -44,6 +45,9 @@ export function* useDenoWorkflowHost(): Operation<WorkflowHost> {
4445
useLifecycle(): Operation<void> {
4546
return useWorkflowLifecycle({ root });
4647
},
48+
useDelivery(): Operation<void> {
49+
return useWorkflowInputDelivery({ root });
50+
},
4751
attach<T>(database: WorkflowRunDatabase, operation: Operation<T>): Operation<T> {
4852
return withWorkflowWorkspace(database, operation);
4953
},

‎packages/cli/src/workflow-management.ts‎

Lines changed: 28 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,18 @@
11
/**
2-
* `xmd workflow status`, `list` and `history` — reading a run without running it.
2+
* `xmd workflow status`, `list`, `history`, `cancel`, `delete` and `answer` —
3+
* everything a run's caller does without running it.
34
*
4-
* These commands answer from immutable lifecycle snapshots. Nothing here opens
5-
* a writable database, acquires the executor lock, replays, attaches a Workspace,
5+
* The readings answer from immutable lifecycle snapshots. Nothing there opens a
6+
* writable database, acquires the executor lock, replays, attaches a Workspace,
67
* materializes a root, imports a document or contacts a provider: what a
78
* reading command may do is read.
89
*
10+
* `answer` writes, and writes exactly one thing: the value a suspended run's
11+
* wait is to continue from. It takes no executor lock either, so a run that is
12+
* about to be resumed can still be answered — and it starts nothing, so its
13+
* success line reports the delivery rather than a status the run did not
14+
* change.
15+
*
916
* ## Two projections of one answer
1017
*
1118
* `--json` is the contract. It writes the retained values structurally — every
@@ -25,7 +32,7 @@
2532

2633
import { scoped } from "effection";
2734
import type { Operation } from "effection";
28-
import { WorkflowLifecycle } from "@executablemd/workflow";
35+
import { WorkflowInputDelivery, WorkflowLifecycle } from "@executablemd/workflow";
2936
import type {
3037
WorkflowHistoryEntry,
3138
WorkflowLifecycleSnapshot,
@@ -45,6 +52,23 @@ export function runWorkflowManagement(
4552
host: WorkflowHost,
4653
): Operation<WorkflowOutcome> {
4754
return scoped(function* () {
55+
if (request.action === "answer") {
56+
yield* host.useDelivery();
57+
const delivered = yield* WorkflowInputDelivery.operations.deliver({
58+
runId: request.runId,
59+
suspensionId: request.suspensionId,
60+
value: request.value,
61+
secretDetection: request.secretDetection,
62+
});
63+
if (!delivered.ok) {
64+
return refuse(delivered.error);
65+
}
66+
// The delivery, and nothing about the run. Its status is unchanged, no
67+
// execution began, and nothing here claims otherwise.
68+
write(`workflow answer: ${delivered.value.runId} (${delivered.value.suspensionId})`);
69+
return { exitCode: 0 };
70+
}
71+
4872
yield* host.useLifecycle();
4973

5074
switch (request.action) {

0 commit comments

Comments
 (0)