Skip to content

Commit ce3462c

Browse files
authored
🔎 Read a workflow run back without advancing it (#458)
* 🔎 Read a workflow run back without advancing it `xmd workflow status`, `list` and `history` answer from immutable lifecycle snapshots. The Deno provider opens each run's database read-only, on a connection outside the pool execution uses, so what a reading command may do is enforced by SQLite rather than promised: bytes, rows and file modes are the ones it found. One foreign, damaged or unparseable candidate fails the whole list — a shorter list would answer a question nobody asked. History is lossless. Every retained event keeps its exact id, its complete filtered protocol event and the Workspace root its row was already associated with. An authored durable operation now carries where it was written under the stable namespaced `executablemd.source-position` field, threaded from the scanner through component imports, code-block contexts, bound-command snapshots and every authored `createDurableOperation` call rather than parsed back out of a formatted name. Only `type` and `name` decide replay, so identity is unchanged. A present field that does not parse makes the entry unreadable, and the diagnostic repeats nothing the field held. The grammar is by action: only `start` reaches Git and a document, only `list` takes a status filter, only the three inspections produce JSON, and an option an action does not have is refused rather than ignored. `cancel` and `delete` are actions of the same grammar and report that no lifecycle provider answers them; their behavior, the executor lease and atomic transitions are the rest of #367. * 🔒 Refuse a run database found where another run's id would put it `list` recognized every candidate structurally and then trusted it. A healthy version-1 database copied or renamed to another valid `<64-hex>.sqlite` name is structurally perfect, so it came back as a second, entirely genuine-looking run — retained identity accepted from a location implying a different one. The check now belongs to the read-only snapshot every read passes rather than to the one caller that had an id to compare against: a run's file name is the hash of its id, so a database whose retained id does not name its file is a collision or a copy, and the whole list fails with that condition. Exact inspection keeps its own question — whether the run found is the run asked for — and both leave every candidate unchanged. WLI5b constructs the case: with the check removed it reports two runs. * 🧾 Hold a retained run id to the rule a requested one is held to `readRunRecord` checked that `workflow_run.run_id` was non-empty text; a request was checked against that plus the NUL rule. Two checks that happened to agree about most values are two checks, and the retained side was the weaker one — so storage could hold an id no caller could ever name and hand it back as though it addressed something. `parseRunId` is now the one rule, and each side keeps its own vocabulary for breaking it: a caller asked for something impossible, and storage holds a column that does not parse. It states the rule without naming a host, because the module it lives in is the provider-neutral one. WLI5c constructs the case the rule is about — a retained id whose stored bytes no reader sees, at that id's own hashed filename — and pins what actually refuses it. On this host that is the location check, not the id rule: SQLite keeps all fourteen bytes while every read stops at the NUL, so the reader compares `release` against a file named for the longer string. The parity is therefore a contract this commit settles, not something the regression proves. * 🆔 Let a run id spell itself, wildcard characters and all `status` and `history` refused a run id containing `*` or `?` as forbidden wildcard syntax. A run id is opaque and carries no meaning beyond equality and lifecycle addressing: the local caller may choose any storage-valid non-empty id, and hashing is what keeps it from becoming a path. So `xmd workflow start --id=release-*` created a run that no command could then ask about — the grammar read a character in an id as a pattern nothing ever evaluates. Every management action now hands its argument on exactly as written. Deletion's contract is unchanged and was never this check: it expands nothing, so `delete release-*` addresses the run called `release-*` and never a set of runs. WFI11 starts `release-*`, `release-?` and `release-1` and inspects each from a directory that is not a working tree; WLI9 does the same through the provider with storage and Git spies installed. Both prove one id reaches one run and matches neither neighbour. Restoring the refusal fails WFI11. * 🎯 Attribute a wildcard-named run's history to that exact run WLI9 proved `history release-*` returned three entries. Three runs of one fixture have three histories that look exactly alike, so a lookup redirected to a neighbour would have satisfied it — the assertion described the shape of an answer rather than whose answer it was. The fixture now stamps each run's id into the events it retains, so two runs differ by what they hold and not only by the ids SQLite assigned. WLI9 reads `release-*`'s rows straight out of its own file, outside the provider, and compares event ids, events and Workspace roots whole; then it shows those ids and events are neither neighbour's, and that each neighbour reads back as itself on the same terms. The roots are compared against the independent read rather than across runs, because three runs that never mutated a Workspace share the one empty root and so it attributes nothing. Redirecting `release-*`'s history to `release-?` now fails WLI9. Under the same mutation the previous assertion passed: the neighbour also holds three rows. --------- Co-authored-by: Taras Mankovski <[email protected]>
1 parent a1de026 commit ce3462c

43 files changed

Lines changed: 2675 additions & 128 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎architecture.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2028,7 +2028,8 @@ Status is measured against main.
20282028
| `xmd workflow start` / `xmd workflow resume` | starts or resumes a workflow run from the CLI, under the Deno entrypoints only | built on the #366 stack; #367 defines the required executor lease, unbuilt |
20292029
| implicit workflow Workspace | retains provider-neutral filesystem, repository and attachment state by run ID | document filesystem built on the #366 stack; repository, process and attachment capabilities unbuilt (#218) |
20302030
| Repository / Worktree / transactional Git effects | compose named checkouts and publish local mutations with their journal result | defined in `specs/workflow-workspace-spec.md`, unbuilt |
2031-
| workflow lifecycle inspection and control | reads status/list/history without advancing a run, enforces one executor, cancels through the exact owner and deletes retained state under lifecycle authority | defined for #367 in this architecture and `specs/workflow-workspace-spec.md`, unbuilt |
2031+
| workflow lifecycle inspection and control | reads status/list/history without advancing a run, enforces one executor, cancels through the exact owner and deletes retained state under lifecycle authority | immutable inspection built on the #367 stack; executor lease, cancellation and deletion defined here and in `specs/workflow-workspace-spec.md`, unbuilt |
2032+
| historical authored source | retains an authored durable operation's normalized `SourcePosition` beside its identity, and history parses it or refuses the entry | built on the #367 stack |
20322033
| history fork | creates a new run from one compatible checkpoint and retained Workspace root | defined in `specs/workflow-workspace-spec.md`, unbuilt (#368) |
20332034
| read-only workflow Agent / generated XMD | lets an Agent inspect a derived view and propose constrained executable changes | defined in `specs/workflow-workspace-spec.md`, unbuilt |
20342035
| Deno-local DOFS provider | owns one authoritative SQLite/DOFS connection per run path, captures arbitrary canonical retained roots, privately restores them, and atomically coordinates one Workspace mutation with its filtered Yield | built on the #365 stack; public document filesystem effects and the CLI lifecycle route to it on the #366 stack |

‎packages/cli/src/cli.ts‎

Lines changed: 19 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@
77
* xmd targets <document.md>
88
* xmd workflow start <document.md> [options]
99
* xmd workflow resume <run-id>
10+
* xmd workflow status|history <run-id> [--json]
11+
* xmd workflow list [--status=<status>] [--json]
1012
*
1113
* A document reference is a path, optionally followed by `#` and one target
1214
* selector naming a section of the document (spec §5.4). `workflow start` takes
@@ -103,6 +105,7 @@ import {
103105
workflowConfig,
104106
} from "./workflow.ts";
105107
import type { HostWorkflowInstaller, WorkflowHost, WorkflowStart } from "./workflow.ts";
108+
import { runWorkflowManagement } from "./workflow-management.ts";
106109
import { establishDefinition } from "./workflow-definition.ts";
107110
import type { EstablishedDefinition } from "./workflow-definition.ts";
108111
import { useWorkflowServiceDenial } from "@executablemd/workflow";
@@ -1282,13 +1285,14 @@ function* prepareWorkflowProps(
12821285
const stray = findPropsFlag(args);
12831286
if (config.action !== "start") {
12841287
if (stray) {
1288+
const action = config.action ?? "resume";
12851289
return {
12861290
args,
12871291
bindings: [],
12881292
workflow,
12891293
error:
1290-
`unrecognized option for xmd workflow ${config.action ?? "resume"}: ${stray} — a resume ` +
1291-
"runs the props its run retained",
1294+
`unrecognized option for xmd workflow ${action}: ${stray} — document properties ` +
1295+
"belong to the start that created the run, whose props it retained",
12921296
};
12931297
}
12941298
return { args, bindings: [], workflow };
@@ -1718,12 +1722,18 @@ function* dispatch(
17181722
yield* exit(1);
17191723
break;
17201724
}
1721-
const request = parseWorkflowRequest(config);
1722-
if (!request.ok) {
1723-
console.error(request.error.message);
1725+
const invocation = parseWorkflowRequest(config, evalFlags.rest);
1726+
if (!invocation.ok) {
1727+
console.error(invocation.error.message);
17241728
yield* exit(1);
17251729
break;
17261730
}
1731+
if (invocation.value.kind === "manage") {
1732+
const managed = yield* runWorkflowManagement(invocation.value.request, workflowHost);
1733+
yield* exit(managed.exitCode);
1734+
break;
1735+
}
1736+
const request = invocation.value.request;
17271737
const props = yield* resolveRunProps(propsPhase);
17281738
if (props.error) {
17291739
console.error(props.error);
@@ -1735,7 +1745,7 @@ function* dispatch(
17351745
? undefined
17361746
: { established: propsPhase.established, props: props.value ?? {} };
17371747
announceSecretDetection(config.secretDetection);
1738-
const outcome = yield* runWorkflow(request.value, start, workflowHost, (execution) =>
1748+
const outcome = yield* runWorkflow(request, start, workflowHost, (execution) =>
17391749
execution.around(
17401750
runScopedDocument(
17411751
{
@@ -1745,10 +1755,10 @@ function* dispatch(
17451755
// repository component fails to resolve rather than resolving
17461756
// to content the definition does not describe.
17471757
componentDir: [],
1748-
verbose: request.value.verbose,
1758+
verbose: request.verbose,
17491759
journal: undefined,
1750-
raw: request.value.raw,
1751-
secretDetection: request.value.secretDetection,
1760+
raw: request.raw,
1761+
secretDetection: request.secretDetection,
17521762
stream: execution.stream,
17531763
// A workflow owns its journal, so its process results are part of
17541764
// the run's retained history: a resumed procedure reads back what

‎packages/cli/src/deno-workflow.ts‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,11 @@ import { homedir } from "node:os";
1818
import { join } from "node:path";
1919
import type { Operation } from "effection";
2020
import { env as readEnv } from "@executablemd/runtime";
21-
import { useWorkflowRunStorage, withWorkflowWorkspace } from "@executablemd/workflow/deno";
21+
import {
22+
useWorkflowLifecycle,
23+
useWorkflowRunStorage,
24+
withWorkflowWorkspace,
25+
} from "@executablemd/workflow/deno";
2226
import type { WorkflowRunDatabase } from "@executablemd/workflow";
2327
import type { WorkflowHost } from "./workflow.ts";
2428

@@ -36,6 +40,9 @@ export function* useDenoWorkflowHost(): Operation<WorkflowHost> {
3640
useStorage(): Operation<void> {
3741
return useWorkflowRunStorage({ root });
3842
},
43+
useLifecycle(): Operation<void> {
44+
return useWorkflowLifecycle({ root });
45+
},
3946
attach<T>(database: WorkflowRunDatabase, operation: Operation<T>): Operation<T> {
4047
return withWorkflowWorkspace(database, operation);
4148
},
Lines changed: 275 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,275 @@
1+
/**
2+
* `xmd workflow status`, `list` and `history` — reading a run without running it.
3+
*
4+
* These commands answer from immutable lifecycle snapshots. Nothing here opens
5+
* a writable database, acquires a lease, replays, attaches a Workspace,
6+
* materializes a root, imports a document or contacts a provider: what a
7+
* reading command may do is read.
8+
*
9+
* ## Two projections of one answer
10+
*
11+
* `--json` is the contract. It writes the retained values structurally — every
12+
* event, its exact id, its associated Workspace root and its optional authored
13+
* source — so a caller parsing it sees what the run holds rather than a
14+
* rendering of it. Human output is a presentation over the same values: it may
15+
* shorten and it may leave a column out, and it can never remove an event from
16+
* the JSON contract or invent one.
17+
*
18+
* ## Where output goes
19+
*
20+
* The answer goes to standard output and diagnostics to standard error, so a
21+
* caller can pipe one without the other. An exit code describes the *request* —
22+
* whether this command could answer — and never the run's own outcome: a
23+
* completed run and a failed run both report successfully.
24+
*/
25+
26+
import { scoped } from "effection";
27+
import type { Operation } from "effection";
28+
import { WorkflowLifecycle, WorkflowLifecycleProviderError } from "@executablemd/workflow";
29+
import type {
30+
WorkflowHistoryEntry,
31+
WorkflowLifecycleSnapshot,
32+
WorkflowRunRecord,
33+
} from "@executablemd/workflow";
34+
import type { DurableEvent } from "@executablemd/durable-streams";
35+
import type { WorkflowHost, WorkflowManagementRequest, WorkflowOutcome } from "./workflow.ts";
36+
37+
/**
38+
* Run one management command.
39+
*
40+
* The lifecycle provider is installed for this scope alone, so nothing the
41+
* command reached outlives the answer.
42+
*/
43+
export function runWorkflowManagement(
44+
request: WorkflowManagementRequest,
45+
host: WorkflowHost,
46+
): Operation<WorkflowOutcome> {
47+
return scoped(function* () {
48+
yield* host.useLifecycle();
49+
50+
switch (request.action) {
51+
case "status": {
52+
const snapshot = yield* WorkflowLifecycle.operations.inspect(request.runId);
53+
if (!snapshot.ok) {
54+
return refuse(snapshot.error);
55+
}
56+
write(request.json ? json(snapshot.value) : renderStatus(snapshot.value));
57+
return { exitCode: 0 };
58+
}
59+
case "list": {
60+
const snapshots = yield* WorkflowLifecycle.operations.list();
61+
if (!snapshots.ok) {
62+
return refuse(snapshots.error);
63+
}
64+
const selected =
65+
request.status === undefined
66+
? snapshots.value
67+
: snapshots.value.filter((snapshot) => snapshot.record.status === request.status);
68+
write(request.json ? json(selected) : renderList(selected));
69+
return { exitCode: 0 };
70+
}
71+
case "history": {
72+
const entries = yield* WorkflowLifecycle.operations.history(request.runId);
73+
if (!entries.ok) {
74+
return refuse(entries.error);
75+
}
76+
write(request.json ? json(entries.value) : renderHistory(entries.value));
77+
return { exitCode: 0 };
78+
}
79+
case "cancel":
80+
case "delete": {
81+
// An operation this host installs no handler for is a refusal to
82+
// report, not a crash: the Api's fail-closed default says exactly which
83+
// operation has no provider, and that sentence is the answer.
84+
try {
85+
const answered =
86+
request.action === "cancel"
87+
? yield* WorkflowLifecycle.operations.cancel(request.runId)
88+
: yield* WorkflowLifecycle.operations.delete(request.runId);
89+
if (!answered.ok) {
90+
return refuse(answered.error);
91+
}
92+
} catch (error) {
93+
if (error instanceof WorkflowLifecycleProviderError) {
94+
return refuse(error);
95+
}
96+
throw error;
97+
}
98+
write(`workflow ${request.action}: ${request.runId}`);
99+
return { exitCode: 0 };
100+
}
101+
}
102+
});
103+
}
104+
105+
function refuse(error: Error): WorkflowOutcome {
106+
console.error(error.message);
107+
return { exitCode: 1 };
108+
}
109+
110+
function write(text: string): void {
111+
console.log(text);
112+
}
113+
114+
/** One JSON value and the newline that ends it. `console.log` supplies the newline. */
115+
function json(value: unknown): string {
116+
return JSON.stringify(value);
117+
}
118+
119+
function renderStatus(snapshot: WorkflowLifecycleSnapshot): string {
120+
const { record } = snapshot;
121+
const lines = [
122+
`run: ${record.runId}`,
123+
`status: ${record.status}`,
124+
`definition: ${describeDefinition(snapshot)}`,
125+
`base: ${record.base}`,
126+
`props: ${JSON.stringify(record.props)}`,
127+
`created: ${record.createdAt}`,
128+
`updated: ${record.updatedAt}`,
129+
];
130+
if (record.stopReason !== undefined) {
131+
lines.push(`stop reason: ${describeReason(record)}`);
132+
}
133+
if (snapshot.retrieval !== undefined) {
134+
lines.push(
135+
`retrieval: revision ${snapshot.retrieval.revision} at ${snapshot.retrieval.updatedAt}`,
136+
);
137+
}
138+
lines.push(`workspace: ${snapshot.currentWorkspaceRootId}`);
139+
lines.push(
140+
snapshot.journalFrontier === undefined
141+
? "journal frontier: none"
142+
: `journal frontier: ${snapshot.journalFrontier.eventId} (Workspace ${
143+
snapshot.journalFrontier.workspaceRootId
144+
})`,
145+
);
146+
lines.push(`executions: ${snapshot.executions.length}`);
147+
for (const execution of snapshot.executions) {
148+
const ended =
149+
execution.stoppedAt === undefined
150+
? "unfinished"
151+
: `${execution.stopStatus ?? "unknown"} at ${execution.stoppedAt}`;
152+
lines.push(` ${execution.executionId} started ${execution.startedAt}, ${ended}`);
153+
}
154+
return lines.join("\n");
155+
}
156+
157+
function renderList(snapshots: readonly WorkflowLifecycleSnapshot[]): string {
158+
if (snapshots.length === 0) {
159+
return "no workflow runs";
160+
}
161+
const rows = [
162+
["RUN", "STATUS", "UPDATED", "DEFINITION", "WORKSPACE"],
163+
...snapshots.map((snapshot) => [
164+
snapshot.record.runId,
165+
snapshot.record.status,
166+
snapshot.record.updatedAt,
167+
describeDefinition(snapshot),
168+
snapshot.currentWorkspaceRootId,
169+
]),
170+
];
171+
return table(rows);
172+
}
173+
174+
function renderHistory(entries: readonly WorkflowHistoryEntry[]): string {
175+
const rows: string[][] = [["EVENT", "OPERATION", "SOURCE", "RESULT", "WORKSPACE"]];
176+
let outcome: WorkflowHistoryEntry | undefined;
177+
178+
for (const entry of entries) {
179+
if (entry.event.type === "close" && entry.event.coroutineId === "root") {
180+
// The root's Close is the run's outcome rather than one more operation,
181+
// so it becomes the footer instead of an otherwise empty row.
182+
outcome = entry;
183+
continue;
184+
}
185+
rows.push([
186+
entry.eventId,
187+
describeOperation(entry.event),
188+
describeSource(entry),
189+
describeResult(entry.event),
190+
entry.workspaceRootId,
191+
]);
192+
}
193+
194+
const footer =
195+
outcome === undefined
196+
? "Outcome: no canonical document outcome was recorded."
197+
: `Outcome: ${describeResult(outcome.event)} at ${outcome.eventId}, Workspace ${
198+
outcome.workspaceRootId
199+
}`;
200+
201+
if (rows.length === 1) {
202+
return footer;
203+
}
204+
return `${table(rows)}\n\n${footer}`;
205+
}
206+
207+
/**
208+
* What one row did, without repeating the protocol's own word for it.
209+
*
210+
* A Yield is its durable operation. A child Close is that coroutine ending, and
211+
* stays associated with the coroutine it belongs to so nesting remains visible.
212+
*/
213+
function describeOperation(event: DurableEvent): string {
214+
if (event.type === "yield") {
215+
return event.description.type;
216+
}
217+
return `end of ${event.coroutineId}`;
218+
}
219+
220+
function describeSource(entry: WorkflowHistoryEntry): string {
221+
const { source } = entry;
222+
if (source === undefined) {
223+
return "-";
224+
}
225+
const at = `${source.line}:${source.column}`;
226+
return source.path === undefined ? at : `${source.path}:${at}`;
227+
}
228+
229+
function describeResult(event: DurableEvent): string {
230+
switch (event.result.status) {
231+
case "ok":
232+
return "completed";
233+
case "err":
234+
return "failed";
235+
case "cancelled":
236+
return "cancelled";
237+
}
238+
}
239+
240+
function describeDefinition(snapshot: WorkflowLifecycleSnapshot): string {
241+
const { definition } = snapshot.record;
242+
const target = definition.targetPath === undefined ? "" : `#${definition.targetPath}`;
243+
return `${definition.objectId} ${definition.rootDocumentPath}${target}`;
244+
}
245+
246+
function describeReason(record: WorkflowRunRecord): string {
247+
const reason = record.stopReason;
248+
if (reason === undefined) {
249+
return "none";
250+
}
251+
return reason.kind === "host" ? `host ${reason.code}` : `journal event ${reason.eventId}`;
252+
}
253+
254+
/**
255+
* Columns wide enough for what they hold.
256+
*
257+
* Deterministic for one answer and derived from it, so spacing describes this
258+
* output rather than becoming a format anything can depend on.
259+
*/
260+
function table(rows: readonly (readonly string[])[]): string {
261+
const widths: number[] = [];
262+
for (const row of rows) {
263+
row.forEach((cell, index) => {
264+
widths[index] = Math.max(widths[index] ?? 0, cell.length);
265+
});
266+
}
267+
return rows
268+
.map((row) =>
269+
row
270+
.map((cell, index) => (index === row.length - 1 ? cell : cell.padEnd(widths[index] ?? 0)))
271+
.join(" ")
272+
.trimEnd(),
273+
)
274+
.join("\n");
275+
}

0 commit comments

Comments
 (0)