Skip to content

Commit bac1368

Browse files
committed
🐛 Restore the - file reference and reap the launched child (#723)
Two corrections to the standard-input run, keeping every accepted behavior of 8225eac. `-` is the one filename configliere's option grammar leaves unwritable, and the reference grammar reaches it again: bare `xmd -` executes the file named `-` through the shorthand run profile, and `xmd run -#Section` executes that file's `Section`. Both leave supplied standard input unread. The document argument is read from the parser's own unconsumed remainder and admitted only when its reference path is exactly `-`, so a mistyped option is still an option and a run written with one refuses for want of a root. `xmd run -` and `xmd run -- -` remain the only spellings that read standard input, and options written on either side of every form still apply. The stdin-capable launcher now owns its child's exit. Teardown is registered before the operation can suspend, terminates the owned process group, escalates to SIGKILL when the group has not closed within the grace period, and returns only once the child's own `close` boundary has been reached — a signal that has been sent is not a process that is gone. Captured output, exit status, environment, the run deadline and write-then-EOF are unchanged. IE29 and IE30 now require a successful run and assert the exact file and selected-section markers while a distinct stdin marker stays absent; SI12 proves the teardown boundary through the stdin-enabled runCli path.
1 parent 8225eac commit bac1368

7 files changed

Lines changed: 299 additions & 96 deletions

File tree

‎architecture.md‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2764,6 +2764,13 @@ three places: a path, standard input, or one `--eval` value. Which one is fixed
27642764
grammar — decided from the command form the caller wrote, before anything is
27652765
read — so a run never discovers its source part-way through preparation.
27662766

2767+
The command form is what separates the three, because a path and standard input
2768+
can be written the same way. `-` is the one filename the option grammar leaves
2769+
unwritable, so it reaches the reference grammar through the parser's own
2770+
unconsumed tokens: bare `xmd -` names that file, `xmd run -#Section` selects a
2771+
section of it, and only an invocation that explicitly names `run` with a
2772+
document argument of exactly `-` means standard input instead.
2773+
27672774
Standard input is the one of the three the shared CLI cannot reach for itself. A
27682775
process's own stdin is host-specific state, so each runtime-named entrypoint
27692776
supplies the operation that reads it, as a value passed into the CLI beside the
@@ -3837,7 +3844,7 @@ Status is measured against main.
38373844
| `Expansion` / `getExpansion()` | describes the current logical element expansion | built on main |
38383845
| document targets | catalogs a root document's addressable static headings, resolves one selector to one exact target, and projects the document to it before expansion | built on the #412 stack |
38393846
| document-aware `xmd run … --help` | describes what one document declares and every target it addresses, each as a full document reference with the description its section states, by inspection alone | built on the #463 stack |
3840-
| standard-input root documents | `xmd run -` and `xmd run -- -` read the whole root document from standard input, once, to end of file, and run it through the ordinary run profile. Fixed grammar selects it — the explicit `run` command form plus a document argument that is exactly `-`, read from the parser's own unconsumed remainder so a `-` another option took as its value is not one — so the shorthand `xmd -`, a `-#Section` reference, another command's `-`, and `--eval -` all keep their existing meanings and read nothing. The reader is a value each runtime-named entrypoint supplies and the shared CLI never reaches a stdin global; what comes back is `retainedSource("<stdin>", source)`, adding no root-source variant, constructor, digest member or public API. The complete input is acquired before inspection, provider setup, the secret-detection announcement, journal creation, root admission and execution, inside the run's existing deadline; a failed read is one fixed sentence carrying no host error, input or path, and cancellation tears the reader down without becoming one | built on this stack |
3847+
| standard-input root documents | `xmd run -` and `xmd run -- -` read the whole root document from standard input, once, to end of file, and run it through the ordinary run profile. Fixed grammar selects it — the explicit `run` command form plus a document argument that is exactly `-`, read from the parser's own unconsumed remainder so a `-` another option took as its value is not one — and every other spelling keeps the meaning it had: the shorthand `xmd -` executes the file named `-`, `xmd run -#Section` executes that file's `Section`, another command's `-` is that command's, and `--eval -` keeps its refusal. `-` is the one filename the option grammar leaves unwritable, so the reference grammar reaches it and nothing else beginning with `-` is read as a document. The reader is a value each runtime-named entrypoint supplies and the shared CLI never reaches a stdin global; what comes back is `retainedSource("<stdin>", source)`, adding no root-source variant, constructor, digest member or public API. The complete input is acquired before inspection, provider setup, the secret-detection announcement, journal creation, root admission and execution, inside the run's existing deadline; a failed read is one fixed sentence carrying no host error, input or path, and cancellation tears the reader down without becoming one | built on this stack |
38413848
| targeted `xmd run` | reads a file argument as a document reference and executes the one exact target its selector resolved to, replacing the selector before execution rereads the file | built on the #412 stack |
38423849
| targeted workflow definition | the V1 workflow definition optionally carries the exact canonical document target, which takes part in definition identity and in compatible reuse | built on the #412 stack; the workflow CLI does not supply one yet |
38433850
| declared Markdown component | a trusted host declares exact first-party Markdown to one execution as immutable data on an `ExecutionInstallation`: name, origin, source, its SHA-256, the accepted forms, an optional statement of the props and return that must agree with the parsed source, and an optional private component closure. Admission parses the bytes and refuses a mismatched digest or schema, a non-canonical form, a name that is not a component name or is structural, a duplicate, a reserved-registration collision and a private name a registration also claims. Resolution places it in the protected tier with reserved registrations, above the workflow component bundle, repository files and every registered default. Live import and retained history are held to the declared origin, digest and bytes, private names resolve only while canonical core expands the declaring bytes' own body — by the authored occurrence rather than by the name, so an answer kept from a legitimate private import authorizes no later site, no alias, no copy of the definition, no invocation that is over and no later execution — including one that declares no Markdown at all — while a private name written anywhere else resolves to nothing before the bundle, the repository or a registration can answer for it — and `xmd syntax` and document validation describe the declared contract from the same declaration without describing the closure. Closure is per name: only the declared component and its private closure become canonical imports, and every other name in the execution stays the ordinary open import middleware may still answer | built on the #660 stack; no public component uses it yet (#660 PR 2) |

‎packages/cli/src/cli.ts‎

Lines changed: 52 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1586,27 +1586,47 @@ function namesRun(args: string[]): boolean {
15861586
return args[0] === "run";
15871587
}
15881588

1589+
interface DocumentArgument {
1590+
/** The reference the caller wrote, as they wrote it. */
1591+
reference: string;
1592+
/** argv with that token, and the separator protecting it, removed. */
1593+
rest: string[];
1594+
}
1595+
1596+
/** A reference's path half: everything before its first raw `#`. */
1597+
function referencePath(reference: string): string {
1598+
const fragment = reference.indexOf("#");
1599+
return fragment === -1 ? reference : reference.slice(0, fragment);
1600+
}
1601+
15891602
/**
1590-
* What is left of `xmd run`'s argv once the sentinel document argument is gone.
1603+
* The document argument a run named that the parser could not take.
1604+
*
1605+
* Configliere refuses every positional beginning with `-` and defines no
1606+
* end-of-options separator, so `-` is the one filename its option grammar
1607+
* leaves unwritable — and a reference selecting a section of that file is
1608+
* written the same way. Both are read out of the parser's own remainder, the
1609+
* tokens it did not consume, rather than out of raw argv, where a `-` another
1610+
* option took as its value (`--journal -`) looks identical.
1611+
*
1612+
* Nothing else beginning with `-` is one. A mistyped option stays an option the
1613+
* parser does not define, so a run written with one still refuses for want of a
1614+
* root rather than going looking for a file named after the flag.
15911615
*
1592-
* Present only when the caller wrote `xmd run -` or `xmd run -- -`. Configliere
1593-
* refuses any positional beginning with `-` and defines no end-of-options
1594-
* separator, so the sentinel is read out of the parser's own remainder — the
1595-
* tokens it did not consume — rather than out of raw argv, where `-` could be
1596-
* another option's value (`--journal -`). Removing it lets everything the
1597-
* caller wrote around it parse as it would around a path.
1616+
* Removing the token is what lets everything written around it — before it or
1617+
* after it — parse as it would around an ordinary path.
15981618
*/
1599-
function takeStandardInputArgument(args: string[], remainder: string[]): string[] | undefined {
1619+
function takeDocumentArgument(args: string[], remainder: string[]): DocumentArgument | undefined {
16001620
const separated = remainder[0] === "--";
1601-
const [sentinel] = separated ? remainder.slice(1, 2) : remainder;
1602-
if (sentinel !== STANDARD_INPUT_ARGUMENT) {
1621+
const [reference] = separated ? remainder.slice(1, 2) : remainder;
1622+
if (reference === undefined || referencePath(reference) !== STANDARD_INPUT_ARGUMENT) {
16031623
return undefined;
16041624
}
16051625
const at = args.length - remainder.length;
16061626
if (args[at] !== remainder[0]) {
16071627
return undefined;
16081628
}
1609-
return [...args.slice(0, at), ...remainder.slice(separated ? 2 : 1)];
1629+
return { reference, rest: [...args.slice(0, at), ...remainder.slice(separated ? 2 : 1)] };
16101630
}
16111631

16121632
/**
@@ -1658,21 +1678,29 @@ function* preparePropsPhase(
16581678
const separated = separateArgs(args);
16591679
const head = workflow ? separated.head : args;
16601680
const scanned = xmd.parse({ args: head });
1661-
// Only the command form the caller wrote can tell `xmd run -` from the
1662-
// shorthand `xmd -`, which resolves to the same parsed command.
1663-
const withoutSentinel = namesRun(args)
1664-
? takeStandardInputArgument(head, scanned.remainder.args ?? [])
1665-
: undefined;
1666-
const standardInput = withoutSentinel !== undefined;
1667-
const fixed = withoutSentinel ?? args;
1668-
const parsed = withoutSentinel ?? head;
1669-
const provisional = standardInput ? xmd.parse({ args: parsed }) : scanned;
1681+
// A run whose parse took no path may still have named one the parser refuses
1682+
// to read. Every other command keeps whatever `-` already means to it.
1683+
const scannedRun = scanned.ok && !scanned.value.config.help ? scanned.value.config : undefined;
1684+
const document =
1685+
!workflow && scannedRun?.name === "run" && scannedRun.config.path === undefined
1686+
? takeDocumentArgument(head, scanned.remainder.args ?? [])
1687+
: undefined;
1688+
// The sentinel is one exact argument on one command form. Only the form the
1689+
// caller wrote separates `xmd run -` from the shorthand `xmd -`, which
1690+
// resolves to the same parsed command and names a file called `-`.
1691+
const standardInput = document?.reference === STANDARD_INPUT_ARGUMENT && namesRun(args);
1692+
const fixed = document?.rest ?? args;
1693+
const parsed = document?.rest ?? head;
1694+
const provisional = document === undefined ? scanned : xmd.parse({ args: parsed });
16701695
// `program` short-circuits on `--version` and leaves no configuration
16711696
// behind, so there is nothing to inspect.
16721697
const selected = provisional.ok ? provisional.value.config : undefined;
16731698
const command = selected && !selected.help ? selected.name : undefined;
1674-
const documentPath =
1699+
const parsedPath =
16751700
selected && !selected.help && selected.name === "run" ? selected.config.path : undefined;
1701+
// The reference this run names, however it was written. Standard input is not
1702+
// one: it has an origin rather than a path.
1703+
const documentPath = standardInput ? parsedPath : (document?.reference ?? parsedPath);
16761704
const [supplied] = evalFlags.values;
16771705

16781706
if (selected && !selected.help && selected.name === "workflow") {
@@ -2109,8 +2137,9 @@ const RUN_SOURCE_HELP = [
21092137
"command that writes a complete document composes with a run:",
21102138
' xmd plan "prepare the release" | xmd run -',
21112139
"",
2112-
`Only that exact spelling reads it: a bare \`xmd -\` does not, and`,
2113-
`${EVAL_OPTION} takes its document as the value.`,
2140+
"Only that exact spelling reads it. A bare `xmd -` names a file called `-`,",
2141+
`\`xmd run -#Section\` selects a section of that file, and ${EVAL_OPTION} takes`,
2142+
"its document as the value.",
21142143
"",
21152144
"A path is a document reference, and everything after its first `#` selects",
21162145
"one section of the document to run:",

‎packages/cli/tests/inline-cli.test.ts‎

Lines changed: 27 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -297,33 +297,48 @@ describe(
297297
/**
298298
* The stdin sentinel is one exact spelling of one exact argument on one
299299
* exact command. These three rows supply real standard input to the
300-
* spellings that are not it, and prove none of them executes what was
301-
* piped in.
300+
* spellings that are not it, and prove each of them does what it has always
301+
* done with what was piped in left unread.
302302
*
303-
* What each of them does instead is unchanged: configliere refuses any
304-
* positional beginning with `-`, so `-` and `-#Section` reach no document
305-
* argument at all and the run refuses for want of a root. Only `xmd run -`
306-
* gives that token a meaning.
303+
* `-` is the one filename the option grammar leaves unwritable, so the
304+
* shorthand form names that file and a reference selects a section of it.
307305
*/
308-
it("IE29: the shorthand form does not read stdin", function* () {
309-
const root = yield* useWorkspace({ "-": "FILE_NAMED_DASH\n" });
306+
it("IE29: the shorthand form runs the file named `-`, not stdin", function* () {
307+
const root = yield* useWorkspace({ "-": "FILE_ROOT_MARKER\n" });
310308
const { code, stdout } = yield* runCli(["-", "--raw"], {
311309
cwd: root,
312310
stdin: "STDIN_SHOULD_NOT_RUN\n",
313311
}).join();
314312

315-
expect(code).toBe(1);
313+
expect(code).toBe(0);
314+
expect(stdout).toContain("FILE_ROOT_MARKER");
316315
expect(stdout).not.toContain("STDIN_SHOULD_NOT_RUN");
317316
});
318317

319-
it("IE30: a reference is not the sentinel argument", function* () {
320-
const root = yield* useWorkspace({ "-": "# Dash\n\n## Section\n\nSECTION_BODY\n" });
318+
it("IE30: a reference selects a section of that file, not stdin", function* () {
319+
const document = [
320+
"# Dash",
321+
"",
322+
"## Section",
323+
"",
324+
"SECTION_MARKER",
325+
"",
326+
"## Other",
327+
"",
328+
"OTHER_MARKER",
329+
"",
330+
].join("\n");
331+
const root = yield* useWorkspace({ "-": document });
321332
const { code, stdout } = yield* runCli(["run", "-#Section", "--raw"], {
322333
cwd: root,
323334
stdin: "STDIN_SHOULD_NOT_RUN\n",
324335
}).join();
325336

326-
expect(code).toBe(1);
337+
expect(code).toBe(0);
338+
expect(stdout).toContain("SECTION_MARKER");
339+
// The selector really selected: the sibling section is excluded, which a
340+
// run of the whole file would have rendered.
341+
expect(stdout).not.toContain("OTHER_MARKER");
327342
expect(stdout).not.toContain("STDIN_SHOULD_NOT_RUN");
328343
});
329344

‎packages/cli/tests/stdin-cli.test.ts‎

Lines changed: 95 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,10 @@ import {
2626
import type { Operation, Result, Task } from "effection";
2727
import { ensureDir, exists, readTextFile, rm, writeTextFile } from "@effectionx/fs";
2828
import { randomUUID } from "node:crypto";
29-
import { readdir } from "node:fs/promises";
29+
import { readFile, readdir } from "node:fs/promises";
3030
import * as os from "node:os";
3131
import * as path from "node:path";
32+
import process from "node:process";
3233
import { PassThrough } from "node:stream";
3334
import { API, Service, useHostFiles } from "@executablemd/runtime";
3435
import { runXmd } from "../src/cli.ts";
@@ -408,6 +409,52 @@ describe(
408409
expect(settled).toEqual(Ok("# One\né"));
409410
});
410411

412+
it("SI12: cancelling a stdin run returns only once its child is gone", function* () {
413+
yield* useFixture({}, function* (dir) {
414+
const idsPath = path.join(dir, "ids.txt");
415+
// A command that ignores SIGTERM and holds the run open behind it.
416+
// Measured: a group SIGTERM leaves this CLI unclosed indefinitely, so
417+
// teardown can only return by escalating and then waiting for the
418+
// child's own close boundary. `$$` is the command's shell and `$PPID`
419+
// is the CLI the launcher owns.
420+
const document = [
421+
"```bash exec",
422+
`trap "" TERM; echo "$$ $PPID" > ${idsPath}; sleep 20`,
423+
"```",
424+
"",
425+
].join("\n");
426+
427+
const run = yield* spawn(function* () {
428+
yield* runCli(["run", "-", "--raw"], {
429+
cwd: dir,
430+
stdin: document,
431+
timeout: 60_000,
432+
}).join();
433+
});
434+
435+
const [command, owned] = yield* until(waitForIds(idsPath));
436+
// `@effectionx/process` detaches an exec child into a group of its own,
437+
// so the command is the running CLI's to reap while it unwinds and
438+
// never the launcher's. One written to survive that unwinding survives
439+
// it; this suite still leaves nothing behind.
440+
yield* ensure(() => signalGroup(command));
441+
442+
const at = Date.now();
443+
yield* run.halt();
444+
const elapsed = Date.now() - at;
445+
446+
// A signal that has been sent is not a process that is gone: teardown
447+
// returning as soon as it signalled would come back at once. Nor may it
448+
// wait forever on a child that is never going to answer SIGTERM — the
449+
// command sleeps for twenty seconds, and escalation is what ends this
450+
// inside the grace period instead.
451+
expect(elapsed).toBeGreaterThanOrEqual(1_500);
452+
expect(elapsed).toBeLessThan(10_000);
453+
// And the group the launcher owns is gone by the time it returns.
454+
expect(yield* until(waitForGroupExit(owned))).toBe(true);
455+
});
456+
});
457+
411458
it("SI9b: the stream adapter's listeners belong to the read's own scope", function* () {
412459
const stream = new PassThrough();
413460
const read = yield* spawn(() => readInputStream(stream));
@@ -590,3 +637,50 @@ function* cancellableRun(
590637
const task = yield* spawn(() => scoped(() => observedRun(args, cwd, reader, state)));
591638
return { task, state };
592639
}
640+
641+
/** The command's own pid and the CLI's, once the run has reached the command. */
642+
async function waitForIds(idsPath: string): Promise<[number, number]> {
643+
const deadline = Date.now() + 30_000;
644+
while (Date.now() < deadline) {
645+
try {
646+
const [command, cli] = (await readFile(idsPath, "utf8"))
647+
.trim()
648+
.split(/\s+/)
649+
.map((value) => Number.parseInt(value, 10));
650+
if (Number.isInteger(command) && Number.isInteger(cli)) {
651+
return [command, cli];
652+
}
653+
} catch {
654+
// Not written yet.
655+
}
656+
await new Promise((resolve) => setTimeout(resolve, 50));
657+
}
658+
throw new Error(`the run never reached its command: ${idsPath} holds no pids`);
659+
}
660+
661+
/**
662+
* Whether that whole process group is gone.
663+
*
664+
* Bounded rather than instantaneous: the group dies together, so the kernel may
665+
* still be reaping a member as the launcher's own child closes.
666+
*/
667+
async function waitForGroupExit(leader: number): Promise<boolean> {
668+
const deadline = Date.now() + 3_000;
669+
while (Date.now() < deadline) {
670+
try {
671+
process.kill(-leader, 0);
672+
} catch {
673+
return true;
674+
}
675+
await new Promise((resolve) => setTimeout(resolve, 50));
676+
}
677+
return false;
678+
}
679+
680+
function signalGroup(leader: number): void {
681+
try {
682+
process.kill(-leader, "SIGKILL");
683+
} catch {
684+
// Already gone.
685+
}
686+
}

0 commit comments

Comments
 (0)