Repository navigation
📦 Compile every packaged asset from one canonical list - #798
Conversation
`deno compile` embeds what `--include` names and nothing else, and reports no omission: the binary compiles, runs, and fails at the moment a person asks it for the asset that was left out. Three sites compiled the `xmd` entrypoint and each kept its own copy of the list, so the three disagreed. `release.yml` named no `components.md` at all, which is why every published binary through v0.12.0 lists every component and answers `xmd syntax TempDir` with a missing-asset error. `verify:clean`'s copy also omitted `packages/cli/src/documents`, so the purity phase proved a binary narrower than the one a release publishes. `scripts/lib/compile.ts` now owns the entrypoint, the isolation flags and every embedded asset; `scripts/compile.ts` is the command a task or a workflow invokes, supplying only the output path and an optional target. `verify:clean` builds the same argv from the module, since it spawns its phases inside a clone of HEAD. `scripts/smoke-documentation.ts` asks a compiled binary for documentation from a directory that is not the checkout. The unknown-name case is what makes it evidence: a binary missing its assets fails both lookups the same way, so a passing lookup alone would not tell the two apart. `release.yml` runs it on the runnable Linux x64 member between the compile and the attestation, so an undocumentable build never becomes an attested subject; `ci.yml`'s smoke job runs the same script on every pull request. `scripts/tests/packaged-document.test.ts` walks the repository for both discoverable kinds of asset and holds the lists to what it finds in both directions — a missing entry ships a binary without its asset, and a stale one embeds nothing while reading as coverage.
`Deno.exit()` returns to the host without unwinding anything, which Effection's own documentation warns against: the temporary directory the probe works in belongs to the run's scope, so a failing run left one behind — on exactly the runs somebody is about to inspect. Every failure now reports the same diagnostic and terminates through `yield* exit(1)`, so `useTempDirectory()` and every other scoped resource tear down. `exit` escapes rather than returning, so each caller follows it with a `return` for the type checker rather than the program. The entry point is `main(...)` rather than `await main(...)`, which is the form the other scripts use and what Code Rule 1 asks for. Every assertion is unchanged, including the ordinary `TemdDir` refusal and the missing-packaged-documentation diagnostic.
PR #798: 📦 Compile every packaged asset from one canonical list35 files, +827 / -115 Scope🔴 PR has 942 lines changed. Split into focused PRs. 🟡 942 lines changed. PRs under 400 receive more thorough review. 🟡 35 files changed. Are all changes related? 🟡 Changes span 19 directories. 🟡 Large PR with no linked issue. 🟡 PR mixes config and source changes. Structural🟡 2 console statements. SlopOxlint slop signals:
Static AnalysisOxlint: 14 diagnostics across 4 files (3 rules) no-console (9): scripts/deps.ts, scripts/smoke-documentation.ts, scripts/compile.ts (+1) CorrectnessFILE: scripts/compile.ts, PATTERN: no-console, CONCERN: Excessive console logging in a command-line tool, QUESTION: Are these logs necessary for user feedback or debugging? |
`scripts/tests/packaged-document.test.ts` runs under Deno, Node and Bun, and it had grown an import of `scripts/verify-clean.ts` — a module that reaches for `Deno.env` and `Deno.execPath` at import time. Every assertion passed under Node and the file then exited 1 on `ReferenceError: Deno is not defined`, which is exactly why `verify-clean.test.ts` is in the Node and Bun exclusion list already. The claim it was making — that `verify:clean`'s release phase compiles the release's own argv, whole — moves to `verify-clean.test.ts`, beside the other phase assertions and inside the suite already scoped to Deno for this reason. Nothing is dropped: the portable suite keeps the discovery sweep and the two workflow sites, which need no Deno-only module. Verified under all three runtimes this time rather than Deno alone.
Stamps every publishable manifest at 0.12.1 so merging this PR ships the packaged-asset fix rather than leaving it for a later release. `xmd --version` reads `packages/cli/deno.json`, and `release.yml`'s preflight refuses a tag the manifests do not declare, so the two cannot diverge. `bun.lock` records a version per workspace member and neither the bump task nor `bun install` restamps it, so those ten entries are restamped here. They were still at 0.11.0 — the 0.12.0 release left them behind. `@executablemd/test-support` is private and stays at 0.0.0, and `acpx` is an unrelated dependency that happens to be at 0.12.0; neither is touched.
Why
Every published
xmdbinary through v0.12.0 cannot document its own components.deno compileembeds what--includenames and nothing else, and reports noomission — so a missing asset costs nothing at compile time and fails at the
moment a person asks for it.
Three sites compiled the
xmdentrypoint and each kept its own copy of theinclude list, so the three disagreed:
code-review-agentcli/src/documentscomponents.mddeno task buildrelease.ymlmatrixverify:cleanrelease phaseThe release's copy named no
components.md, which is the shipped defect. Thepurity phase's copy was narrower still, so
verify:cleanproved a binary thatis not the one a release publishes.
What changes
Before:
After: the same command renders the component's metadata and its long-form
documentation, and nothing can publish a binary where it does not.
How it works
scripts/lib/compile.tsowns the entrypoint, the isolation flags and everyembedded asset, in three lists that differ in how they are maintained:
EMBEDDED_PACKAGES(a package the binary executes Markdown out of — a decision,not a layout),
PACKAGED_DOCUMENTS(each package'ssrc/documents/, embeddedwhole), and
PACKAGED_DOCUMENTATION(eachcomponents.md, named individuallybecause embedding those directories whole would carry the package TypeScript
into the binary a second time).
scripts/compile.tsis the command a task or a workflow invokes. It suppliesonly what genuinely differs per site —
--outputand an optional--target—and refuses an argument it does not define, because a silently dropped
--targetwould compile the runner's own platform and upload it under anotherplatform's artifact name.
verify:cleanbuilds the same argv from the moduledirectly, since it spawns its phases inside a clone of
HEAD.The probe shuts down through Effection's
yield* exit(1)rather thanDeno.exit(), which returns to the host without unwinding: the temporarydirectory it works in belongs to the run's scope, and a process that vanished
mid-scope left one behind on every failing run — the runs somebody is about to
inspect. Effection's own documentation warns against
Deno.exit()for exactlythis reason.
What must stay true
scripts/lib/compile.tsbeing the only list, checked byscripts/tests/packaged-document.test.ts, which walks the repository for bothdiscoverable kinds and compares in both directions.
COMPILE_FLAGS,checked by
publish-workflow-membership.test.tsand empirically byverify:clean.xmdentrypoint on adeno compileline of its own— checked by
publish-workflow-membership.test.ts. Other entrypoints (e.g.scripts/files-contract-probe.ts) keep the original per-invocation flag scan.— enforced by step order in
release.yml, checked byrelease-targets.test.ts.How to verify it
scripts/smoke-documentation.tsproves a compiled binary answers for its owndocumentation from a directory that is not the checkout, and fails if any
components.mdis dropped. Verified adversarially: compiled against the oldrelease argv, it fails with
the binary carries no packaged documentation.xmd syntax TemdDir) is what makes the probe evidencerather than a coincidence: a binary missing its assets fails both lookups
the same way, so a passing
xmd syntax TempDiralone would not distinguish aworking build from a broken one.
packaged-document.test.tsfails if acomponents.mdis added and notlisted. Verified adversarially by removing
packages/web/src/components.mdfrom the list —
name every components.md that exists, and no otherfails.plan-component-compiled.test.tsis unchanged and still passes against abinary built through the new command.
$TMPDIR/xmd-smoke-documentation-*directories is identical either side of afailing run, so the run created its directory and removed it.
Manually:
Scope
Included
routed through it.
release.yml(pre-attestation, Linuxx64) and in
ci.yml's smoke job (every PR).specs/release-process-spec.md§10.Intentionally unchanged
deno task deps:target) and--target.plan-component-compiled.test.ts, which remains the compiledxmd planinformation-request evidence.
New abstractions
scripts/lib/compile.tsexists because three sites needed the same argv anddrifted apart while each held a copy. Consumers:
scripts/compile.ts,scripts/verify-clean.ts, and three test files.scripts/smoke-documentation.tsexists because packaged documentation is theone thing a compiled binary can silently lack — metadata comes from the module
graph and renders either way.
Risks and limitations
scripts/smoke-fetch.ts. That output is the evidence being checked, but it isverbose in a release log.
scripts/compile.tsis added todeps.tsENTRYPOINTS, so preparation cachesits graph. This is dependency state, which is why
verify:cleanis deliveryevidence here. It passes:
the chain holds: preparation installs, and nothing else does, withnode_modules, the cache's dependency roots anddeno.lockbyte-identical after every build phase including the wider release compile.
scripts/smoke-fetch.tscarries the sameawait main(...)+Deno.exit(1)shape this PR corrects in
smoke-documentation.ts. Left unchanged as outsidethis PR's purpose; worth a follow-up.