Skip to content

📦 Compile every packaged asset from one canonical list - #798

Merged
taras merged 4 commits into
mainfrom
agent/canonical-compile-inputs
Sep 10, 2026
Merged

taras merged 4 commits into
mainfrom
agent/canonical-compile-inputs

Conversation

@taras

@taras taras commented Sep 10, 2026

Copy link
Copy Markdown
Owner

Why

Every published xmd binary through v0.12.0 cannot document its own components.
deno compile embeds what --include names and nothing else, and reports no
omission — so a missing asset costs nothing at compile time and fails at the
moment a person asks for it.

Three sites compiled the xmd entrypoint and each kept its own copy of the
include list, so the three disagreed:

site code-review-agent cli/src/documents six components.md
deno task build ✓ ✓ ✓
release.yml matrix ✓ ✓ ✗
verify:clean release phase ✓ ✗ ✗

The release's copy named no components.md, which is the shipped defect. The
purity phase's copy was narrower still, so verify:clean proved a binary that
is not the one a release publishes.

What changes

Before:

$ xmd syntax TempDir            # a released binary
the packaged component documentation is missing from this build
(looked in file:///.../deno-compile-xmd/packages/core/src/components/components.md)

After: the same command renders the component's metadata and its long-form
documentation, and nothing can publish a binary where it does not.

How it works

scripts/lib/compile.ts (the list) → compileArguments() → scripts/compile.ts → deno compile
                                                       ↘ verify:clean phases()

scripts/lib/compile.ts owns the entrypoint, the isolation flags and every
embedded asset, in three lists that differ in how they are maintained:
EMBEDDED_PACKAGES (a package the binary executes Markdown out of — a decision,
not a layout), PACKAGED_DOCUMENTS (each package's src/documents/, embedded
whole), and PACKAGED_DOCUMENTATION (each components.md, named individually
because embedding those directories whole would carry the package TypeScript
into the binary a second time).

scripts/compile.ts is the command a task or a workflow invokes. It supplies
only what genuinely differs per site — --output and an optional --target —
and refuses an argument it does not define, because a silently dropped
--target would compile the runner's own platform and upload it under another
platform's artifact name. verify:clean builds the same argv from the module
directly, since it spawns its phases inside a clone of HEAD.

The probe shuts down through Effection's yield* exit(1) rather than
Deno.exit(), which returns to the host without unwinding: the temporary
directory it works in belongs to the run's scope, and a process that vanished
mid-scope left one behind on every failing run — the runs somebody is about to
inspect. Effection's own documentation warns against Deno.exit() for exactly
this reason.

What must stay true

  • Every packaged asset that exists is embedded — enforced by
    scripts/lib/compile.ts being the only list, checked by
    scripts/tests/packaged-document.test.ts, which walks the repository for both
    discoverable kinds and compares in both directions.
  • A release compile stays offline and cache-pure — enforced by COMPILE_FLAGS,
    checked by publish-workflow-membership.test.ts and empirically by
    verify:clean.
  • No workflow compiles the xmd entrypoint on a deno compile line of its own
    — checked by publish-workflow-membership.test.ts. Other entrypoints (e.g.
    scripts/files-contract-probe.ts) keep the original per-invocation flag scan.
  • A binary that cannot document its components never becomes an attested subject
    — enforced by step order in release.yml, checked by
    release-targets.test.ts.

How to verify it

  • scripts/smoke-documentation.ts proves a compiled binary answers for its own
    documentation from a directory that is not the checkout, and fails if any
    components.md is dropped. Verified adversarially: compiled against the old
    release argv, it fails with the binary carries no packaged documentation.
  • The unknown-name case (xmd syntax TemdDir) is what makes the probe evidence
    rather than a coincidence: a binary missing its assets fails both lookups
    the same way, so a passing xmd syntax TempDir alone would not distinguish a
    working build from a broken one.
  • packaged-document.test.ts fails if a components.md is added and not
    listed. Verified adversarially by removing packages/web/src/components.md
    from the list — name every components.md that exists, and no other fails.
  • plan-component-compiled.test.ts is unchanged and still passes against a
    binary built through the new command.
  • Scope teardown on the failing path was measured, not assumed: the set of
    $TMPDIR/xmd-smoke-documentation-* directories is identical either side of a
    failing run, so the run created its directory and removed it.

Manually:

deno task build
deno run --allow-all --frozen scripts/smoke-documentation.ts          # exit 0

# and the defect this PR fixes, reproduced:
deno compile --node-modules-dir=none --cached-only --frozen \
  --exclude-unused-npm --allow-all \
  --include packages/code-review-agent --include packages/cli/src/documents \
  --output dist/xmd-broken packages/cli/src/compiled.ts
deno run --allow-all --frozen scripts/smoke-documentation.ts dist/xmd-broken   # exit 1

Scope

Included

  • One canonical argument builder and command wrapper; all three compile sites
    routed through it.
  • The release documentation smoke, run in release.yml (pre-attestation, Linux
    x64) and in ci.yml's smoke job (every PR).
  • specs/release-process-spec.md §10.

Intentionally unchanged

  • Per-target dependency preparation (deno task deps:target) and --target.
  • plan-component-compiled.test.ts, which remains the compiled xmd plan
    information-request evidence.
  • The v0.12.0 assets. The fix ships in a later version; this PR does not bump.

New abstractions

  • scripts/lib/compile.ts exists because three sites needed the same argv and
    drifted apart while each held a copy. Consumers: scripts/compile.ts,
    scripts/verify-clean.ts, and three test files.
  • scripts/smoke-documentation.ts exists because packaged documentation is the
    one thing a compiled binary can silently lack — metadata comes from the module
    graph and renders either way.

Risks and limitations

  • The probe runs the binary and lets its output through, following
    scripts/smoke-fetch.ts. That output is the evidence being checked, but it is
    verbose in a release log.
  • scripts/compile.ts is added to deps.ts ENTRYPOINTS, so preparation caches
    its graph. This is dependency state, which is why verify:clean is delivery
    evidence here. It passes: the chain holds: preparation installs, and nothing else does, with node_modules, the cache's dependency roots and deno.lock
    byte-identical after every build phase including the wider release compile.
  • scripts/smoke-fetch.ts carries the same await main(...) + Deno.exit(1)
    shape this PR corrects in smoke-documentation.ts. Left unchanged as outside
    this PR's purpose; worth a follow-up.

`deno compile` embeds what `--include` names and nothing else, and reports
no omission: the binary compiles, runs, and fails at the moment a person
asks it for the asset that was left out.

Three sites compiled the `xmd` entrypoint and each kept its own copy of
the list, so the three disagreed. `release.yml` named no `components.md`
at all, which is why every published binary through v0.12.0 lists every
component and answers `xmd syntax TempDir` with a missing-asset error.
`verify:clean`'s copy also omitted `packages/cli/src/documents`, so the
purity phase proved a binary narrower than the one a release publishes.

`scripts/lib/compile.ts` now owns the entrypoint, the isolation flags and
every embedded asset; `scripts/compile.ts` is the command a task or a
workflow invokes, supplying only the output path and an optional target.
`verify:clean` builds the same argv from the module, since it spawns its
phases inside a clone of HEAD.

`scripts/smoke-documentation.ts` asks a compiled binary for documentation
from a directory that is not the checkout. The unknown-name case is what
makes it evidence: a binary missing its assets fails both lookups the same
way, so a passing lookup alone would not tell the two apart. `release.yml`
runs it on the runnable Linux x64 member between the compile and the
attestation, so an undocumentable build never becomes an attested subject;
`ci.yml`'s smoke job runs the same script on every pull request.

`scripts/tests/packaged-document.test.ts` walks the repository for both
discoverable kinds of asset and holds the lists to what it finds in both
directions — a missing entry ships a binary without its asset, and a stale
one embeds nothing while reading as coverage.
`Deno.exit()` returns to the host without unwinding anything, which
Effection's own documentation warns against: the temporary directory the
probe works in belongs to the run's scope, so a failing run left one
behind — on exactly the runs somebody is about to inspect.

Every failure now reports the same diagnostic and terminates through
`yield* exit(1)`, so `useTempDirectory()` and every other scoped resource
tear down. `exit` escapes rather than returning, so each caller follows it
with a `return` for the type checker rather than the program.

The entry point is `main(...)` rather than `await main(...)`, which is the
form the other scripts use and what Code Rule 1 asks for.

Every assertion is unchanged, including the ordinary `TemdDir` refusal and
the missing-packaged-documentation diagnostic.
@github-actions

github-actions Bot commented Sep 10, 2026 •

Copy link
Copy Markdown

PR #798: 📦 Compile every packaged asset from one canonical list

35 files, +827 / -115

Scope

🔴 PR has 942 lines changed. Split into focused PRs.

🟡 942 lines changed. PRs under 400 receive more thorough review.

🟡 35 files changed. Are all changes related?

🟡 Changes span 19 directories.

🟡 Large PR with no linked issue.

🟡 PR mixes config and source changes.

Structural

🟡 2 console statements.

Slop

Oxlint slop signals:

  • no-console ×9: scripts/deps.ts, scripts/smoke-documentation.ts, scripts/compile.ts (+1)

Static Analysis

Oxlint: 14 diagnostics across 4 files (3 rules)
Density: 0.017 violations/added-line

no-console (9): scripts/deps.ts, scripts/smoke-documentation.ts, scripts/compile.ts (+1)
no-floating-promises (4): scripts/compile.ts, scripts/smoke-documentation.ts, scripts/verify-clean.ts (+1)
no-shadow (1): scripts/verify-clean.ts

Correctness

FILE: scripts/compile.ts, PATTERN: no-console, CONCERN: Excessive console logging in a command-line tool, QUESTION: Are these logs necessary for user feedback or debugging?
FILE: scripts/compile.ts, PATTERN: no-floating-promises, CONCERN: Unhandled promise in command-line tool, QUESTION: Are these promises intentionally unhandled?
FILE: scripts/smoke-documentation.ts, PATTERN: no-console, CONCERN: Excessive console logging in a test script, QUESTION: Are these logs necessary for test verification?
FILE: scripts/smoke-documentation.ts, PATTERN: no-floating-promises, CONCERN: Unhandled promise in test script, QUESTION: Are these promises intentionally unhandled?
FILE: scripts/verify-clean.ts, PATTERN: no-shadow, CONCERN: Variable shadowing in test script, QUESTION: Is the shadowed variable intentional?
FILE: scripts/lib/compile.ts, PATTERN: no-floating-promises, CONCERN: Unhandled promise in compile function, QUESTION: Are these promises intentionally unhandled?
FILE: scripts/tests/packaged-document.test.ts, PATTERN: no-console, CONCERN: Excessive console logging in test suite, QUESTION: Are these logs necessary for test verification?

`scripts/tests/packaged-document.test.ts` runs under Deno, Node and Bun,
and it had grown an import of `scripts/verify-clean.ts` — a module that
reaches for `Deno.env` and `Deno.execPath` at import time. Every assertion
passed under Node and the file then exited 1 on
`ReferenceError: Deno is not defined`, which is exactly why
`verify-clean.test.ts` is in the Node and Bun exclusion list already.

The claim it was making — that `verify:clean`'s release phase compiles the
release's own argv, whole — moves to `verify-clean.test.ts`, beside the
other phase assertions and inside the suite already scoped to Deno for
this reason. Nothing is dropped: the portable suite keeps the discovery
sweep and the two workflow sites, which need no Deno-only module.

Verified under all three runtimes this time rather than Deno alone.
Stamps every publishable manifest at 0.12.1 so merging this PR ships the
packaged-asset fix rather than leaving it for a later release. `xmd
--version` reads `packages/cli/deno.json`, and `release.yml`'s preflight
refuses a tag the manifests do not declare, so the two cannot diverge.

`bun.lock` records a version per workspace member and neither the bump
task nor `bun install` restamps it, so those ten entries are restamped
here. They were still at 0.11.0 — the 0.12.0 release left them behind.
`@executablemd/test-support` is private and stays at 0.0.0, and `acpx`
is an unrelated dependency that happens to be at 0.12.0; neither is
touched.
@taras
taras merged commit e751721 into main Sep 10, 2026
38 checks passed
@taras
taras deleted the agent/canonical-compile-inputs branch September 10, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant