fix(pack): include assets matched by publish.include in the tarball#35331
Conversation
ReviewSolid, well-scoped fix. The approach correctly mirrors Strengths
Suggestions (non-blocking)
Correctness / compile
Approve with minor nits. None are blocking; the symlink-drop and raw- |
|
Thanks for the review. Pushed a follow-up (7f9f1e1) addressing the nits:
On the two behaviors you flagged for confirmation:
Left the double FS traversal as-is — noted as a future consolidation, not a correctness issue. |
7f9f1e1 to
6c7e1db
Compare
|
Thanks for the fix — the approach is right: walking the filesystem with the publish 1. (medium) The root-level 2. (low) The test doesn't lock in the no-duplicate guarantee. It only asserts 3. (low) Exclusion is exact path-equality. De-dup relies on 4. (low, confirm) Parity when Nothing blocking beyond #1; #2 is cheap insurance. Nice work. |
|
Pushed a follow-up addressing the feedback: #1 — Removed the root-level #2 — The spec test now also asserts #4 — Confirming this is intended: with no |
|
CI was failing on Fix (44f9e41): restore the root-level |
`deno pack` only walked the module graph and kept JS/TS modules, so non-source assets listed in `publish.include` (e.g. `assets/icon.svg`) were never added to the generated `.tgz`. Mirror `deno publish`: after collecting and transpiling graph modules, additionally walk the filesystem with the package's publish `FilePatterns` (via `deno_config::glob::FileCollector`) to gather asset files. Source modules (transpiled separately), the source `deno.json`/`deno.jsonc`, and the auto-included README/LICENSE are excluded to avoid duplicates, and the `--ignore`/`--exclude` CLI flags are applied so they affect assets too. Root-level `.tgz` files are treated as pack output artifacts and never bundled, preserving reproducibility across repeated runs. Assets are appended via the existing reproducible tar helper and sorted for deterministic output. Adds a spec test (`pack/publish_include_assets`) reproducing the issue. Closes denoland#35295
… assets Address review feedback on asset collection: - Warn (mirroring deno publish's diagnostic) when a symlink matched by the package's publish config is skipped, instead of silently dropping it. - Exclude the resolved --output tarball path so an output file written inside an included dir (e.g. pack --output ./assets/foo.tgz) isn't re-bundled on a later run; clarify that the root-level .tgz heuristic only covers the default-named case.
…istic Replace the blanket root-level *.tgz drop in collect_asset_files with an exclusion of the exact default output filename. The default name is fully knowable (scope-name-version.tgz), so a user who legitimately lists a root-level tarball in publish.include no longer has it silently dropped. Extract the default-filename computation into npm_tarball::default_tarball_filename so the name is derived in one place and reused by both the tarball writer and the asset exclusion. Also strengthen the spec test to assert the de-dup guarantee: the transpiled src/mod.js is present while the raw src/mod.ts source is not.
…included The reproducible-tarball spec test packs twice in the same directory (`--output a.tgz` then `--output b.tgz`); the second run collected the `a.tgz` written by the first as an asset and embedded it, so the two tarballs were no longer byte-identical. Restore the root-level `.tgz` artifact drop (previously removed in favor of exact-output-name exclusion), but gate it on the package not having an explicit `publish.include`. When include is unset (the default of "everything"), a root-level `.tgz` is almost certainly a pack/npm-pack artifact and is dropped; when the package explicitly opts a tarball in via `publish.include`, it is shipped verbatim. The exact tarball being written is still always excluded.
44f9e41 to
d99e8a4
Compare
bartlomieju
left a comment
There was a problem hiding this comment.
LGTM. The approach correctly mirrors deno publish — walking the publish FilePatterns with the same FileCollector chain, then subtracting source modules (by original path), the config file, README/LICENSE, and the output tarball. Verified the de-dup, Windows path normalization (to_tar_path), self-exclusion against initial_cwd(), and the path-traversal guard in default_tarball_filename all hold up. The root-level .tgz gate (drop only when publish.include is absent) is the right resolution to the reproducibility regression, and the test locks in both the fix and the no-duplicate guarantee. Nice work.
Summary
deno packdid not include non-source assets listed inpublish.include(e.g.assets/icon.svg) in the generated.tgz, even when explicitly listed.The root cause:
cli/tools/pack/mod.rsonly collected files by walking the module graph (graph.modules()→Module::Js). Non-source assets are never part of the module graph, so they were silently dropped regardless ofpublish.include. By contrast,deno publishcollects files by walking the filesystem withFileCollectorand the package's publishFilePatterns(which honorpublish.include/exclude).This change makes
deno packadditionally collect non-module files matched by the package's publish config patterns (viamember_dir.to_publish_config()?.files, mirroringdeno publish), read them verbatim, and append them to the npm tarball alongside the transpiled modules. Files already represented as graph modules (which get transpiled) are excluded so the original source isn't duplicated, and the existing hard-excludes (.git,node_modules,.env*),.gitignore, README/LICENSE auto-inclusion, and reproducible tar entries are all preserved. Asset entries are sorted for deterministic output.Behavior
Given the repro from the issue, the tarball now contains:
Tests
Added a spec test under
tests/specs/pack/reproducing the issue from #35295 and asserting thatpackage/assets/icon.svgis present in the generated tarball.Closes #35295.