Repository navigation
Commit df89eb7
authored
refactor(nix): manage Nix-built JS tooling with bun2nix (#1489)
* ci(nix-cache): fall back to the newest prior store
`restore-prefixes-first-match` only carried the prefix that embeds the
deps hash, so any change to flake.nix, flake.lock or nix/packages.nix
sent the job to a completely cold Nix store even when the change merely
added developer tooling and left every Cargo artefact untouched.
Add a second, hashless prefix after it. The hashed prefix still wins
when it matches, and otherwise the newest prior entry warms the deps
instead of forcing a full rebuild.
* refactor(nix): build publint with bun2nix
nix/publint.nix carried a full pnpm-lock.yaml as a heredoc inside a Nix
string, fed to fetchPnpmDeps. Bumping publint meant hand-editing that
embedded lockfile and its fetcherVersion/hash pair, which is how the
lockfile drifted from the workspace catalogue in the first place.
Move the dependency set into nix/tools/publint as real files instead:
package.json, the bun.lock that `bun install` resolved from it, and the
bun.nix that `bun2nix` derived from that lockfile. The Nix expression
lives next to them as default.nix, so the tool is self-contained.
nix/bun-node-modules.nix is the shared piece: it turns one of those
manifests into a node_modules tree via bun2nix's setup hook, scoping src
to just package.json and bun.lock so neither the sibling Nix expression
nor a tool's own scripts force the dependency tree to rebuild.
Two wrinkles worth recording:
- bun2nix stages each dependency through its own derivation and patches
shebangs there against bun's `bun-with-fake-node` shim, so publint
arrives bound to Bun's Node emulation. dontPatchShebangs cannot undo
that, since the path is already baked into the cached package, so the
install phase repoints it at real Node. publint is a Node packaging
linter, and the baked-in path would otherwise keep bun in the runtime
closure, which is now just bash and nodejs.
- The hook only honours a wholesale replacement of its default install
flags, so the linker and backend are restated alongside
--frozen-lockfile.
The publint flake check now skips nix/tools/*/package.json: those are
dependency manifests for Nix-built tooling, not publishable packages.
* fix(treefmt): tolerate oxlint batches that are wholly ignored
treefmt matches files against a formatter's `includes` and passes them as
arguments, so a batch can consist entirely of paths that
nix/oxlint-check.json ignores. oxlint exits non-zero with "No files found
to lint" in that case, which treefmt reports as a formatting failure for a
file that was deliberately excluded.
Pass --no-error-on-unmatched-pattern, oxlint's own switch for exactly this
situation, mirroring the flag oxfmt is already given here.
* refactor(nix): resolve models-dev-gen deps with bun2nix
The generator's two runtime dependencies were vendored by hand: two
pkgs.fetchurl calls with npm integrity strings copied out of the pinned
models-dev input's bun.lock, extracted into a flat node_modules with tar.
Bumping the input meant transcribing hashes between two lockfiles.
Resolve them through bun2nix instead, from a package.json that pins the
same remeda 2.33.7 and zod 3.24.2 that input's lockfile records. The
generated bun.nix carries hashes identical to the fetchurl calls it
replaces, and both snapshots the build emits are unchanged.
Also move the tool into nix/tools/models-dev-gen so its Nix expression,
the scripts it copies into the workspace and its dependency manifests sit
together: models-dev-pricing.nix becomes default.nix, models-dev-gen.ts
becomes gen.ts, and models-dev-compact.ts drops its now-redundant prefix.
The remaining references follow: the node test invocation in the justfile
and in the git hook, the tsconfig exclude and the oxlint ignore pattern.
* build(nix): guard bun.nix drift and add gen-bun-nix
Each tool under nix/tools now carries a bun.lock and the bun.nix that
bun2nix derived from it. Nothing tied the two together: Renovate bumps
bun.lock without knowing bun.nix exists, and a stale pair only shows up
as a confusing "install wants the network" failure inside the sandbox.
Add a bun-nix flake check that re-derives bun.nix from each committed
bun.lock and diffs the result. bun2nix only parses the lockfile, so this
needs no network access and runs in the sandbox like the other checks.
`just gen-bun-nix` is the fix the check points at: it re-resolves every
tool and regenerates its bun.nix. Both bun and the bun2nix CLI join the
dev shell so that recipe can run.
* fix(oxlint): follow the models-dev-gen rename
23b7d24 moved nix/models-dev-gen.ts to
nix/tools/models-dev-gen/gen.ts and updated nix/oxlint-check.json, but
.oxlintrc.json carries its own copy of the same ignore list and was left
pointing at the old path.
`just typecheck` therefore started type-checking the generator, which
imports `./packages/core/src/generate.ts` from the upstream models.dev
worktree it is copied into at build time. That module does not exist in
this repository, so the file cannot type-check here by construction.
* refactor(nix): extract a bun CLI wrapper builder
nix/tools/publint/default.nix held the whole recipe for turning a
bun2nix-resolved node_modules tree into a wrapped CLI, including the
non-obvious part: bun2nix bakes `bun-with-fake-node` into the shebangs of
the dependencies it stages, so a Node CLI has to be repointed at real Node
or it runs under Bun's emulation and drags bun into the runtime closure.
That is exactly the kind of detail the next tool would silently omit, so
move it into nix/bun-cli.nix. The tool's own expression shrinks to the
package name and its metadata.
`runtimeInputs` is there for CLIs that shell out to other programs; publint
does not need it.
* refactor(release): build changelogithub with bun2nix
changelogithub was a root devDependency purely so the release workflow
could reach it through `nix develop --command pnpm changelogithub`: a full
dev shell plus a pnpm workspace install, to run one CLI that touches
nothing in this repository's source.
Move it to nix/tools/changelogithub and run it as `nix run
.#changelogithub`. It shells out to git to resolve the release tag, so git
is wired into the wrapper instead of being inherited from the runner.
This also retires the `changelogithub>c12` pnpm override. That existed
only to keep chokidar 4.0.3 out of pnpm-lock.yaml, which pnpm rejected
under `trustPolicy: no-downgrade` (see f7f7be5). bun resolves c12 3.3.4
and chokidar 5.0.0 on its own, so the constraint is satisfied by plain
resolution rather than by an override. Dropping the dependency removes 751
lines from pnpm-lock.yaml.
* build(deps): update nixpkgs for zizmor 1.27
zizmor 1.26.1 predates the parallel-step keys GitHub Actions shipped on
2026-06-25, so it failed schema validation on ci.yaml and audited nothing
there. That gap was invisible: treefmt hands the matched files to zizmor in
batches, and a batch containing any other workflow still exits zero, so the
warning about ci.yaml scrolled past while its findings went uncollected.
A batch consisting of ci.yaml alone exits non-zero instead, which is how it
finally surfaced.
zizmor 1.27.0 added experimental support for parallel steps, and nixpkgs now
carries it. ci.yaml audits clean with no findings, so this turns an
unaudited workflow into an audited one rather than merely quieting an error.
* ci(pkg-pr-new): fetch on demand with pnpm dlx
pkg-pr-new was a root devDependency, so every workspace install carried it
even though the preview-release CI job is the only thing that runs it and it
resolves nothing against the workspace.
Fetch it with `pnpm dlx` at the point of use instead. The version is pinned
in the workflow because Renovate does not update `dlx` specifiers, which is
noted next to the call so the pin does not quietly rot.
* refactor(types): move @types/node to the tool that needs it
The root package.json carried @types/node solely so the root tsconfig could
set `types: ["node"]`, and that tsconfig existed solely to cover
`nix/**/*.ts`. With the last of those files now living under
nix/tools/models-dev-gen, both belonged there rather than at the root: no
`*.ts` file sits at the repository root, and apps/ccusage and docs each
declare @types/node themselves.
So the tool directory gets its own tsconfig, standalone like the other two
in this repository, and pins @types/node next to the dependencies it already
pins. gen.ts stays excluded for the same reason the oxlint config ignores
it. The root package.json is left with no devDependencies at all.
`just typecheck` still resolves node types for compact.test.ts, now out of
nix/tools/models-dev-gen/node_modules, which `just gen-bun-nix` populates.
* docs(nix): fix stale paths in the tool manifests
Both nix/tools manifests described themselves in terms of the file layout
that existed before the move: models-dev-gen pointed at `nix/models-dev-gen.ts`
and publint at `nix/publint.nix`. Neither path exists any more; each tool's
Nix expression and scripts are siblings of the manifest now.
Reported on #1489 by pullfrog for the first of the two; the second is the
same mistake, found by grepping for the pre-rename paths.
* build(just): add an install recipe covering the nix/tools deps
CONTRIBUTING tells contributors to run `just typecheck`, and moving
@types/node into nix/tools/models-dev-gen broke that on a fresh checkout:
oxlint resolves that directory's types out of its own node_modules, which
nothing created. `pnpm install` does not cover it, since the tool
directories sit outside the pnpm workspace.
Consolidate setup into `just install`: the workspace install plus a
`bun install --frozen-lockfile` for every tool directory carrying a
tsconfig. Nix builds each tool's dependencies itself, so the ones with no
TypeScript in them need nothing locally and are skipped.
`git wt`'s hook now calls the recipe instead of running pnpm directly, so
new worktrees stay usable without a manual step, and CONTRIBUTING lists it
ahead of the checks it is a prerequisite for.1 parent 0d968b9 commit df89eb7
37 files changed
Lines changed: 1272 additions & 987 deletions
File tree
- .github
- actions/setup-nix-cache
- workflows
- nix
- tools
- changelogithub
- models-dev-gen
- publint
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
46 | | - | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
47 | 52 | | |
48 | 53 | | |
49 | 54 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
275 | 275 | | |
276 | 276 | | |
277 | 277 | | |
278 | | - | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
279 | 283 | | |
280 | 284 | | |
281 | 285 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
166 | 166 | | |
167 | 167 | | |
168 | 168 | | |
169 | | - | |
| 169 | + | |
170 | 170 | | |
171 | 171 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| 47 | + | |
47 | 48 | | |
48 | 49 | | |
49 | 50 | | |
50 | 51 | | |
51 | 52 | | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
52 | 56 | | |
53 | 57 | | |
54 | 58 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
6 | 10 | | |
7 | 11 | | |
8 | 12 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
22 | 35 | | |
23 | 36 | | |
24 | 37 | | |
| |||
32 | 45 | | |
33 | 46 | | |
34 | 47 | | |
35 | | - | |
| 48 | + | |
36 | 49 | | |
37 | 50 | | |
38 | 51 | | |
| |||
50 | 63 | | |
51 | 64 | | |
52 | 65 | | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
53 | 76 | | |
54 | 77 | | |
55 | 78 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
0 commit comments