Skip to content
This repository was archived by the owner on Jul 21, 2026. It is now read-only.
This repository was archived by the owner on Jul 21, 2026. It is now read-only.

Partial install with --frozen-lockfile: read+write node_modules/.pnpm/lock.yaml and skip unchanged snapshots #433

Description

@zkochan

Background

Pacquet's frozen-lockfile installer re-fetches and re-links every snapshot on every run, even when node_modules already has the right content. Upstream pnpm avoids this by writing a current lockfile at <project>/node_modules/.pnpm/lock.yaml after each install: it records what was actually materialized, and the next install diffs it against pnpm-lock.yaml (the wanted lockfile) to skip snapshots whose resolution + dependencies + optionalDependencies are unchanged and whose directory still exists on disk. This issue tracks both halves of that loop — they're two Roadmap (#299) checkboxes that only make sense together.

A correct partial-install path is the difference between a warm pacquet install --frozen-lockfile that finishes in well under a second and one that re-walks every snapshot in the graph. It is also load-bearing for the alot7 warm-install perf gap I've been investigating.

How upstream does it

References at pnpm v11 94240bc046:

  • Read. readCurrentLockfile(internalPnpmDir, ...) reads <internalPnpmDir>/lock.yaml. internalPnpmDir is <rootModulesDir>/.pnpm (installing/deps-restorer/src/index.ts:226), so the file lives at node_modules/.pnpm/lock.yaml regardless of where the wanted lockfile sits (workspace root vs. project root).

  • Diff at the snapshot level. lockfileToDepGraph decides per depPath whether to skip the fetch + import:

    const depIsPresent = !isDirectoryDep &&
      currentPackages[depPath] &&
      equals(currentPackages[depPath].dependencies, pkgSnapshot.dependencies)
    
    const depIntegrityIsUnchanged = isIntegrityEqual(pkgSnapshot.resolution, currentPackages[depPath]?.resolution)

    If both hold and the optional-deps map is empty on both sides, the existing directory at <virtualStoreDir>/<dir-in-virtual-store>/node_modules/<pkgName> is taken at face value: no fetch, no extract, no relink for that snapshot. A missing directory triggers a _broken_node_modules debug log and the snapshot is re-installed. Under GVS there's an additional .pnpm-needs-build marker check (lockfileToDepGraph.ts:246) — out of scope here unless GVS lands first (Add global virtual store support to pacquet install --frozen-lockfile #432).

  • Write. writeCurrentLockfile(virtualStoreDir, lockfile) is called from installing/deps-installer/src/install/index.ts:1597 at end-of-install. It rewrites <virtualStoreDir>/lock.yaml atomically (writeFileAtomic) with the filtered lockfile — i.e. the wanted lockfile narrowed to the importers and engine that were actually installed. Empty lockfiles are deleted rather than written.

  • Reporter signal. pnpm:context carries currentLockfileExists: boolean (installing/context/src/index.ts). The flag matters to the reporter; pacquet currently hard-codes it to false at crates/package-manager/src/install.rs:128-132, with a TODO that lines up with this work.

Today

Pacquet:

  • Never reads node_modules/.pnpm/lock.yaml. The current_lockfile_exists field is hard-coded false at crates/package-manager/src/install.rs:132, and the TODO at install.rs:126-128 already calls this work out.
  • Never writes that file. crates/lockfile/src/save_lockfile.rs:30 writes pnpm-lock.yaml only. There is no writeCurrentLockfile-equivalent.
  • Has a load-bearing TODO at crates/package-manager/src/install_package_by_snapshot.rs:118 (TODO: skip when already exists in store?) confirming the per-snapshot skip is not implemented. As a result, every snapshot goes through the full install path on a warm reinstall.
  • Has no concept of "current packages" to diff against — create_virtual_store.rs builds the virtual store unconditionally from the wanted lockfile.

Plan

A. Read the current lockfile

  • Add a read_current_lockfile(internal_pnpm_dir: &Path) -> io::Result<Option<Lockfile>> helper in crates/lockfile, mirroring readCurrentLockfile. internal_pnpm_dir is <root_modules_dir>/.pnpm. Returns Ok(None) on ENOENT. Use the same lockfile-version checks as the wanted-lockfile path.
  • Wire it into Install::run so current_lockfile_exists on pnpm:context reflects reality. Drop the TODO.

B. Per-snapshot skip

  • In crates/package-manager/src/install_package_by_snapshot.rs (today's "always install" path), accept a borrowed current_packages: Option<&HashMap<DepPath, PackageSnapshot>> and, for the current dep_path, evaluate three conditions before any fetch:

    • current.dependencies == wanted.dependencies (structural eq, both empty-equivalent).
    • current.optional_dependencies == wanted.optional_dependencies (or both empty).
    • isIntegrityEqual(current.resolution, wanted.resolution) — port the helper from lockfileToDepGraph.ts:366. It compares integrity field on tarball resolutions and handles directory-resolution edge cases.

    When all three hold and the destination dir under the virtual store exists, skip fetch + extract + symlink for that snapshot.

  • When the directory is missing despite the cache key matching, log the equivalent of pnpm's _broken_node_modules event (debug-level) and fall through to the full install path. Use a fresh log emit consistent with CODE_STYLE_GUIDE.md's reporter conventions.

  • Drop the TODO at install_package_by_snapshot.rs:118.

C. Write the current lockfile at end-of-install

  • Add a write_current_lockfile(virtual_store_dir: &Path, lockfile: &Lockfile) in crates/lockfile, mirroring writeCurrentLockfile. Writes to <virtual_store_dir>/lock.yaml atomically (write-temp + rename, matching write-file-atomic's behavior); deletes the file when the lockfile is empty.
  • Call it from Install::run after the install succeeds, before the pnpm:summary emit (matching upstream ordering at installing/deps-installer/src/install/index.ts:1593-1600). For now pass the wanted lockfile through unmodified — once workspace install (Add workspace support to pacquet install --frozen-lockfile #431) lands, narrow to the filtered lockfile (selected importers + engine filter). Capture the filtered-write follow-up here as a checkbox.
  • On a partial install where some snapshots were skipped, the current-lockfile write still records the full materialized graph (i.e. the wanted lockfile narrowed by importer/engine filters, not narrowed by what was newly fetched). This is the upstream behavior — result.currentLockfile at the write site is filteredLockfile, not "freshly installed only."

D. Reporter

  • Plumb the read result into ContextLog::current_lockfile_exists. Remove the hard-coded false and the TODO comment at crates/package-manager/src/install.rs:126-132.
  • Confirm pnpm:stats added count reflects only the newly-installed snapshots, not the skipped ones. Mirrors installing/deps-restorer/src/index.ts:397-400.

E. Tests

  • Cold install followed by an immediate warm reinstall: assert that the second install fetches zero tarballs (e.g. via the existing mock-registry hook-count or a counter on the registry client). Today this assertion would fail.
  • Warm reinstall after deleting one package directory under the virtual store: only the missing package gets fetched/extracted; _broken_node_modules-equivalent debug log is emitted.
  • Warm reinstall after editing pnpm-lock.yaml to bump one package's integrity: only that snapshot is re-fetched.
  • Warm reinstall after dropping a dependency from a project: the removed dep is not re-installed, surplus directory cleanup is not in scope (it's a separate prune story) — verify the rest of the install still skips cleanly.
  • node_modules/.pnpm/lock.yaml round-trips: read after first install equals what was written. Empty lockfile case: the file is deleted, not written.
  • pnpm:context.current_lockfile_exists is false on first install and true on the second.

Interactions / out of scope

  • GVS (Add global virtual store support to pacquet install --frozen-lockfile #432). When GVS is on, packages live at <storeDir>/links/..., the per-project hoist dir lives at <project>/node_modules/.pnpm/node_modules, and the current lockfile still lives at <project>/node_modules/.pnpm/lock.yaml. The skip logic gains a .pnpm-needs-build marker check (lockfileToDepGraph.ts:246-256). Track the GVS-specific bits there, not here — but design the per-snapshot skip so adding the marker check is a small extension, not a rewrite.
  • Workspace install (Add workspace support to pacquet install --frozen-lockfile #431). Once it lands, result.currentLockfile becomes the filtered lockfile (selected importers × engine filter). Until then, write the wanted lockfile unfiltered — there's only one importer to filter to.
  • Surplus / stale package cleanup. Removing packages that exist in node_modules/.pnpm but no longer appear in the wanted lockfile is upstream's pruneVirtualStore (installing/deps-installer/src/install/index.ts:362). Out of scope.
  • Lockfile compatibility versions. readCurrentLockfile uses ignoreIncompatible: false at the deps-restorer call site, so a version mismatch surfaces as an error. Match that behavior; pacquet already asserts v9 in install.rs:172.

Related


Written by an agent (Claude Code, claude-opus-4-7).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions