feat(exec): check dependencies before running scripts by KSXGitHub · Pull Request #8645 · pnpm/pnpm

KSXGitHub · 2024-10-14T08:10:34Z

Resolves #8585

zkochan · 2024-11-05T16:21:24Z

Filtered install writes the full lockfile at the root of the project but a filtered lockfile at node_modules/.pnpm/lock.yaml. The hidden lockfile inside .pnpm mirrors what is inside node_modules. It only lists the installed packages.

KSXGitHub · 2024-11-05T16:26:00Z

Well, 9cf363b should skip check on filtered install.

zkochan · 2024-11-06T13:21:27Z

actually, it looks like even when filtered install is used, the importers field in the inner lockfile will contain all projects and all dependencies. So, if you check that the importers of the two lockfiles deeply equal, you can say that node_modules is up to date.

zkochan · 2024-11-07T01:14:12Z

pkg-manager/plugin-commands-installation/src/installDeps.ts

+  if (opts.allProjects && opts.workspaceDir) {
+    await updatePackagesList({
+      allProjects: opts.allProjects,
+      catalogs: opts.catalogs,
+      filtered: (opts.filter?.length ?? 0) > 0,
+      lastValidatedTimestamp: Date.now(),
+      workspaceDir: opts.workspaceDir,
+    })
+  }


how can you set lastValidatedTimestamp at this point if installation wasn't yet performed? This file should be written at the end of the function. Maybe even moved to @pnpm/core.

IIRC, we don't need lastValidatedTimestamp to to newer than any of the lockfiles, we only need it to be newer than the manifest files.

I think both. But in any case, at this point installation didn't finish and you cannot know in advance if it will succeed, right? They are validated only if installation succeeded.

I had hoped that using .modules.yaml to store packages list would have solved this problem, but now that it proves to not make sense, I wonder where should this code be moved to. Is there a place in the code where all successful install will execute?

Unfortunately, the fact that we have non-shared lockfile workspaces complicates things. We can't do it inside @pnpm/core. We should save the file somewhere in @pnpm/plugin-commands-installation after mutateModules finishes.

I think this should fix it: 6c1809e

I initially wanted to add it to recursive directly, but it has too many return statement, so I think wrapper is nicer.

zkochan · 2024-11-07T01:17:03Z

workspace/packages-list-cache/src/loadPackagesList.ts

There is already a state file at node_modules/.modules.yaml. Should there be a new file?

I have experimented with the idea of storing packages list in .modules.yaml. It work fine when the lockfile is shared, but when the workspace doesn't use a shared lockfile, it fails as there is no shared .modules.yaml either.

I have push the code of the experiment to a branched named tmp.save-packages-list-in-modules-file, its last commit is fa237d0.

I see. That is unfortunate. Let's give the file some more meaningful name maybe. Like .pnpm-workspace-state.json.

Let's give the file some more meaningful name maybe. Like .pnpm-workspace-state.json.

1be0b40

deps/status/src/checkLockfilesUpToDate.ts

KSXGitHub · 2024-11-08T15:29:07Z

actually, it looks like even when filtered install is used, the importers field in the inner lockfile will contain all projects and all dependencies. So, if you check that the importers of the two lockfiles deeply equal, you can say that node_modules is up to date.

0764be4 has reverted the feature that ignores filtered install.

pnpm/test/verifyDepsBeforeRun/multiProjectWorkspace.ts

zkochan · 2024-11-12T13:01:15Z

workspace/packages-list-cache/package.json

@@ -0,0 +1,49 @@
+{
+  "name": "@pnpm/workspace.packages-list-cache",


maybe call it

Suggested change

"name": "@pnpm/workspace.packages-list-cache",

"name": "@pnpm/workspace.state",

Copilot

Pull request overview

This PR implements a feature to check that dependencies are up-to-date before running scripts, addressing issue #8585. When enabled via the --config.verify-deps-before-run=true flag, pnpm will verify that lockfiles and dependencies match the manifest files before executing pnpm run or pnpm exec commands. This helps catch outdated dependencies early and ensures scripts run with the correct dependency versions.

Key changes:

Introduces two new packages: @pnpm/workspace.state for caching workspace package metadata, and @pnpm/deps.status for checking lockfile and dependency status
Modifies pnpm run and pnpm exec commands to optionally validate dependencies before execution
Refactors lockfile verification utilities to support the new checking functionality

Reviewed changes

Copilot reviewed 54 out of 55 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
workspace/state/*	New package that tracks and caches workspace package paths and catalog versions to optimize repeated checks
deps/status/*	New package that validates lockfiles, manifests, and installed dependencies are synchronized
lockfile/verification/src/linkedPackagesAreUpToDate.ts	Extracted function for checking if workspace-linked packages are current
lockfile/verification/src/getWorkspacePackagesByDirectory.ts	Extracted helper for mapping workspace packages by directory
lockfile/verification/src/allProjectsAreUpToDate.ts	Refactored to use extracted functions, improving modularity
exec/plugin-commands-script-runners/src/run.ts	Added dependency checking before script execution with environment variable to prevent nested checks
exec/plugin-commands-script-runners/src/exec.ts	Added dependency checking before exec command execution
exec/plugin-commands-script-runners/src/shouldRunCheck.ts	New utility determining when to skip dependency checks (lifecycle scripts, nested runs)
pkg-manager/plugin-commands-installation/src/installDeps.ts	Updates workspace state cache after recursive installations
pkg-manager/plugin-commands-installation/src/recursive.ts	Exports types needed by other modules
packages/constants/src/index.ts	Adds MANIFEST_BASE_NAMES constant for package.json variants
config/config/src/Config.ts	Adds verifyDepsBeforeRun configuration option
pnpm/test/verifyDepsBeforeRun/*	Comprehensive integration tests for single/multi-project workspaces, exec command, and edge cases
pnpm/test/utils/execPnpm.ts	Enhanced with cwd support, expectSuccess option for better test debugging
Various package.json and tsconfig.json	Dependency and reference updates for new packages

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

Comments suppressed due to low confidence (1)

lockfile/verification/src/allProjectsAreUpToDate.ts:54

Variable 'importer' cannot be of type null, but it is compared to an expression of type null.

    return importer != null &&

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-06T23:46:11Z

pnpm/test/utils/execPnpm.ts

+    const stderr = execResult.stderr?.toString().split('\n').map(line => `\t${line}`).join('\n')
+    let message = `Process exits with code ${execResult.status}`
+    if (stdout.trim()) message += `\nSTDOUT:\n${stdout}`
+    if (stderr.trim()) message += `\nSTDOUT:\n${stderr}`


The error message incorrectly labels stderr output as "STDOUT". This should be "STDERR" to accurately reflect what is being displayed.

Copilot · 2026-01-06T23:46:11Z

pnpm/test/verifyDepsBeforeRun/multiProjectWorkspace.ts

+  manifests.baz = {
+    name: 'bar',


The variable name 'baz' is used but the manifest has the name 'bar', which creates confusion. Either the variable should be named differently (e.g., 'bazManifest') or the manifest name property should be 'baz' to match the variable name and location.

Copilot · 2026-01-06T23:46:11Z

pnpm/test/verifyDepsBeforeRun/multiProjectWorkspace.ts

+  manifests.baz = {
+    name: 'bar',


The variable name 'baz' is used but the manifest has the name 'bar', which creates confusion. Either the variable should be named differently or the manifest name property should be 'baz' to match the variable name and location.

Copilot · 2026-01-06T23:46:12Z

pnpm/test/verifyDepsBeforeRun/exec.ts

+  manifests.baz = {
+    name: 'bar',
+    private: true,


The variable name 'baz' is used but the manifest has the name 'bar', which creates confusion. Either the variable should be named differently or the manifest name property should be 'baz' to match the variable name and location.

Copilot · 2026-01-06T23:46:12Z

workspace/state/package.json

+{
+  "name": "@pnpm/workspace.state",
+  "version": "0.0.0",
+  "description": "Track the list of actual paths of workspace packages in a cache",
+  "main": "lib/index.js",
+  "types": "lib/index.d.ts",
+  "engines": {
+    "node": ">=18.12"
+  },
+  "files": [
+    "lib",
+    "!*.map"
+  ],
+  "scripts": {
+    "test": "pnpm run compile && pnpm run _test",
+    "_test": "jest",
+    "lint": "eslint \"src/**/*.ts\" \"test/**/*.ts\"",
+    "prepublishOnly": "pnpm run compile",
+    "compile": "tsc --build && pnpm run lint --fix"
+  },
+  "repository": "https://github.com/pnpm/pnpm/blob/main/workspace/state",
+  "keywords": [
+    "pnpm10",
+    "pnpm",
+    "mtime"
+  ],
+  "license": "MIT",
+  "bugs": {
+    "url": "https://github.com/pnpm/pnpm/issues"
+  },
+  "homepage": "https://github.com/pnpm/pnpm/blob/main/workspace/state#readme",
+  "funding": "https://opencollective.com/pnpm",
+  "dependencies": {
+    "@pnpm/catalogs.types": "workspace:*",
+    "@pnpm/logger": "workspace:*",
+    "@pnpm/types": "workspace:*",
+    "ramda": "catalog:"
+  },
+  "devDependencies": {
+    "@pnpm/prepare": "workspace:*",
+    "@pnpm/workspace.state": "workspace:*"
+  },
+  "exports": {
+    ".": "./lib/index.js"
+  },
+  "jest": {
+    "preset": "@pnpm/jest-config"
+  }
+}


Missing 'funding' field in package.json. Other similar packages in this PR (like deps/status/package.json) include a 'funding' field pointing to the OpenCollective page. This should be added for consistency.

zkochan · 2026-01-06T23:48:14Z

I requested a review just to get a summary from copilot.

KSXGitHub added 30 commits October 1, 2024 04:38

refactor: break a long line into multiple lines

e1c55aa

chore(git): merge from main

123fc52

chore(git): merge from main

a489aeb

chore(git): merge from v10

178eac8

feat: cache that tracks workspace structures

69b7469

feat: handle hash collisions

5a9d07f

docs(changeset): packages-list-cache

4cc2e9e

feat(packages-list-cache): store mtime

1e65461

fix(packages-list-cache): JSON5 and YAML manifests

036690a

feat(packages-list-cache): add catalogs

f46e531

style: sort fields alphabetically

8209188

fix: actually fix it

47b4ee7

lint: fix

2f9a64f

lint: fix

64dbb97

test(packages-list-cache): test

82c96dd

feat(exec): check deps before run scripts

9590bdb

Resolves #8585

style: fix eslint

5ec1ef6

feat: use a single lastValidatedTimestamp

04168fa

refactor: rearrange

41a35ef

perf: don't do pointless comparisons

a66b48b

perf: optimize non-workspace

c7c08ac

perf: optimize sharedWorkspaceLockfile=false

c7d8598

perf: remove unnecessary fs reads

75c63ce

refactor: statManifestFile

3e4a725

perf: skip comparing manifest to lockfile by stats

72728ce

feat: add wantedLockfileDir to error message

9ec696a

refactor: shorten a function name

48efb88

refactor: rename a function

e5ec15f

docs: improve wordings

1862df0

feat: export linkedPackagesAreUpToDate

7e1a5f2

KSXGitHub requested a review from zkochan November 5, 2024 15:23

KSXGitHub added 5 commits November 5, 2024 22:38

refactor: remove unnecessary Promise.all

c425136

refactor: use virtualStoreDir from config

9d4a1cd

refactor: split opts into ctx and opts

85ba93a

test: fix

8c4e197

style: fix eslint

3337c1e

KSXGitHub added 2 commits November 6, 2024 00:55

test: fix windows

fad6941

feat(exec): add verifyDepsBeforeRun to exec

2da7c13

zkochan reviewed Nov 7, 2024

View reviewed changes

KSXGitHub added 2 commits November 8, 2024 06:53

refactor: sync stat

2a29cf8

feat: stop ignoring filtered install

0764be4

zkochan reviewed Nov 8, 2024

View reviewed changes

pnpm/test/verifyDepsBeforeRun/multiProjectWorkspace.ts Outdated Show resolved Hide resolved

KSXGitHub added 2 commits November 11, 2024 14:17

test: filtered install

1316817

refactor: rearrange imports

feb015a

zkochan reviewed Nov 12, 2024

View reviewed changes

KSXGitHub added 3 commits November 12, 2024 20:32

feat: rename "packages list" to "workspace state"

1be0b40

test: fix

92ae296

fix: workspace state on failed install

6c1809e

KSXGitHub requested a review from zkochan November 12, 2024 18:14

zkochan merged commit 19d5b51 into main Nov 15, 2024

zkochan deleted the ensure-lockfile-up-to-date-before-executing-scripts-8585 branch November 15, 2024 00:01

zkochan added this to the v10.0 milestone Nov 15, 2024

zkochan requested a review from Copilot January 6, 2026 23:42

Copilot started reviewing on behalf of zkochan January 6, 2026 23:43 View session

Copilot AI reviewed Jan 6, 2026

View reviewed changes

		@@ -0,0 +1,49 @@
		{
		"name": "@pnpm/workspace.packages-list-cache",

	"name": "@pnpm/workspace.packages-list-cache",
	"name": "@pnpm/workspace.state",

Uh oh!

Conversation

KSXGitHub commented Oct 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zkochan commented Nov 5, 2024

Uh oh!

KSXGitHub commented Nov 5, 2024

Uh oh!

zkochan commented Nov 6, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KSXGitHub Nov 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

KSXGitHub commented Nov 8, 2024

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

zkochan commented Jan 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

KSXGitHub commented Oct 14, 2024 •

edited

Loading

KSXGitHub Nov 12, 2024 •

edited

Loading