Skip to content

Commit 2aa1c7c

Browse files
authored
fix(arborist): validate peerOptional conflicts in no-save mutations (npm#9605)
## Problem Arborist currently treats `save=false` as a signal that invalid `peerOptional` edges from the lockfile can be trusted and skipped. That is correct for non-mutating lockfile reads such as `npm ci`, but it is wrong for commands that still mutate the ideal tree and write a new lockfile while leaving `package.json` unchanged. For example, `npm update` runs with `save=false`, but it can still update package versions in `package-lock.json`. If one of those updates introduces or exposes an invalid optional peer relationship, Arborist can finish successfully and write a lockfile that a later plain `npm install` rejects with `ERESOLVE`. ## Fix This separates two concepts that were previously conflated: - `#requestedTreeMutation`: whether the command explicitly requested an add, remove, or update operation. - `#mutateTree`: whether the ideal-tree build has actually changed the tree during placement. Invalid `peerOptional` policy now uses `#requestedTreeMutation` instead of `#mutateTree`: - `save=false` + explicit add/rm/update: invalid `peerOptional` edges are treated as resolver problems. - `save=false` + no requested mutation: invalid `peerOptional` edges remain trusted, preserving the existing `npm ci`-style behavior. The requeue path also uses `#requestedTreeMutation`, so a `save=false` update can reprocess an already-seen node when a later placement invalidates one of its `peerOptional` edges. ## Behavior Changed This fixes bad lockfile output for `save=false` commands that mutate the dependency graph, including: - `npm update` - `npm install <pkg> --no-save` - other explicit add/rm/update Arborist builds with `save=false` Those commands should not silently produce an ideal tree or lockfile containing invalid `peerOptional` edges that a subsequent install rejects. ## Behavior Preserved Non-mutating `save=false` builds continue to trust the lockfile. In those cases, an invalid `peerOptional` edge is still ignored rather than forcing re-resolution, preserving the behavior needed by `npm ci`-style lockfile reads. Fixes npm#9604. ## Tests Added regression coverage for: - `npm update`-style `save=false` mutation rejecting an invalid `peerOptional` graph instead of writing a bad lockfile. - `npm install <pkg> --no-save`-style mutation rejecting the same class of invalid graph. - `save=false` update requeueing an already-seen node when later placement invalidates its `peerOptional` edge. - non-mutating `save=false` builds continuing to ignore invalid `peerOptional` problem edges. Validated with: ```sh git diff --check npm exec -- tap # from workspaces/arborist npm run lint --workspace=@npmcli/arborist ``` Co-authored-by: Dale Lakes <[email protected]>
1 parent 690bf17 commit 2aa1c7c

2 files changed

Lines changed: 231 additions & 21 deletions

File tree

‎workspaces/arborist/lib/arborist/build-ideal-tree.js‎

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,7 @@ module.exports = cls => class IdealTreeBuilder extends cls {
103103
#mutateTree = false
104104
#packageExtensions = null
105105
#npmExtension = null
106+
#requestedTreeMutation = false
106107
// a map of each module in a peer set to the thing that depended on
107108
// that set of peers in the first place. Use a WeakMap so that we
108109
// don't hold onto references for nodes that are garbage collected.
@@ -401,12 +402,13 @@ module.exports = cls => class IdealTreeBuilder extends cls {
401402

402403
// set if we add anything, but also set here if we know we'll make
403404
// changes and thus have to maybe prune later.
404-
this.#mutateTree = !!(
405+
this.#requestedTreeMutation = !!(
405406
options.add ||
406407
options.rm ||
407408
update.all ||
408409
update.names.length
409410
)
411+
this.#mutateTree = this.#requestedTreeMutation
410412
}
411413

412414
// load the initial tree, either the virtualTree from a shrinkwrap,
@@ -484,7 +486,9 @@ module.exports = cls => class IdealTreeBuilder extends cls {
484486
filter: node => node,
485487
visit: node => {
486488
for (const edge of node.edgesOut.values()) {
487-
const skipPeerOptional = edge.type === 'peerOptional' && this.options.save === false
489+
const skipPeerOptional = edge.type === 'peerOptional' &&
490+
this.options.save === false &&
491+
!this.#requestedTreeMutation
488492
if (!skipPeerOptional && (!edge.to || !edge.valid)) {
489493
this.#depsQueue.push(node)
490494
break // no need to continue the loop after the first hit
@@ -1176,9 +1180,16 @@ This is a one-time fix-up, please be patient...
11761180
}
11771181
const { from, valid, peerConflicted } = edgeIn
11781182
if (!peerConflicted && !valid) {
1179-
if (this.#depsSeen.has(from) && this.options.save) {
1180-
// Re-queue already-processed nodes when a newly placed dep creates an invalid edge during npm install (save=true).
1181-
// This handles the case where a peerOptional dep was valid (missing) when the node was first processed, but becomes invalid when the dep is later placed by another path with a version that doesn't satisfy the peer spec.
1183+
if (this.#depsSeen.has(from) &&
1184+
(this.options.save ||
1185+
(this.options.save === false && this.#requestedTreeMutation))) {
1186+
// Re-queue already-processed nodes when a newly placed dep
1187+
// creates an invalid edge during npm install or another
1188+
// lockfile-mutating operation. This handles the case where a
1189+
// peerOptional dep was valid (missing) when the node was
1190+
// first processed, but becomes invalid when the dep is later
1191+
// placed by another path with a version that doesn't satisfy
1192+
// the peer spec.
11821193
// See npm/cli#8726.
11831194
this.#depsSeen.delete(from)
11841195
this.#depsQueue.push(from)
@@ -1384,11 +1395,16 @@ This is a one-time fix-up, please be patient...
13841395
continue
13851396
}
13861397

1387-
// If the edge has an error, there's a problem, unless it's peerOptional and we're not saving (e.g. npm ci), in which case we trust the lockfile and skip re-resolution.
1388-
// When saving (npm install), peerOptional invalid edges ARE treated as problems so the lockfile gets fixed.
1398+
// If the edge has an error, there's a problem, unless it's peerOptional
1399+
// and we're not saving or otherwise mutating (e.g. npm ci), in which
1400+
// case we trust the lockfile and skip re-resolution. When saving (npm
1401+
// install) or updating, peerOptional invalid edges ARE treated as
1402+
// problems so the lockfile gets fixed.
13891403
// See npm/cli#8726.
13901404
if (!edge.valid) {
1391-
if (edge.type !== 'peerOptional' || this.options.save !== false) {
1405+
if (edge.type !== 'peerOptional' ||
1406+
this.options.save !== false ||
1407+
this.#requestedTreeMutation) {
13921408
problems.push(edge)
13931409
}
13941410
continue

‎workspaces/arborist/test/arborist/build-ideal-tree.js‎

Lines changed: 207 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -4541,14 +4541,53 @@ t.test('re-queue already-seen nodes when placed dep invalidates peerOptional (sa
45414541
t.ok(tree.children.get('shared'), 'shared is in the tree')
45424542
})
45434543

4544-
t.test('skip invalid peerOptional edges in problemEdges when save=false (#8726)', async t => {
4545-
// With save=false (npm ci behavior), invalid peerOptional edges should NOT be treated as problems.
4546-
// We use update.names to force alpha into #problemEdges while [email protected] (invalid for alpha's peerOptional spec of 1.0.0) is already in the tree from the lockfile.
4544+
t.test('re-queue already-seen nodes when save=false update invalidates peerOptional', async t => {
45474545
const registry = createRegistry(t, false)
45484546

4549-
const utilPacks = registry.packuments(['1.0.0', '1.0.1'], 'util')
4550-
const utilManifest = registry.manifest({ name: 'util', packuments: utilPacks })
4551-
await registry.package({ manifest: utilManifest })
4547+
const alphaPack = registry.packument({
4548+
name: 'alpha',
4549+
version: '1.0.0',
4550+
peerDependencies: { shared: '1.0.0' },
4551+
peerDependenciesMeta: { shared: { optional: true } },
4552+
})
4553+
const alphaManifest = registry.manifest({ name: 'alpha', packuments: [alphaPack] })
4554+
await registry.package({ manifest: alphaManifest })
4555+
4556+
const betaPack = registry.packument({
4557+
name: 'beta',
4558+
version: '1.0.0',
4559+
dependencies: { shared: '^1.0.0' },
4560+
})
4561+
const betaManifest = registry.manifest({ name: 'beta', packuments: [betaPack] })
4562+
await registry.package({ manifest: betaManifest })
4563+
4564+
const sharedPacks = registry.packuments(['1.0.0', '1.1.0'], 'shared')
4565+
const sharedManifest = registry.manifest({ name: 'shared', packuments: sharedPacks })
4566+
await registry.package({ manifest: sharedManifest, times: 2 })
4567+
4568+
const path = t.testdir({
4569+
'package.json': JSON.stringify({
4570+
name: 'test-8726-save-false-update',
4571+
version: '1.0.0',
4572+
dependencies: {
4573+
alpha: '1.0.0',
4574+
beta: '1.0.0',
4575+
},
4576+
}),
4577+
})
4578+
4579+
const arb = newArb(path, { save: false })
4580+
const tree = await arb.buildIdealTree({ update: true })
4581+
4582+
t.ok(tree.children.get('alpha'), 'alpha is in the tree')
4583+
t.ok(tree.children.get('beta'), 'beta is in the tree')
4584+
t.ok(tree.children.get('shared'), 'shared is in the tree')
4585+
})
4586+
4587+
t.test('skip invalid peerOptional edges when save=false and not mutating (#8726)', async t => {
4588+
// With save=false and no tree mutation (npm ci behavior), invalid peerOptional
4589+
// edges should NOT be treated as problems. The locked tree is trusted as-is.
4590+
createRegistry(t, false)
45524591

45534592
const path = t.testdir({
45544593
'package.json': JSON.stringify({
@@ -4573,7 +4612,6 @@ t.test('skip invalid peerOptional edges in problemEdges when save=false (#8726)'
45734612
'node_modules/alpha': {
45744613
version: '1.0.0',
45754614
resolved: 'https://registry.npmjs.org/alpha/-/alpha-1.0.0.tgz',
4576-
dependencies: { util: '^1.0.0' },
45774615
peerDependencies: { shared: '1.0.0' },
45784616
peerDependenciesMeta: { shared: { optional: true } },
45794617
},
@@ -4586,22 +4624,178 @@ t.test('skip invalid peerOptional edges in problemEdges when save=false (#8726)'
45864624
version: '1.1.0',
45874625
resolved: 'https://registry.npmjs.org/shared/-/shared-1.1.0.tgz',
45884626
},
4589-
'node_modules/util': {
4590-
version: '1.0.0',
4591-
resolved: 'https://registry.npmjs.org/util/-/util-1.0.0.tgz',
4592-
},
45934627
},
45944628
}),
45954629
})
45964630

45974631
const arb = newArb(path, { save: false })
4598-
const tree = await arb.buildIdealTree({ update: { names: ['util'] } })
4632+
const tree = await arb.buildIdealTree()
45994633

46004634
t.ok(tree.children.get('alpha'), 'alpha is in the tree')
46014635
t.ok(tree.children.get('beta'), 'beta is in the tree')
46024636
t.equal(tree.children.get('shared').version, '1.1.0',
46034637
'shared stays at 1.1.0 - peerOptional mismatch is not treated as a problem')
4604-
t.ok(tree.children.get('util'), 'util is in the tree')
4638+
})
4639+
4640+
t.test('save=false non-mutating build ignores invalid peerOptional problem edges', async t => {
4641+
const registry = createRegistry(t, false)
4642+
4643+
const gammaPack = registry.packument({
4644+
name: 'gamma',
4645+
version: '1.0.0',
4646+
})
4647+
const gammaManifest = registry.manifest({ name: 'gamma', packuments: [gammaPack] })
4648+
await registry.package({ manifest: gammaManifest })
4649+
4650+
const path = t.testdir({
4651+
'package.json': JSON.stringify({
4652+
name: 'test-8726-save-false-non-mutating-problem-edges',
4653+
version: '1.0.0',
4654+
dependencies: {
4655+
alpha: '1.0.0',
4656+
},
4657+
}),
4658+
'package-lock.json': JSON.stringify({
4659+
name: 'test-8726-save-false-non-mutating-problem-edges',
4660+
version: '1.0.0',
4661+
lockfileVersion: 3,
4662+
requires: true,
4663+
packages: {
4664+
'': {
4665+
name: 'test-8726-save-false-non-mutating-problem-edges',
4666+
version: '1.0.0',
4667+
dependencies: { alpha: '1.0.0' },
4668+
},
4669+
'node_modules/alpha': {
4670+
version: '1.0.0',
4671+
resolved: 'https://registry.npmjs.org/alpha/-/alpha-1.0.0.tgz',
4672+
dependencies: { gamma: '1.0.0' },
4673+
peerDependencies: { shared: '1.0.0' },
4674+
peerDependenciesMeta: { shared: { optional: true } },
4675+
},
4676+
'node_modules/shared': {
4677+
version: '1.1.0',
4678+
resolved: 'https://registry.npmjs.org/shared/-/shared-1.1.0.tgz',
4679+
},
4680+
},
4681+
}),
4682+
})
4683+
4684+
const arb = newArb(path, { save: false })
4685+
const tree = await arb.buildIdealTree()
4686+
4687+
t.ok(tree.children.get('gamma'), 'missing non-peer dependency is resolved')
4688+
})
4689+
4690+
t.test('update does not leave invalid peerOptional edges when save=false', async t => {
4691+
// npm update defaults to save=false because it should not rewrite package.json
4692+
// ranges. It still mutates and writes the lockfile, so it cannot use the
4693+
// npm-ci behavior of trusting invalid peerOptional edges in the locked tree.
4694+
const registry = createRegistry(t, false)
4695+
4696+
const alphaPacks = [
4697+
registry.packument({ name: 'alpha', version: '1.0.0' }),
4698+
registry.packument({
4699+
name: 'alpha',
4700+
version: '1.1.0',
4701+
peerDependencies: { shared: '^1.0.0' },
4702+
peerDependenciesMeta: { shared: { optional: true } },
4703+
}),
4704+
]
4705+
const alphaManifest = registry.manifest({ name: 'alpha', packuments: alphaPacks })
4706+
await registry.package({ manifest: alphaManifest })
4707+
4708+
const sharedPacks = registry.packuments(['1.0.0', '2.0.0', '2.0.1'], 'shared')
4709+
const sharedManifest = registry.manifest({ name: 'shared', packuments: sharedPacks })
4710+
await registry.package({ manifest: sharedManifest, times: 2 })
4711+
4712+
const path = t.testdir({
4713+
'package.json': JSON.stringify({
4714+
name: 'test-peer-optional-update',
4715+
version: '1.0.0',
4716+
dependencies: {
4717+
alpha: '^1.0.0',
4718+
shared: '^2.0.0',
4719+
},
4720+
}),
4721+
'package-lock.json': JSON.stringify({
4722+
name: 'test-peer-optional-update',
4723+
version: '1.0.0',
4724+
lockfileVersion: 3,
4725+
requires: true,
4726+
packages: {
4727+
'': {
4728+
name: 'test-peer-optional-update',
4729+
version: '1.0.0',
4730+
dependencies: { alpha: '^1.0.0', shared: '^2.0.0' },
4731+
},
4732+
'node_modules/alpha': {
4733+
version: '1.0.0',
4734+
resolved: 'https://registry.npmjs.org/alpha/-/alpha-1.0.0.tgz',
4735+
},
4736+
'node_modules/shared': {
4737+
version: '2.0.0',
4738+
resolved: 'https://registry.npmjs.org/shared/-/shared-2.0.0.tgz',
4739+
},
4740+
},
4741+
}),
4742+
})
4743+
4744+
const arb = newArb(path, { save: false })
4745+
await t.rejects(arb.buildIdealTree({ update: true }), {
4746+
code: 'ERESOLVE',
4747+
}, 'lockfile-mutating update should reject instead of keeping an invalid peerOptional edge')
4748+
})
4749+
4750+
t.test('add does not leave invalid peerOptional edges when save=false', async t => {
4751+
// npm install <pkg> --no-save also mutates and writes the lockfile without
4752+
// changing package.json. It should not use npm-ci behavior either.
4753+
const registry = createRegistry(t, false)
4754+
4755+
const alphaPack = registry.packument({
4756+
name: 'alpha',
4757+
version: '1.0.0',
4758+
peerDependencies: { shared: '^1.0.0' },
4759+
peerDependenciesMeta: { shared: { optional: true } },
4760+
})
4761+
const alphaManifest = registry.manifest({ name: 'alpha', packuments: [alphaPack] })
4762+
await registry.package({ manifest: alphaManifest })
4763+
4764+
const sharedPacks = registry.packuments(['1.0.0', '2.0.0'], 'shared')
4765+
const sharedManifest = registry.manifest({ name: 'shared', packuments: sharedPacks })
4766+
await registry.package({ manifest: sharedManifest, times: 2 })
4767+
4768+
const path = t.testdir({
4769+
'package.json': JSON.stringify({
4770+
name: 'test-peer-optional-add',
4771+
version: '1.0.0',
4772+
dependencies: {
4773+
shared: '^2.0.0',
4774+
},
4775+
}),
4776+
'package-lock.json': JSON.stringify({
4777+
name: 'test-peer-optional-add',
4778+
version: '1.0.0',
4779+
lockfileVersion: 3,
4780+
requires: true,
4781+
packages: {
4782+
'': {
4783+
name: 'test-peer-optional-add',
4784+
version: '1.0.0',
4785+
dependencies: { shared: '^2.0.0' },
4786+
},
4787+
'node_modules/shared': {
4788+
version: '2.0.0',
4789+
resolved: 'https://registry.npmjs.org/shared/-/shared-2.0.0.tgz',
4790+
},
4791+
},
4792+
}),
4793+
})
4794+
4795+
const arb = newArb(path, { save: false })
4796+
await t.rejects(arb.buildIdealTree({ add: ['[email protected]'] }), {
4797+
code: 'ERESOLVE',
4798+
}, 'lockfile-mutating add should reject instead of keeping an invalid peerOptional edge')
46054799
})
46064800

46074801
t.test('peerOptional prefers existing tree node over registry fetch (#9249)', async t => {

0 commit comments

Comments
 (0)