Skip to content

Saving packages with their dependency set in the store #1001

@zkochan

Description

@zkochan

As of now (v1.30.0) pnpm stores in the global store only the "pure" unpacked files of every package. Packages with their resolved dependencies are placed in node_modules/.<package ID>/node_modules/.

Cons of this approach:

  • This design is bad for using pnpm in monorepo projects (Support peer dependencies when symlinking in development #604). Monorepos need singletons.
  • This solution allows only one dependency graph per package@version in a project, which makes it impossible to support npm's lockfile format (Install from npm-shrinkwrap.json #134).
  • Each installation requires lots of filesystem operations even when everything is already in the store. Symlinking packages, that don't have dependencies, directly from the store, makes installation 8% faster. (Using the experimental independent-leaves config)

Saving packages with their dependency graphs in the store.

Instead of saving packages with their dependencies in project/node_modules/.<package ID>/node_modules, we can save them in store/<package ID>/<dependency graph hash>/node_modules.

No changes to the lockfile are needed! By adding the new hashes to the lockfile we could save a lot of cryptography operations. However, the domino effect of hash changes would make lockfile merge conflicts unbearable.

Pros:

Cons:

  • More manipulations are required when packages have peer dependencies which are resolved by the root dependencies of the project
  • Linking of packages can be started only once all the subdependencies of the package are resolved. This might not be a performance problem as we already wait for this. We have to resolve all the peer dependencies of subdependencies before we start linking packages
  • Packages won't have access to the hoisted dependencies in the root of the project's node_modules and in node_modules/.pnpm/node_modules. We can mitigate this issue by adding this locations to the NODE_PATH env variable.

cc @etamponi, @vjpr


related: #604
we'll need information about projects using the store. Related issue: #6929

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions