Repository navigation
Packages should be published in topological order #238
Description
Affected Packages
I think @changesets/assemble-release-plan should do the sorting.
Problem
A broken release (for example stopped in the middle) might leave NPM registry in a broken state.
Proposed solution
Packages should be published in topological order - dependencies before dependents.
I'm not sure what should be done about cycles.
Reacted by Steven, Nathan Rajlich, Sean Massa, Kenrick, Diego Sogari and Haz
Activity
emmatown commentedon Jan 3, 2020
This sounds fine to me.
@changesets/assemble-release-planwouldn't do the sorting though since it's not related to publishing, it would happen in@changesets/cli.Andarist commentedon Jan 3, 2020
My reasoning was that
@changesets/assemble-release-plancould output "ready to be published" list - so already presorted with this in mind.zkochan commentedon Apr 15, 2020
pnpm publish -ractually publishes packages in topological order. We have a package for sorting the packages: @pnpm/sort-packages.You could probably use the graph-sequencer package directly
Andarist commentedon Apr 15, 2020
@zkochan could u look into
internalPublishto see if it satisfies pnpm users? it is executed on per-package basis and it seems that maybe this strategy might not be ideal for some reason when using pnpm?zkochan commentedon Apr 15, 2020
The code looks fine. It will probably work fine with pnpm. But I need it to run in topological order because in the pnpm monorepo, the main package should be published last. Before the main package is published, we actually install all of its dependencies before publish, and pack them (kind of like bundled dependencies work).
Also, I think it would be a good idea to publish all packages with some temporary dist tag and only update all the versions to "latest" when each package has been successfully published.
This is how Lerna does publishing. pnpm recursive publish also works like that.
zkochan commentedon Apr 16, 2020
Actually, even if this will be fixed, I will stick with pnpm for doing the publish.
changeset publishis extremely slow due to callingnpm infoon each package in the monorepo. pnpm also checks each version of each package and whether it was released. But pnpm does it using its own programmatic API, concurrently and fast.Andarist commentedon Apr 16, 2020
Would you have any direct tips on what you are doing differently to achieve the same result?
zkochan commentedon Apr 16, 2020
We have this library, called
@pnpm/npm-resolver. We run it for every package in the monorepo. It is really fast because it also uses a cache in the filesystem. So if you already checked if[email protected]is published to the repository, the check will not be repeated. Also, it happens concurrently (16 requests at a time) and in the same process, npm is not started for each package, separately.Here's the actual step where we do it
https://github.com/pnpm/pnpm/blob/ee4a5b030b87b28f6086301540bbd806d844910e/packages/plugin-commands-publishing/src/recursivePublish.ts#L61-L69
Andarist commentedon Apr 16, 2020
I'm not too intimate with the algorithm used here but it seems that this part should be roughly the same as changesets should not need to check if a single package is published multiple times, or does it repeat those calls for some reason?
The note about reusing npm - I took a quick look at your implementation, npm resolver etc. Could you confirm that you call npm APIs directly rather than using their CLI? And thus it is more lightweight?
zkochan commentedon Apr 16, 2020
pnpm has its own API for making requests to the npm registry. That's what we use.
Use `pnpm publish -r` to publish packages
Use `pnpm publish -r` to publish packages (#10022)
dsogari commentedon Mar 8, 2024
For anybody who is using plain npm, the
"workspaces"field can be specified in topological order, andnpm publish --workspaceswill respect that order. Unfortunately,changeset publishdoes not respect it. 😢The only issue with using npm directly is that (quoting the docs):