changeset publish does not replace workspace: ranges with current version in workspace before publishing with yarn as a package manager

@zkochan

Affected Packages

@changesets/cli

yarn version: 2.1.1
changeset cli version: 2.9.2

Problem

@zkochan has already raised an issue with support of workspace ranges in dependencies, and he already proposed a solution for his immediate need of bumping workspace:x.y.z dependencies when changeset version is invoked.

However, this only bumps the semver range in the package.json but keeps the workspace: int it. That is the expected behavior locally but is not the expected behavior during publish.

workspace: with a semver range should have the 'workspace:' stripped before the publish.
workspace:* should have the '*' resolved to the workspace version and the the 'workspace:' stripped before the publish.

changeset publish does not do that resulting in the NPM package being published with 'workspace:' in the version specified which obviously does not work.

Proposed solution

Not sure, but @zkochan already had a comment regarding this at: https://github.com/atlassian/changesets/pull/311/files#r407058482 and to my knowledge yarn2 properly publishes such packages when invoked with 'yarn npm publish'. Not sure about pnpm.
Maybe the solution is to let the package manager handle this?

Workaround so far:
I am using yarn v2, so the workaround for me is ugly but produces consistent results.

Just stop at changeset version
Use yarn npm publish to publish packages
manually create git tags
push to git

Additional information regarding what Yarn V2 does during the publish:

Yarn packs the package before submitting to the registry,
prior to packing through the yarn-plugin-pack, a hook is invoked: beforeWorkspacePacking
that hooks resolves the workspace ranges and resplaces them in the generated manifest for the packing tool

https://github.com/yarnpkg/berry/blob/b9ca9d35e5b8592c5b84653122c832dc194793f5/packages/plugin-pack/sources/index.ts#L40

A possible solution would be to let the package manager, in this case yarn do the packing and publishing with its knowledge of the workspace and dependencies. However it seems that changeset only detects pnpm or npm and is not aware of yarn when publishing:
https://github.com/atlassian/changesets/blob/a57d16355ad7d67b18b768c8f79224d80afa507c/packages/cli/src/commands/publish/npm-utils.ts#L32

async function getPublishTool(cwd: string): Promise<"npm" | "pnpm"> {
  const pm = await preferredPM(cwd);
  if (!pm) return "npm";
  return pm.name === "pnpm" ? "pnpm" : "npm";
}

It also seems from the rest of that file that there are some strong assumptions that the publishing tool is npm or behaves like npm, rather than just an external command. Yarn may not conform to that but it remains quite simple to just delegate that task of publishing to its command line.

Can yarn recursively publish all the packages that have new versions?

pnpm has the pnpm publish -r command that (at the moment) works better than using changesets to publish the bumped versions. I assume Yarn also has something like that.

Maybe the solution is to let the package manager handle this?

Yes, I believe this is the best option for moving forward.

However it seems that changeset only detects pnpm or npm and is not aware of yarn when publishing:

Can using Yarn 2 be easily distinguished from Yarn 1? I don't know details, but Yarn 1 is explicitly not used for publishing right now because there were some problems with it.

@zkochan

Can yarn recursively publish all the packages that have new versions?

pnpm has the pnpm publish -r command that (at the moment) works better than using changesets to publish the bumped versions. I assume Yarn also has something like that.

Not to my knowledge in its current version. Yarn npm publish only acts on the current workspace. It can however just gracefully ignore republish with --tolerate-republish. So I assume, that the intent of whoever wrote that part was to just call this on every workspace and just ignore the ones whose versions was not bumped.

Note that prior to publish, you are supposed to have used yarn version plugin or something else like changesets to bump the versions in manifests that had changesets.

@zkochan I am also playing around with pnpm + rush, and I only switched to yarn today to check the differences in yarn v2. The couple of things that really struck me in yarn v2 are:

yarn + changesets = all you need for monorepo, and if you need more, you can just write a yarn plugin or a repo tool,
zero-install + pnp are much cleaner, node_modules is a mess with linking for debugging.
pnpm is cool but not enough on its own. monorepo tools are bloated with useless crap, redo what pnpm/yarn do better and integrate poorly with pnpm workspaces and yarn workspaces. I don't know if there is an alternative way to reduce that node_modules mess with pnpm and linking.

Maybe the solution is to let the package manager handle this?

Yes, I believe this is the best option for moving forward.

However it seems that changeset only detects pnpm or npm and is not aware of yarn when publishing:

Can using Yarn 2 be easily distinguished from Yarn 1? I don't know details, but Yarn 1 is explicitly not used for publishing right now because there were some problems with it.

Well it can definitely be distinguished but not with just 'preferred-pm' module. I believe that module could use more precise heuristics to detect yarn v2 from yarn v1. For example the presence of .yarn/releases/yarn-xxxx.js or ultimately it's a yarn and then we do a yarn --version :-)

BTW, for the workspace: problem, ultimately we can simply request yarn v2 to pack the workspace in a tgz, and slightly adjust the npm/pnpm command line for publishing to use the packed output. That's it. The rest gets taken care of by changesets as usual.

BTW, for the workspace: problem, ultimately we can simply request yarn v2 to pack the workspace in a tgz, and slightly adjust the npm/pnpm command line for publishing to use the packed output. That's it. The rest gets taken care of by changesets as usual.

Seems like a more complex thing to do (with no benefits?). If we could just offload both packing and publishing to a single command then this would make our implementation easier, right?

@Andarist

@Andarist I'm always an advocate for separation of concerns. In this case, leaving the package manager do what it is intended to do is the right approach. If something is missing in yarn/npm/pnpm to make things work flawlessly for publishing before changesets take the control back for tagging the release, we should just push for it to happen there.

The second thing that comes to my mind as a suggestion is to evaluate the positioning of the yarn version plugin with regard to changesets. That plugin is justified from a yarn point of view as it does handle something that the PM knows better than anyone else: dependency versions in a workspace. At the same time, changesets does something else with change logs that the PM has nothing to do with. There may be some synergies here and some collaboration to happen between the PM for version management, identifying the changes in a workspace, and using that information to produce a workspace friendly changelog, handle git tagging, handle pre-releases etc...

Such a thing could also be used with PNPM as well if it publishes similar APIs and allows for plugin architecture that hooks into the PM lifecycle.

Ultimately, the end goal should be that PM + changesets + few PM plugins => monorepos that work everywhere, in dev environments, in CI, with vscode, in docker, offline, ... without having to hack the monorepo tools (i.e. lerna, rush, nx,...). to bypass/eject/... things they do which circumvent what the underlying PM usually does. It should be easier today that both yarn and pnpm bring builtin support for workspaces.

Tagging this issue from yarn berry which is of extreme interest to changesets imho :-)
yarnpkg/berry#1510

@Andarist

Can using Yarn 2 be easily distinguished from Yarn 1? I don't know details, but Yarn 1 is explicitly not used for publishing right now because there were some problems with it.

@Andarist Yes, if the lockfile contains __metadata it's v2, otherwise it's v1

Any plan to fix this or coordinate with yarn berry?

@abdes

@abdes yes, we definitely want to fix that - sorry for not being responsive lately for this issue. Having a lot on my plate.

Would you be willing to work on fixing this? As discussed - I believe we should switch to using appropriate package managers for publishing, but we should keep using npm for Yarn 1 repositories. This would be the easiest solution for us, from what I understand.

Q: Is everything that is needed for this to work with Yarn 2 included in their core? Or are some parts of this covered by their plugins and people have to opt-in?

@Andarist

@Andarist I don't wanna rush this tbh :-). I saw that you have started the discussion with the lead maintainer of berry and I think this is the right thing to do first. He is extremely open and supportive, so I suggest that the first step should be to agree with @arcanis on how this would work. I feel that he is willing to consider a solution that perfectly integrates changeset with berry and that's what we want :-)

@Andarist

Can using Yarn 2 be easily distinguished from Yarn 1? I don't know details, but Yarn 1 is explicitly not used for publishing right now because there were some problems with it.

@Andarist Yes, if the lockfile contains __metadata it's v2, otherwise it's v1

Just spent some time on a package manager detection library: https://www.npmjs.com/package/@lerepo/detect-package-manager,

There is a CLI package with it for a quick test if you wanna check it out in your environemnt. https://www.npmjs.com/package/@lerepo/detect-package-manager-cli

It detects npm, pnpm and yarn, and provides the version installed in the system. Note that when using berry, the version of yarn is the one applied in the project/workspace and not the global one. So you may have a yarn v1 installed globally but a yarn workspace setup with yarn v2.1.1. The detection will return 2.1.1 as a version and not the global yarn. This is already something implemented in yarn itself.

If the project is using yarn v1, then the version will be that of the globally installed yarn.

@Andarist

@Andarist I am just getting back into the office from being out and trying not to look at any code. Let me kick the tires on that PR late this week to see how it works. Our use case is definitely more limited than others, but I can at least validate how it is working on our end.

@Andarist

@Andarist, Eagerly waiting for those snapshot releases.
As yarn add local-dependency adds dependency using the workspace:^ protocol by default and manually finding and replacing the real package version is a pain.

@HosseinAgha

@HosseinAgha till the feature lands, you can use this trick:

Run yarn pack to create an archive (workspace:* versions will be replaced)
untar the generated package.tgz into a _release folder
set publishConfig.directory to point this folder
call changeset publish

Basically:

 "publishConfig": {
    "directory": "_release/package"
  },
  "scripts": {
    // build - >pack -> untar archive in publishConfig folder
    "build-publishable-release": "yarn build && rimraf ./_release && yarn pack && mkdir ./_release && tar zxvf ./package.tgz --directory ./_release && rm ./package.tgz",
}

As tested here: soluble-io/cache-interop#135

@Andarist

@Andarist I have yet to circle around to test this... but we do have some big version bumps coming in some of our packages this week. When we do that, I want to try this branch in our pipeline and see how it does and I can report back. It should be out by Friday so I'll report back, though again... I suspect due to our limited use case with version ranges, this PR should easily work for us.

yarn changeset version
yarn workspaces foreach --from '@org/*' npm publish --access public
yarn changeset tag

Does the trick for me.

fix: use default version and publish

fix: add changeset-monorepo

fix: iso-error versioning

@belgattitude

@belgattitude thank you so much! This is exactly what I needed.
In my case, I'm using preconstruct to build all packages simultaneously and combine it with yarn's workspaces foreach command to do the manual packing you suggested.

The release is fully automated with changesets/action@v1 action.

If someone else uses preconstruct - here are my relevant release scripts.

  "scripts": {
    "build": "preconstruct build",
    "build:release": "yarn build && yarn forallpackages exec rm -rf _release && yarn forallpackages pack && yarn forallpackages exec \"mkdir _release && tar zxvf package.tgz --directory _release && rm package.tgz\"",
    "forallpackages": "yarn workspaces foreach --no-private --verbose",
    "release": "yarn build:release && changeset publish",
  },

And in the release action, simply call yarn release.

  - name: Create release PR or release packages
    uses: changesets/action@v1
    with:
      publish: yarn release
      commit: version-packages
      title: Version Packages
      createGithubReleases: true
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

Full example is here.

@Pagebakers

@Pagebakers workaround works perfectly and is quite elegant, and less error prune of creating temp directories and changing the package.json.

Minor issue with our monorepo as it contained private packages, be sure to use the private flag if you have private packages:

yarn changeset version
yarn workspaces foreach --no-private --from '@neo4j-ndl/*' npm publish --access public
yarn changeset tag

Edit:
The current approach while it works if you need to publish all the packages, if you have continuous publishing pipelines, and you don't publish all the packages together it will break, as it will try to publish all of them. Need some config to avoid this.

fix changeset dependency publishing (#3145)

@konsalex

As @konsalex points out, the workaround above releases all packages. If you're on yarn v4, you can use the --since argument and override the changeSetBaseRefs property in your .yarnrc.yml so that anything that gets changed by
changeset version will be used to run the publish command in only those packages. (make sure you don't have other changes from other CI/CD).

# ,yarnrc.yml

# We use this so that changesets can work in a monorepo with yarn
changesetBaseRefs: ["HEAD"]

yarn changeset version
yarn workspaces foreach --no-private --since npm publish --access public
yarn changeset tag

docs(scripts): expand publish.sh comments and add workspace: guard

Navigation Menu

Repository navigation

changeset publish does not replace workspace: ranges with current version in workspace before publishing with yarn as a package manager #432

Affected Packages

Problem

Proposed solution

12 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

changeset publish does not replace workspace: ranges with current version in workspace before publishing with yarn as a package manager #432

Description

Affected Packages

Problem

Proposed solution

Activity

abdes commented on Aug 4, 2020

zkochan commented on Aug 4, 2020

Andarist commented on Aug 4, 2020

abdes commented on Aug 4, 2020

abdes commented on Aug 4, 2020

Andarist commented on Aug 4, 2020

abdes commented on Aug 5, 2020

abdes commented on Aug 5, 2020

merceyz commented on Aug 5, 2020

abdes commented on Aug 18, 2020

Andarist commented on Aug 18, 2020

abdes commented on Aug 18, 2020

abdes commented on Aug 25, 2020

12 remaining items

brianespinosa commented on Nov 29, 2021

HosseinAgha commented on Dec 4, 2021

belgattitude commented on Dec 4, 2021

brianespinosa commented on Dec 13, 2021

Pagebakers commented on Jan 19, 2022

eegli commented on Jul 3, 2022

konsalex commented on Sep 26, 2022

hanseltime commented on Jun 12, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions