You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[BUG] NPM 7.x broke the "--json" CLI parameter #2740
NPM's --json option allows scripts to invoke an NPM command and receive machine-readable JSON that can be easily processed. In previous NPM versions, the JSON was written to STDOUT and all other messages were written to STDERR, making it easy for a script to extract the JSON.
With NPM 7.x, everything is mixed together on STDERR. This defeats the point of the --json feature, which was to make parsing easy. Instead, a script has to implement heuristics to guess which STDERR lines are JSON or not. Scripts developed for NPM 6 now fail because they are reading the wrong stream.
Steps To Reproduce:
Correct behavior: With NPM 6.x, invoke this command:
npm search --json
The JSON output is written to STDOUT:
{"error": {"code": null,"summary": "search must be called with arguments","detail": ""}}
The console messages are written to STDERR:
npm ERR! search must be called with arguments
npm ERR! A complete log of this run can be found in:
Broken behavior: With NPM 7.5.4 (the latest version), invoke the same command. Both outputs are mixed together on STDERR. Nothing is written to STDOUT.
This regression appears to have been introduced with NPM 7.0.0.
Environment:
OS: All OS's
Node: 12.17.0
npm: 7.5.4
Reacted by Jordan Harband, Bryan Jones, Luke Hamburg, Patrick Braune, azu, Ross Gardiner, Mikael Finstad, william-will-angi, panlina and Tom Jenkinson
This was a significant regression for tools that rely on the NPM installation provided by Node.js: Since we can't assume that the developer has the latest Node.js, even if NPM eventually fixes this bug, it may be years before we can safely remove our workaround.
Looks like @RammasEchor made a PR but it was rejected.
Sounds like switching to yarn is the best approach. "Will get attention when we're freed up" sounds an awful lot like "Will sit in GitHub issues for seven years."
I did some digging and found that this was introduced in #2155 (which was to fix #2150).
It looks like npm ls --json (not sure about other commands yet) would print their own json blob and then the exit handler would print an additional json blob with the error. This caused unparseable json so the second blob was switched to stderr. Obviously, this caused regressions like this one.
A better solution I think would be to switch the main exit handler back to stdout and special case any commands that are already printing their own json output to not print more. I'm more confident (than I am about #2773) that this fix can come before any output refactor work we're doing. So hopefully it's less than seven years 😄
The OP suggests that everything was previously sent to stderr - that doesn't sound right though. If I understand the issue correctly the change only affected JSON errors - they are now written to stderr where previously they were written to stdout. So the change is of a smaller scope than OP suggests. Can anyone confirm this? Don't take me wrong - this is still super annoying and affecting me directly (I'm a Changesets maintainer and this has caused problems in our publishing scripts), I just want to make sure that I understand the issue correctly so I can fix it properly on our end.
Activity
RammasEchor commentedon Jun 14, 2021
Hi! I've been looking up this issue, and I would like to work on it.
bdkjones commentedon Sep 11, 2021
Any progress on this? It's a big regression from
6.xfor those of us who integrate npm into tooling. Thanks.octogonz commentedon Sep 12, 2021
This was a significant regression for tools that rely on the NPM installation provided by Node.js: Since we can't assume that the developer has the latest Node.js, even if NPM eventually fixes this bug, it may be years before we can safely remove our workaround.
Looks like @RammasEchor made a PR but it was rejected.
If it helps, here's the workaround that we used:
https://github.com/microsoft/rushstack/blob/b2aa54fa2a284e166529cdebbc3918bcd7dc7503/apps/rush-lib/src/logic/setup/SetupPackageRegistry.ts#L486
In brighter news, Node 16 will let you use Yarn without having to install it.
yarn infohas equivalent functionality, and Yarn's--jsonparameter is correctly implemented. 👍rossng commentedon Sep 24, 2021
This regression is causing problems for atlassian/changesets users. For example: changesets/changesets#397
bdkjones commentedon Oct 7, 2021
Sounds like switching to yarn is the best approach. "Will get attention when we're freed up" sounds an awful lot like "Will sit in GitHub issues for seven years."
lukekarrys commentedon Oct 7, 2021
I did some digging and found that this was introduced in #2155 (which was to fix #2150).
It looks like
npm ls --json(not sure about other commands yet) would print their own json blob and then the exit handler would print an additional json blob with the error. This caused unparseable json so the second blob was switched to stderr. Obviously, this caused regressions like this one.A better solution I think would be to switch the main exit handler back to stdout and special case any commands that are already printing their own json output to not print more. I'm more confident (than I am about #2773) that this fix can come before any output refactor work we're doing. So hopefully it's less than seven years 😄
Andarist commentedon Nov 14, 2021
The OP suggests that everything was previously sent to stderr - that doesn't sound right though. If I understand the issue correctly the change only affected JSON errors - they are now written to stderr where previously they were written to stdout. So the change is of a smaller scope than OP suggests. Can anyone confirm this? Don't take me wrong - this is still super annoying and affecting me directly (I'm a Changesets maintainer and this has caused problems in our publishing scripts), I just want to make sure that I understand the issue correctly so I can fix it properly on our end.
make check-already-published hit api directly
make check-already-published hit api directly (#4892)
feat: output json formatted errors on stdout
feat: output json formatted errors on stdout
feat: output json formatted errors on stdout
feat: output json formatted errors on stdout (#5716)
feat: output json formatted errors on stdout (#5716)
Work around npm bug with error reporting (#645)
make check-already-published hit api directly (video-dev#4892)