Skip to content

nodePackages: fix builds on x86_64-darwin#193533

Merged
happysalada merged 1 commit intoNixOS:masterfrom
lilyinstarlight:fix/nodepackages-darwin
Sep 30, 2022
Merged

nodePackages: fix builds on x86_64-darwin#193533
happysalada merged 1 commit intoNixOS:masterfrom
lilyinstarlight:fix/nodepackages-darwin

Conversation

@lilyinstarlight
Copy link
Member

Description of changes

Follow-up to #193337 to fix several x86_64-darwin build failures due to missing xcrun/xcodebuild (see https://hydra.nixos.org/eval/1782541#tabs-now-fail)

I'm waiting on OfBorg to build these because I do not personally have access to any darwin systems. It probably also fixes aarch64-darwin errors, but those are additionally failing because the nodejs build had a transient failure (and the failure has been since cached - see #193331)

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Sep 29, 2022
@ofborg ofborg bot requested a review from midchildan September 29, 2022 14:54
@ofborg ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. labels Sep 29, 2022
@lilyinstarlight lilyinstarlight marked this pull request as ready for review September 29, 2022 15:00
@lilyinstarlight
Copy link
Member Author

@ofborg build bitwarden-cli epgstation joplin mastodon-bot theLoungePlugins.plugins.giphy

Follow-up to NixOS#193337 to fix several x86_64-darwin build failures due to
missing xcrun/xcodebuild
@lilyinstarlight
Copy link
Member Author

@ofborg build bitwarden-cli epgstation joplin mastodon-bot theLoungePlugins.plugins.giphy

@veprbl
Copy link
Member

veprbl commented Sep 29, 2022

Result of nixpkgs-review pr 193533 run on x86_64-darwin 1

1 package failed to build:
  • epgstation
4 packages built:
  • bitwarden-cli
  • joplin
  • mastodon-bot
  • theLoungePlugins.plugins.giphy

@winterqt
Copy link
Member

FYI, we have a patch for Node 14 to disable the use of xcbuild (no clue why it wasn't applied to any future versions).

As part of my work on buildNpmPackage, I was going to port it to newer versions, which would (hopefully) mean that it's not required for these packages. If you'd like to do it instead, let me know -- no need to duplicate work :)

@lilyinstarlight
Copy link
Member Author

Result of nixpkgs-review pr 193533 run on x86_64-darwin 1

1 package failed to build:

  • epgstation

4 packages built:

  • bitwarden-cli
  • joplin
  • mastodon-bot
  • theLoungePlugins.plugins.giphy

@veprbl, do you have logs for that epgstation failure? It works in OfBorg

FYI, we have a patch for Node 14 to disable the use of xcbuild (no clue why it wasn't applied to any future versions).

As part of my work on buildNpmPackage, I was going to port it to newer versions, which would (hopefully) mean that it's not required for these packages. If you'd like to do it instead, let me know -- no need to duplicate work :)

Ah, that would explain why it suddenly became required then! I was wondering but didn't really dive into it because I'm not too familiar with the Node.js ecosystem and just kinda wanted node2nix/nodePackages working again. I'll see if I can forward-port that patch real quick and test whether it fixes these builds as an alternative to adding xcbuild to stuff

(Also the buildNpmPackage stuff looks lovely and I can't wait for it to be available)

@veprbl
Copy link
Member

veprbl commented Sep 29, 2022

Result of nixpkgs-review pr 193533 run on x86_64-darwin 1

5 packages built:
  • bitwarden-cli
  • epgstation
  • joplin
  • mastodon-bot
  • theLoungePlugins.plugins.giphy

@veprbl
Copy link
Member

veprbl commented Sep 29, 2022

The failure was due to a known nix bug with sandboxing.

@winterqt
Copy link
Member

The failure was due to a known nix bug with sandboxing.

Just for posterity, logs/what bug? (I'd build myself but I can't at the moment and figure you have them on hand :)

@lilyinstarlight
Copy link
Member Author

Could either of you run nix build github:lilyinstarlight/nixpkgs/fix/nodepackages-darwin-v2#bitwarden-cli on darwin to see if nodejs and bitwarden-cli build with just the xcodebuild patch?

That branch is effectively @winterqt's suggestion above, but I don't have a Mac to test it on

@veprbl
Copy link
Member

veprbl commented Sep 29, 2022

The failure was due to a known nix bug with sandboxing.

Just for posterity, logs/what bug? (I'd build myself but I can't at the moment and figure you have them on hand :)

This one NixOS/nix#2311

@bobby285271 bobby285271 added the 12.approvals: 1 This PR was reviewed and approved by one person. label Sep 29, 2022
@lilyinstarlight
Copy link
Member Author

lilyinstarlight commented Sep 30, 2022

@winterqt, I tested out my branch in the macOS runner in GitHub Actions and it does look like just forward-porting that patch will work to remove the need for xcbuild in these packages

It also looks like the buildNodejs function patches xcode_emulation.py in postPatch and postInstall to deal with lack of Xcode, but not in a way that seems to work with any recent Node.js versions. So I could instead update that to patch it in a somewhat interpreter agnostic way -- it could just be somewhat fragile and version-specific patches like the one for v14 may be better long-term

Also, I'm assuming a PR to patch Node.js to avoid the need for xcbuild would have to go to staging, since touching nodejs makes quite a few rebuilds (~1k on x86_64-linux alone) even if it is a low-risk change?

Edit: I also did not see any reason that patch was dropped for the v16 update in #120008

@winterqt
Copy link
Member

It also looks like the buildNodejs function patches xcode_emulation.py in postPatch and postInstall to deal with lack of Xcode, but not in a way that seems to work with any recent Node.js versions.

It looks like the patches you talk about are ancient (introduced in 822abc4), so that doesn't surprise me. We can definitely drop them in favor of version-specific patches, IMO.

Also, I'm assuming a PR to patch Node.js to avoid the need for xcbuild would have to go to staging, since touching nodejs makes quite a few rebuilds (~1k on x86_64-linux alone) even if it is a low-risk change?

The high amount of rebuilds was caused by your nodePackages change, I presume. I think that's a candidate for staging then, yes. (@vcunat can confirm, though.)

Edit: I also did not see any reason that patch was dropped for the v16 update in #120008

Yeah, I assume it was just forgotten about. 😔

@vcunat
Copy link
Member

vcunat commented Sep 30, 2022

Also, I'm assuming a PR to patch Node.js to avoid the need for xcbuild would have to go to staging, since touching nodejs makes quite a few rebuilds (~1k on x86_64-linux alone) even if it is a low-risk change?

Yes, sounds like staging to me. (Doesn't sound urgent, like security fixes.)

@happysalada happysalada merged commit dc642a9 into NixOS:master Sep 30, 2022
@winterqt
Copy link
Member

@lilyinstarlight Your nodePackages PR added, at most, 500 packages to this rebuild count -- any clue what the other 500+ packages that would be affected by the Node change would be? I'd really only think it would be nodePackages at the most (at least now, since it's not using Node 14), though we also have assorted web apps and such.

@vcunat
Copy link
Member

vcunat commented Sep 30, 2022

OfBorg lists all rebuilt packages if you click on "Details" on the "ofborg-eval" line.

@lilyinstarlight lilyinstarlight deleted the fix/nodepackages-darwin branch September 30, 2022 16:25
@winterqt
Copy link
Member

OfBorg lists all rebuilt packages if you click on "Details" on the "ofborg-eval" line.

Somehow I forgot this... thanks.

@lilyinstarlight
Copy link
Member Author

I've opened #193759 targeting staging with the change to patch nodejs as before and reverted some of the changes from this PR that added xcbuild in overrides

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: darwin Running or building packages on Darwin 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. 12.approvals: 1 This PR was reviewed and approved by one person.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants