This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/proposal-for-installing-prebuilt-binaries-on-darwin/40843/1

I like the abstraction made by the dynamic unpack hook for DMG and PKG. Package complexity will drop (IMO) and package's lines reduced by a good margin.

But I think we should make sure every possible option is covered. As for some specific behavior, this might even add more complexity inside the package and possibly inside the hook.

If this solution is chosen, we should create a detailed documentation to avoid looking everywhere to find options, default behaviors, etc.

Love it 👍

Thank you for taking the time to have a look at this and your comment, @DataHearth, very much appreciated!

In your mind what is a good way to ensure that "every possible option is covered"? My first approach would be to refactor the packages in pkgs/os-specific/darwin and the ones that have undmg or _7zz as a build input.

Where would be a good place for such documentation? Somewhere in docs or rather pkgs/os-specific/darwin/README.md?

Looking at fetchzip I wonder if that could be changed or creating a similar fetcher, e.g. fetchdmgpkg would be more? helpful than the unpack-dmg-pkg hook? 🤔

I love the idea of having a unified way to package binaries for MacOS.

Regarding the treewide change,

1. hello: refactor to use darwin.installBinaryPackage would be easier to read comparing to treewide: refactor to use darwin.installBinaryPackage (hello).
2. It would make this PR easier to review if we change only some packages for demonstration and testing purpose, and split others into subsequent PRs.

Thanks for your input, @ShamrockLee, much appreciated!

1. Yes, that is a working title and when this draft PR is promoted to a PR I'll squash all those commits into one with an updated commit message

2. Agreed 🙂

Any opinion or advice you may have on the "💭 A Few Thoughts" I listed in the initial description?

Hey! Regarding creating a fetcher, I'm not a big fan of. As the "universal" way seems to fetch then unpack for most scenarios, I would keep it as a new package helper.

I think the best way to find out if you missed something, it's to check other packages which are using an unpack like 7z or unzip, etc... list all available options, and see if they're applicable to dmg and pkg files. And of course, test it on packages ^^.

I agree with @ShamrockLee , I think this would me much easier to review for a code maintainer with only 1 or 2 packages as examples.

Thanks for the feedback, @DataHearth, that's helpful.

I'll do some code level analysis and prepare some notes. In addition to that I'll split this PR into two: one for the install binary package changes and one for the treewide package changes (with a possible follow-up).

As requested this PR now affects fewer files and the majority of the actual package rewrites to use the changes proposed in this PR will be in #293961.
Please allow for some more time for a more methodical analysis and summarising the findings in a few notes (posted here later).

Also isn't this a copy of #147567

Thanks for taking a look at this, @Eveeifyeve, I've lost some context since last working on this, but I remember this PR going beyond what #147567 did, if I remember correctly undmg does not handle disk image using APFS.

Happy to have another look at #147567 to combine the best of the two.

I really like this approach and have started to try and write some nixos tests based on this pull request #441396 to

a. better understand how this would work and
b. allow us to ensure that further evolutions of this feature do not break existing packages.

This seems especially relevant to me, as installBinaryPackage tries to guess what actions are required to install a given binary package, which I suspect we will only be able to make work in about 90 to 98 % of cases.

Still, more is probably not required to make binary installers on macOS / Darwin much easier to create and maintain.

Some further developments I would like:

• AppCast support for automated updates using an update script to ease the maintenance burden
• Some Documentation on how this is supposed to be evolved, how much complications we want to absorb into the guessing script and at what point we draw the line and direct people to using mkDerivation again.
• More documentation on how this works.

All of these should probably happen in their own pull request later on, perhaps except the docs on. how we want this to evolve.

@afh I would like to add something like this to make it easier to debug why a build is failing:

diff --git a/pkgs/os-specific/darwin/install-binary-package/default.nix b/pkgs/os-specific/darwin/install-binary-package/default.nix
index d1c9b2a8f238..339a464187ec 100644
--- a/pkgs/os-specific/darwin/install-binary-package/default.nix
+++ b/pkgs/os-specific/darwin/install-binary-package/default.nix
@@ -43,6 +43,14 @@ stdenvNoCC.mkDerivation (
         runHook preInstall
 
         mkdir -p $out/Applications
+
+        if [ ! -d "${appName}" ]; then
+          echo "ERROR: Application '${appName}' not found in the current directory"
+          echo "Contents of the current directory:"
+          find .
+          exit 1
+        fi
+
         cp -R "${appName}" $out/Applications
 
         runHook postInstall

What do you think, is that too un nix like to spit out so much information on a build failure?

At first glance this feels more suitable for the checkPhase and either if ls -lsa isn't suitable enough I'd like to suggest to add -depth 2 to the find command and/or pipe it to head -100 to limit the error output somewhat.

What do you think?

At first glance this feels more suitable for the checkPhase and either if ls -lsa isn't suitable enough I'd like to suggest to add -depth 2 to the find command and/or pipe it to head -100 to limit the error output somewhat.

What do you think?

Both sounds great. Just a bit of output to get an orientation what's going on and why it is failing.

@afh and me had a bit of a conversation thinking about wether an alternative approach would be to provide darwin.fetchPKG and darwin.fetchDMG functions to provide almost the same drop in package complexity, but without having to explain darwin.installBinaryPackage to everyone. Still just a thought experiment, but never the less an interesting idea. Feedback welcome.

@afh and me had a bit of a conversation thinking about wether an alternative approach would be to provide darwin.fetchPKG and darwin.fetchDMG functions to provide almost the same drop in package complexity, but without having to explain darwin.installBinaryPackage to everyone. Still just a thought experiment, but never the less an interesting idea. Feedback welcome.

that would be really good, however, how do we deal with maintaining notarization though? as fixup can break many packages

@auscyber I just had a talk with @afh and the preliminary conclusion is, that we probably want both, fetchers and install binaryPackage as it allows us to expose arguments with the fetchers that give us a very uniform result, while removing much of the magic this module currently has, while still giving us enough control to provide a really short derivation.