I believe to understand the distinction between checkPhase and installCheckPhase in the meaning of the stdenv chapter in the manual. What I don't understand is why Python adopted one over the other, and what the motivation and benefit of rolling that decision back is.

This decision seems to date back to 2009.

@mweinelt Thank you for finding out the history of the decision.

The issue is that we are

• specifying the installCherkPhase attributes using the checkPhase arguments of buildPython* and
• pretending that we are using checkPhase in the corresponding section of the Nixpkgs Manual.

It is not about choosing between chcekPhase and installCheckPhase but about informing users of the build helpers about this choice.

Why then not move us into checkPhase if it is all the same? I don't see the value in adopting another lingo when a whole bunch of packages have already adopted it.

That's indeed another way to go.

The decision to implement doCheck with doInstallCheck dates back to 2016, in commit 3e05cce made by @FRidh in #18143. The comment mentioning the long-term goal to move to installCheckPhase was added in 2019 in commit f7e28bf.

The issue is that we are

• specifying the installCherkPhase attributes using the checkPhase arguments of buildPython* and
• pretending that we are using checkPhase in the corresponding section of the Nixpkgs Manual.

It is not about choosing between chcekPhase and installCheckPhase but about informing users of the build helpers about this choice.

The manual notes that checkPhase is used instead of installCheckPhase.
However, I do not know why we adopted checkPhase and not installCheckPhase as it was.
Perhaps because installCheckPhase was rarely used and unfamiliar to most people?

The checkPhase for python maps to the installCheckPhase on a normal derivation. This is due to many python packages not behaving well to the pre-installed version of the package. Version info, and natively compiled extensions generally only exist in the install directory, and thus can cause issues when a test suite asserts on that behavior.

https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/python.section.md#testing-python-packages-testing-python-packages

I get that installCheckPhase runs post-install and is more useful for the reasons mentioned.

The frustrating thing here is, that we simplified the lingo to build-system and dependencies, only to move from checkInputs to nativeCheckInputs, and now to nativeInstallCheckInputs. I get that all of these bits and pieces have meaning, but we'll be dealing with the resulting churn for the next two releases at least.

And then there is the matter of nix-community/nix-init#419 🫠

I get that installCheckPhase runs post-install and is more useful for the reasons mentioned.

The frustrating thing here is, that we simplified the lingo to build-system and dependencies, only to move from checkInputs to nativeCheckInputs, and now to nativeInstallCheckInputs. I get that all of these bits and pieces have meaning, but we'll be dealing with the resulting churn for the next two releases at least.

And then there is the matter of nix-community/nix-init#419 🫠

Build system and dependencies are essentially another layer on top. The underlying stdenv atttributes will always be needed for all other cases.

I think motivation at the time was consistency. Probably it was not given much thought.

After some thought, I'm personally against the migration to installCheckPhase for now.
It has been over six months since we started the switch to build-system/dependencies, and this migration is painful.

Indeed, this change is correct and should be done to ensure consistency with the phase definition.
However, it will be more difficult to encourage contributors to migrate just for that reason, even if it is backwards compatible.

Regarding installCheckPhase, I feel that many of its functions have delegated passthru.tests, such as tester.testVersion.
It is no exception in python, where #272177 tries to separate build and test.
I don't know how it will finally be implemented, but it seems that it would be less burdensome if this PR change were adopted at the same time as the separation.

After some thought, I'm personally against the migration to installCheckPhase for now.
It has been over six months since we started the switch to build-system/dependencies, and this migration is painful.

Indeed, this change is correct and should be done to ensure consistency with the phase definition.
However, it will be more difficult to encourage contributors to migrate just for that reason, even if it is backwards compatible.

Thank you for sharing your precious experiences about attribute renaming.

Your right. The change wouldn't be helpful to the community if the pain caused by the mass renaming outweighs the benefits of the name correctness.

Regarding installCheckPhase, I feel that many of its functions have delegated passthru.tests, such as tester.testVersion.
It is no exception in python, where #272177 tries to separate build and test.
I don't know how it will finally be implemented, but it would be less burdensome if this PR change were adopted at the same time as the separation.

Delegating checks to passthru.test (as package tests) sounds like the right way to go. Let's push that one first.

If this is not the change we want to make now, should we close this pull request or mark it as a draft?

I reopened this PR to prepare for future adoption of fixed-point arguments (buildPythonPackage (finalAttrs: { })) and <pkg>.overrideAttrs.

• buildPython*: support fixed-point arguments #271387
• Tracking: Python packages' overrideAttrs overriding and finalAttrs referencing #379602

Since the finalAttrs are from stdenv.mkDerivation, we will have finalAttrs.doInstallCheck, finalAttrs.installCheckInputs. For example, when #271387 merged, there will be expressions like

buildPythonPackage (finalAttrs: {
  doCheck = false;
  checkInputs = lib.optionals finalAttrs.doInstallCheck [
    # installCheckInputs elements
  ];
  passthru = {
    tests = {
      python-check = finalAttrs.finalPackage.overrideAttrs {
        doInstallCheck = true;
      };
    };
  };
});