Skip to content

Repeatable fetcher tests #133739

@roberth

Description

@roberth

Describe the bug

Fetchers are hard to test because when the output sha is verified, Nix won't download it again.
This leads to confusion and PRs that are effectively untested because the others do not realize that they need to remove the store path before testing.

Examples:

#133515

Steps To Reproduce

Steps to reproduce the behavior:

  1. change a fetcher
  2. nix-build
  3. break the fetcher
  4. nix-build
  5. "success" but the fetcher is broken
  6. make a PR
  7. wasted effort

Expected behavior

The fetcher runs every time. This is expected but not realistic, because Nix can not guess what to re-fetch.
Instead, we can make fetcher testing easier by adding fetcher tests following the pattern:

let
  plain = fetchFoo { ... };

  computeSaltedHash = salt: runCommand "source" {} ''
    cp -r --no-preserve-mode ${plain} src
    echo ${salt} >src/fetcher-test-salt

    # not sure which command this should be
    compute-nar-hash src >$out
  '';

  fetchWithSaltedHash = salt: plain.override {
    postFetch = "echo ${salt} >$out/fetcher-test-salt";
    sha256 = builtins.readFile computeSaltedHash; # "import from derivation"
  }

in
f {
  tests.fetch-salted = fetchWithSaltedHash;
}

So this uses IFD which is probably ok for ofborg?
Regarding f; instead of passthru somewhere, we'll have to use __functor to set tests.fetch-salted on the fetchers because they're functions.
The salt may need to be passed externally in case of pure evaluation. We could make it a Nixpkgs config parameter that defaults to builtins.currentTime.

Screenshots

Additional context

Notify maintainers

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
 - fetchgit
 - fetchFromGitHub
 - fetchsvn
 - dockerTools.pullImage
 - fetch*
# a list of nixos modules affected by the problem
module:

Metadata

Metadata

Assignees

No one assigned

    Labels

    6.topic: testingTooling for automated testing of packages and modules

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions