Skip to content

Comments

python3Packages.limits: fix input tarball non-reproducibility#171832

Merged
vcunat merged 1 commit intoNixOS:staging-nextfrom
delroth:python-limits-fix
May 7, 2022
Merged

python3Packages.limits: fix input tarball non-reproducibility#171832
vcunat merged 1 commit intoNixOS:staging-nextfrom
delroth:python-limits-fix

Conversation

@delroth
Copy link
Contributor

@delroth delroth commented May 6, 2022

Description of changes

Fixes regression from last bump (22f5d32), it looks like the
release tarball was changed upstream.

Tested: nix-build . -A home-assistant now passes on staging-next.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@delroth delroth requested review from FRidh and jonringer as code owners May 6, 2022 16:05
@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label May 6, 2022
@delroth delroth mentioned this pull request May 6, 2022
Merged
@delroth
Copy link
Contributor Author

delroth commented May 6, 2022

cc @fabaff

@dotlambda
Copy link
Member

dotlambda commented May 6, 2022

it looks like the release tarball was changed upstream.

Can we please tell upstream that's not okay?

@ofborg ofborg bot added 10.rebuild-darwin: 101-500 This PR causes between 101 and 500 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 1001-2500 This PR causes many rebuilds on Linux and should target the staging branches. labels May 6, 2022
@fabaff
Copy link
Member

fabaff commented May 6, 2022

It could be that I messed things up with the hash.

@delroth
Copy link
Contributor Author

delroth commented May 6, 2022

It could be that I messed things up with the hash.

I'm still investigating, but ofborg was fine with that hash according to the logs, and it doesn't match any other version I can find still published.

Can we please tell upstream that's not okay?

I'll file an issue when I've confirmed it's on their side. I suggest not blocking on this so staging-next can get greener faster? :)

@delroth
Copy link
Contributor Author

delroth commented May 6, 2022

Successful fetch on ofborg:

trying https://github.com/alisaifee/limits/archive/2.6.1.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   133  100   133    0     0    413      0 --:--:-- --:--:-- --:--:--   413
100 88449    0 88449    0     0  87918      0 --:--:--  0:00:01 --:--:-- 87918

Failed fetch on Hydra:

trying https://github.com/alisaifee/limits/archive/2.6.1.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 88440    0 88440    0     0   150k      0 --:--:-- --:--:-- --:--:--  150k

The archive size seems to have changed.

@dotlambda
Copy link
Member

dotlambda commented May 6, 2022

The problem is https://github.com/alisaifee/limits/blob/master/.gitattributes

@dotlambda
Copy link
Member

dotlambda commented May 6, 2022

See #84312

@delroth
Copy link
Contributor Author

delroth commented May 6, 2022

Interesting, thank you! I didn't know about this quirk.

@dotlambda
Copy link
Member

dotlambda commented May 6, 2022

This is how I solved it for another package but I don't think it will work in this case:

nixpkgs/pkgs/development/python-modules/pikepdf/default.nix

Line 41 in d21c94c

rm "$out/.git_archival.txt"

@delroth
pythonPackages.limits: hack around versioneer non-reproducibility
6e0f386 
Context: NixOS#84312 documents that using .gitattributes on GitHub results in
non-reproducible release tarballs. versioneer uses this to provide
version numbers for Python libraries.

Hack around that issue by deleting the file containing substitutions in
the fixed hash input derivation, and recreate the required info from the
derivation's version info at patchPhase time.
@delroth delroth force-pushed the python-limits-fix branch from c061bde to 6e0f386 Compare May 6, 2022 16:57
@delroth
Copy link
Contributor Author

delroth commented May 6, 2022

@dotlambda thanks for the reference, and there are actually a few other Python packages using versioneer that have similar workarounds. PTAL.

@delroth delroth changed the title pythonPackages.limits: fix wrong source sha256 pythonPackages.limits: fix input tarball non-reproducibility May 6, 2022
@delroth
Copy link
Contributor Author

delroth commented May 6, 2022

@ofborg build python39Packages.limits python310Packages.limits

@vcunat vcunat changed the title pythonPackages.limits: fix input tarball non-reproducibility python3Packages.limits: fix input tarball non-reproducibility May 7, 2022
@vcunat vcunat merged commit bb0c0b9 into NixOS:staging-next May 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FRidh FRidh Awaiting requested review from FRidh

@jonringer jonringer Awaiting requested review from jonringer

Assignees

No one assigned

Labels

6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 101-500 This PR causes between 101 and 500 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 1001-2500 This PR causes many rebuilds on Linux and should target the staging branches.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@delroth @dotlambda @fabaff @vcunat