tree-wide: cudaPackages attributes should not cause default eval to fail#274319
Conversation
d8b7e2f to
e7cf70f
Compare
|
Note Template PR=274319; \
SYSTEM="aarch64-linux"; \
CUDA_SUPPORT="true"; \
CUDA_CAPABILITIES='[ "7.5" ]'; \
nixpkgs-review pr "$PR" \
--system "$SYSTEM" \
--no-shell \
--checkout commit \
--allow aliases \
--build-args "--max-jobs 1" \
--skip-package-regex 'python3(\d){1,2}Packages\.(pytorch-pfn-extras|ffcv)' \
--skip-package-regex 'python3(\d){1,2}Packages\.(shap|mlflow|optuna)' \
--extra-nixpkgs-config "{
allowUnfree = true;
allowBroken = false;
cudaSupport = ${CUDA_SUPPORT:-false};
cudaCapabilities = ${CUDA_CAPABILITIES:-[]};
}"The packages Template log archival command: PR=274319; \
RUN_NUMBER=13; \
SYSTEM="aarch64-linux"; \
CUDA_CAPABILITIES="7_5"; \
tar -cvf "$SYSTEM-cap-$CUDA_CAPABILITIES-pr-$PR-$RUN_NUMBER.tar.zst" \
-I "zstd -T0 --ultra -22" \
-C "$HOME/.cache/nixpkgs-review/pr-$PR-$RUN_NUMBER" \
.Logs are made available at https://drive.google.com/drive/folders/1GgABhwa2ooKeZXMqf5He6PcyClNrE6cQ?usp=share_link ❌
|
philiptaron
left a comment
There was a problem hiding this comment.
I took a look. Really appreciate the aim of this issue. Will continue to look as it gets towards completion.
4f9e5d1 to
138ac2c
Compare
|
My understanding is that I don't think we need to specify |
d981cdb to
4c27877
Compare
Ah, yes I agree. I've switched to using
nixpkgs/pkgs/development/cuda-modules/flags.nix Lines 148 to 166 in c7530f3 And here's the new code: As a result, we can now see platforms for packages which aren't available on the current host (previously this would return the empty list when evaluated on $ nix eval --impure .#cudaPackages.cuda_compat.meta.platforms
[ "aarch64-linux" ] |
|
OfBorg, you're killing me. I'm running I suspect it's something to do with EDIT: Oh, right, EDIT2: Now it's failing while evaluating CUDA overrides: https://gist.github.com/GrahamcOfBorg/27410dc21a9025acf2a21363e2df48d2 |
4c27877 to
d83fc41
Compare
3d1531b to
90d6577
Compare
9330978 to
64daa11
Compare
cudaPackages: guard expressions against null values
…supported platform
9a89180 to
b2f97e1
Compare
| let | ||
| isBadPlatform = lists.any trivial.id (attrsets.attrValues finalAttrs.badPlatformsConditions); | ||
| in | ||
| lists.optionals isBadPlatform finalAttrs.meta.platforms; |
There was a problem hiding this comment.
"huh! Why are platforms and badPlatforms the same?"
Is this less or more confusing than adding just the current platform?
There was a problem hiding this comment.
I think it's fine, personally.
There was a problem hiding this comment.
Yup probably a good thing we get the same meta regardless of where we evaluate
philiptaron
left a comment
There was a problem hiding this comment.
I see ofborg is happy. The code changes look reasonable to me.
| # `all-packages.nix`, which is evaluated on all systems. As such, we need to handle unsupported | ||
| # systems gracefully. | ||
| # getNixSystem :: String -> String | ||
| getNixSystem = redistArch: attrsets.attrByPath [ redistArch ] "unsupported-${redistArch}" { |
There was a problem hiding this comment.
To be clear, I introduced the stringly string because I was in a rush. We should probably return some sort of Result/{ value, error } (e.g. in the format of tryEval)
I'll add a ticket to the project board for that
|
I wonder if |
How about that |
Description of changes
Closes #273794.
Most of the packages in
cudaPackagesare not source-available (examples includecuda_nvcc,cudnn, andlibcutensor). However, some packages, likencclandnccl-testsare source-available.Because
srcis evaluated as part ofOfBorg'sevalcheck, we must make sure it is always able to evaluate. For packages which are not source-available, we setsrc = nullwhen building for an unsupported platform or configuration (because NVIDIA does not provide such a package for download) and updatemeta.platformsandmeta.brokenappropriately. For packages which are source-available, we specifymeta.badPlatformsandmeta.broken.Previously,
source-availablepackages were excluded from the package set when using an unsupported platform or configuration; this lead to downstream users of such packages using existence in the package set as an indication of whether such a package would be functional.With this change, downstream users should check the derivation's
meta.unsupportedattribute.Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.