Add the conda package manager#296461
Conversation
github-actions
bot
added
the
6.topic: python
ofborg
bot
added
8.has: package (new)
pkgs/development/python-modules/conda-libmamba-solver/default.nix
Outdated
Show resolved
Hide resolved
pkgs/development/python-modules/conda-libmamba-solver/default.nix
Outdated
Show resolved
Hide resolved
pkgs/development/python-modules/conda-package-handling/default.nix
Outdated
Show resolved
Hide resolved
pkgs/development/python-modules/conda-package-streaming/default.nix
Outdated
Show resolved
Hide resolved
pkgs/development/python-modules/conda-package-streaming/default.nix
Outdated
Show resolved
Hide resolved
github-actions
bot
added
6.topic: nixos
|
@SomeoneSerge I've made all of the changes. |
ofborg
bot
added
10.rebuild-darwin: 11-100
EricTheMagician
force-pushed
the
conda
branch
3 times, most recently
from
a694418 to
3fb0646
Compare
There was a problem hiding this comment.
The same comment about ordering the attributes applies: sourceRoot and {pre,post}Patch go right after src, before .*Inputs
There was a problem hiding this comment.
| ] ++ lib.optionals withConda [ | |
| (lib.cmakeBool "ENABLE_CONDA" true) | |
| (lib.cmakeBool "ENABLE_CONDA" withConda) |
There was a problem hiding this comment.
| platforms = lib.platforms.all; |
nixpkgs has not really python support outside of the default linux + darwin, so we might as well ommit it like in many other packages.
There was a problem hiding this comment.
... different python package sets ... https://github.com/NixOS/nixpkgs/pull/296461/files#r1573356428
(libmamba.override { python3Packages = pythonPackages; }) with pythonPackages taken from callPackage
As brought up by @SuperSandro2000, the current version is broken for python3XXPackages != python3Packages:
❯ nix path-info --recursive github:EricTheMagician/nixpkgs/conda#python310Packages.libmambapy --derivation | grep python3-
/nix/store/xg1ykawijjmpnky86808jb8kgkr64yl7-python3-minimal-3.11.9.drv
/nix/store/rxkj7xkj1wyzqvzzs2x564bx07b0nz0y-python3-minimal-3.11.9.drv
/nix/store/h1lywp34xgl1i5ilsddnqyi327yihzi7-python3-minimal-3.11.9.drv
/nix/store/wspwq9dgw9ck99dymf67sd3j1g1r4rnr-python3-3.11.9.drv
/nix/store/z60c5ran5vqkva3y0d1d5hxzykrw8pfq-python3-3.10.14.drv
/nix/store/ryvpn8k6cv9mjd5g8d3w26zsjqy92m6a-python3-3.11.9-env.drv(the above should only list 3.10, not 3.11
|
I've found some usability issues: while the basic functionality does work, I think the patch for conda I added makes it really fragile. Setting the environment variable |
| - if context.dev: | ||
| - return { | ||
| - "CONDA_EXE": sys.executable, | ||
| - # do not confuse with os.path.join, we are joining paths with ; or : delimiters | ||
| - "PYTHONPATH": os.pathsep.join( | ||
| - (CONDA_SOURCE_ROOT, os.environ.get("PYTHONPATH", "")) | ||
| - ), | ||
| - "_CE_M": "-m", | ||
| - "_CE_CONDA": "conda", | ||
| - "CONDA_PYTHON_EXE": sys.executable, | ||
| - } | ||
| - else: | ||
| - bin_dir = "Scripts" if on_win else "bin" | ||
| - exe = "conda.exe" if on_win else "conda" | ||
| - # I was going to use None to indicate a variable to unset, but that gets tricky with | ||
| - # error-on-undefined. | ||
| - return { | ||
| - "CONDA_EXE": os.path.join(sys.prefix, bin_dir, exe), | ||
| - "_CE_M": "", | ||
| - "_CE_CONDA": "", | ||
| - "CONDA_PYTHON_EXE": sys.executable, | ||
| - } | ||
| + import sys | ||
| + return { | ||
| + "CONDA_EXE": sys.executable, | ||
| + # do not confuse with os.path.join, we are joining paths with ; or : delimiters | ||
| + "PYTHONPATH": os.pathsep.join( | ||
| + [CONDA_SOURCE_ROOT, os.environ.get("PYTHONPATH", "")] + [path for path in sys.path if "site-packages" in path] | ||
| + ), | ||
| + "_CE_M": "-m", | ||
| + "_CE_CONDA": "conda", | ||
| + "CONDA_PYTHON_EXE": sys.executable, |
There was a problem hiding this comment.
while the basic functionality does work, I think the patch for conda I added makes it really fragile. Setting the environment variable PYTHONPATH breaks the conda executable, which brings it full circle to this comment:
Could you walk us through the patch? When is context.dev true? Is the difference that upstream sets a clean PYTHONPATH that only refers to the conda environment and your patch propagates the dependencies of the conda script itself (sys.path)? Why was that required?
github-actions
bot
removed
6.topic: nixos
|
I've reduced the scope for now. |
SomeoneSerge
left a comment
SomeoneSerge
left a comment
There was a problem hiding this comment.
I'm going to submit a different PR for replacing the top-level conda package when that is ready.
Sounds like a good decision. @SuperSandro2000 would you unpin your "change request"?
SuperSandro2000
left a comment
SuperSandro2000
left a comment
There was a problem hiding this comment.
I've just pushed the last little change myself, LGTM
|
@ofborg build python311Packages.conda python312Packages.conda |
6 participants
Description of changes
I wanted to add a newer version of the conda package manager without breaking the existing conda package manager.
There is one at the top-level and that one is based on downloading miniconda and setting up a FHS. This one introduces some patches
libsolvandcondaitself to make it work in the context of nix without needing the FHS hierarchy.After some initial discussions with some maintainers, the default conda package has now been replaced this PR, which does not depend on the FHS. @jluttine @bhipple
Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.