Skip to content

Question: How can one use the passthru in the overrides? #514

Closed

Description

@vale981

I recently fixed a typo that I came across while trying to get the passthru argument to work:
390173b

Unfortunately, I still could not find out how that could be achieved.
Specifying

matplotlib = super.matplotlib.overridePythonAttrs (
                      old: {
                        passthru.enableTk = true;
                      }
                    );

in the overrides didn't seem to do anything.

Activity

  1. changed the title [-]Question: How to use passthru in the overrides.[/-] [+]Question: How can one use passthru in the overrides?[/+] on Jan 18, 2022
  2. changed the title [-]Question: How can one use passthru in the overrides?[/-] [+]Question: How can one use the passthru in the overrides?[/+] on Jan 18, 2022
  3. adisbladis commented on Jan 18, 2022

    @adisbladis
    Member

    This isn't at all obvious, but passthru.args is set to whatever is passed to the function that creates the derivation.

    Matplotlib didn't look at passthru.args, but instead looked at passthru.
    This should be fixed in #516.

    With this change you should be able to create an override looking something like:

    matplotlib = super.matplotlib.override { enableTk = true; }
    
  4. vale981 commented on Jan 20, 2022

    @vale981
    ContributorAuthor

    thank you very much for the hint.
    I'll try it out :)

  5. mschneiderwng commented on Jan 23, 2022

    @mschneiderwng

    How is this applied? I tried

            poetryEnv = pkgs.poetry2nix.mkPoetryEnv {
              projectDir = ./.;
              overrides = pkgs.poetry2nix.overrides.withDefaults (final: prev: {
                matplotlib = prev.matplotlib.override { enableTk = true; };
              });
              editablePackageSources = {
                my-app = ./src;
              };
            };
    

    and get the error

    import _tkinter # If this fails your Python may not be configured for Tk
    ModuleNotFoundError: No module named '_tkinter'
    
  6. adisbladis commented on Jan 23, 2022

    @adisbladis
    Member

    How is this applied? I tried

            poetryEnv = pkgs.poetry2nix.mkPoetryEnv {
              projectDir = ./.;
              overrides = pkgs.poetry2nix.overrides.withDefaults (final: prev: {
                matplotlib = prev.matplotlib.override { enableTk = true; };
              });
              editablePackageSources = {
                my-app = ./src;
              };
            };
    

    and get the error

    import _tkinter # If this fails your Python may not be configured for Tk
    ModuleNotFoundError: No module named '_tkinter'
    

    That does indicate that it is working, but you would also need to pass python3Full to mkPoetryEnv as the default python build doesn't enable tk.

  7. mschneiderwng commented on Jan 24, 2022

    @mschneiderwng

    I got the impression that the flags enableTk and enableQt load the necessary packages with these lines.

    At the moment, enableTk and enableQt have no effect. Qt works if and only if I pass pkgs.qt5.full to buildInputs and Tk only works if I pass pkgs.python3Full to buildInputs.

    What is the purpose then of enableTk and enableQt ?

  8. sepiabrown commented on Apr 10, 2022

    @sepiabrown
    Contributor

    Changing the below code in overrides/default.nix

     matplotlib = super.matplotlib.overridePythonAttrs (
          old:
            let
              enableGhostscript = old.passthru.args.enableGhostscript or false;
    

    into

     matplotlib = super.matplotlib.overridePythonAttrs (
          old:
            let
              enableGhostscript = old.passthru.args.enableGhostscript;
    

    makes the nix repl moan the following.

    error: attribute 'enableGhostscript' missing

    shell.nix ``` { pkgs ? import ../nixpkgs {} }: let myAppEnv = pkgs.poetry2nix.mkPoetryEnv { projectDir = ./.; python = pkgs.python3Full; overrides = pkgs.poetry2nix.overrides.withDefaults (final: prev: { matplotlib = prev.matplotlib.override { enableGhostscript = true; }; }); }; in {inherit myAppEnv pkgs;} ```
    pyproject.toml ``` [tool.poetry] name = "my-python-package" version = "0.1.0" description = "" authors = ["Suwon Park "]

    [tool.poetry.dependencies]
    python = "3.9.10"
    matplotlib = "^3.5.1"
    kiwisolver = "<=1.3.2"
    cppy = "<=1.1.0"

    [tool.poetry.dev-dependencies]

    [build-system]
    requires = ["poetry-core>=1.0.0"]
    build-backend = "poetry.core.masonry.api"

    </details>
    
  9. sepiabrown commented on Apr 10, 2022

    @sepiabrown
    Contributor

    My hypothesis is that when using pkgs.poetry2nix.overrides.withDefaults, if the python package you want to override is either in the file overrides/default.nix or overrides/build-systems.json, you cannot override with the method that @adisbladis showed as follows.

    This isn't at all obvious, but passthru.args is set to whatever is passed to the function that creates the derivation.

    Matplotlib didn't look at passthru.args, but instead looked at passthru. This should be fixed in #516.

    With this change you should be able to create an override looking something like:

    matplotlib = super.matplotlib.override { enableTk = true; }
    

    That said, the above method works with pkgs.poetry2nix.overrides.withoutDefaults and works with packages like pyro-ppl that are neither in overrides/default.nix nor overrides/build-systems.json.

    I've also tested the above hypothesis with matplotlib which is only in overrides/default.nix. After commenting out matplotlib part in overrides/default.nix, matplotlib has passthru.args with correct values!

    shell.nix ``` sepiabrown@localhost:~/testtesttest$ cat shell.nix { pkgs ? import ../nixpkgs {} }: let myAppEnv = pkgs.poetry2nix.mkPoetryEnv { projectDir = ./.; overrides = pkgs.poetry2nix.overrides.withDefaults (final: prev: { peewee = prev.peewee.override { withPostgres = true; test_peewee = "peewee"; }; pyro-ppl = prev.pyro-ppl.override { test_pyro = "pyro"; }; tomli = prev.tomli.override { test_tomli = "tomli"; }; emoji = prev.emoji.override { test_emoji = "emoji"; }; matplotlib = prev.matplotlib.override { enableGhostscript = true; test_mat = "mat"; }; }); }; in {inherit myAppEnv pkgs;} # myAppEnv.env ```
    pyproject.toml ``` [tool.poetry] name = "my-python-package" version = "0.1.0" description = "" authors = ["Suwon Park "]

    [tool.poetry.dependencies]
    python = "3.9.10"
    numpy = "^1.22.3"
    pyro-ppl = "^1.8.1"
    peewee = "^3.14.10"
    tomli = "^2.0.1"
    emoji = "^1.7.0"
    matplotlib = "^3.5.1"

    [tool.poetry.dev-dependencies]

    [build-system]
    requires = ["poetry-core>=1.0.0"]
    build-backend = "poetry.core.masonry.api"

    </details>
    
  10. AlexChalk commented on Jul 5, 2023

    @AlexChalk
    Contributor

    This works for me:

    { pkgs ? import <nixos> { } }:
    
    pkgs.poetry2nix.mkPoetryEnv {
      projectDir = ./.;
      python = pkgs.python3Full;
      overrides = pkgs.poetry2nix.overrides.withDefaults (self: super: {
        matplotlib = super.matplotlib.overridePythonAttrs (
          old: {
            enableTk = true;
          }
        );
      });
    }
    
  11. jicksaw commented on Mar 28, 2024

    @jicksaw

    super.matplotlib.overridePythonAttrs didn't work for me, though I'm using a flake. The derivation's attribute does change, but it doesn't affect matplotlib's buildInputs. super.matplotlib.override does work.

    There's separate error when using enableTk; The pkgs.libX11 input isn't found. I'm guessing because it's a spliced package, and for some reason the spliced top-level packages aren't in the pkgs that's being evaluated.

    ++ lib.optionals enableTk [ pkgs.tcl pkgs.tk self.tkinter pkgs.libX11 ]

    Here's the poetry2nix flake template modified to use Qt instead:

    {
      description = "Application packaged using poetry2nix";
    
      inputs = {
        flake-utils.url = "github:numtide/flake-utils";
        nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
        poetry2nix = {
          url = "github:nix-community/poetry2nix";
          inputs.nixpkgs.follows = "nixpkgs";
        };
      };
    
      outputs = { self, nixpkgs, flake-utils, poetry2nix }:
        flake-utils.lib.eachDefaultSystem (system:
          let
            # see https://github.com/nix-community/poetry2nix/tree/master#api for more functions and examples.
            pkgs = nixpkgs.legacyPackages.${system};
            inherit (poetry2nix.lib.mkPoetry2Nix { inherit pkgs; }) mkPoetryApplication overrides;
          in
          {
            packages = {
              myapp = mkPoetryApplication {
                projectDir = self;
                python = break pkgs.python3Full;
                overrides = overrides.withDefaults (self: super: {
                  matplotlib = super.matplotlib.override (
                    old: {
                      enableQt = true;
                    }
                  );
                });
              };
              default = self.packages.${system}.myapp;
            };
    
            devShells.default = pkgs.mkShell {
              inputsFrom = [ self.packages.${system}.myapp ];
              packages = [ pkgs.poetry ];
    
              # Setup QT_PLUGIN_PATH
              # Source: https://discourse.nixos.org/t/python-qt-woes/11808/12
              buildInputs = [
                pkgs.qt5.wrapQtAppsHook
                pkgs.makeWrapper
              ];
              shellHook = ''
                setQtEnvironment=$(mktemp --suffix .setQtEnvironment.sh)
                makeWrapper "/bin/sh" "$setQtEnvironment" "''${qtWrapperArgs[@]}"
                sed "/^exec/d" -i "$setQtEnvironment"
                source "$setQtEnvironment"
              '';
            };
          });
    }
  12. vale981 commented on May 13, 2024

    @vale981
    ContributorAuthor

    I totally forgot that I ever opened that issue and recently "re-discovered" the solution.
    For me, the original tip by @adisbladis works in a flake.

    overrides = poetry2nix.overrides.withDefaults (final: prev: {
                  matplotlib = with pkgs; prev.matplotlib.override (
                    {
                      enableGtk3 = true;
                    }
                  );
                });

    For me this resolves the issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions