Skip to content

Provide Python bindings as a package.#1856

Merged
borg323 merged 2 commits intoLeelaChessZero:masterfrom
ddobbelaere:python-bindings
Apr 18, 2023
Merged

Provide Python bindings as a package.#1856
borg323 merged 2 commits intoLeelaChessZero:masterfrom
ddobbelaere:python-bindings

Conversation

@ddobbelaere
Copy link
Copy Markdown
Contributor

@ddobbelaere ddobbelaere commented Mar 4, 2023

This PR makes the Python bindings (introduced by #1261) available as a pip-installable package. The compactness (addition of a single pyproject.toml) can be completely attributed to leveraging of the neat meson-python build backend (maintained by the official Meson team).

Example usage:

$ pip install --user .
Processing /home/dieter/Documents/projects/lc0
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: lczero-bindings
  Building wheel for lczero-bindings (pyproject.toml) ... done
  Created wheel for lczero-bindings: filename=lczero_bindings-0.1.0-cp311-cp311-linux_x86_64.whl size=8907199 sha256=b788d4a45303238bdc94c27d86f6944322316b99f9cec9d4f11dbbcc6b96e06a
  Stored in directory: /tmp/pip-ephem-wheel-cache-9mbnfknw/wheels/da/3d/c2/73dbc7c0dd5d92afb2e20d09c8dbbb09235998ef2766adc241
Successfully built lczero-bindings
Installing collected packages: lczero-bindings
Successfully installed lczero-bindings-0.1.0

$ pip show lczero-bindings 
Name: lczero-bindings
Version: 0.1.0
Summary: Leela Chess Zero Python bindings
Home-page: https://github.com/LeelaChessZero/lc0
Author: The LCZero Authors
Author-email: 
License: GNU GENERAL PUBLIC LICENSE
                       Version 3, 29 June 2007

 Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
...

$ python                         
Python 3.11.2 (main, Feb  8 2023, 00:00:00) [GCC 12.2.1 20221121 (Red Hat 12.2.1-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import lczero.backends
>>> dir(lczero.backends)
['Backend', 'BackendCapabilities', 'GameState', 'Input', 'LczeroException', 'Output', 'Weights', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__']

@borg323
Copy link
Copy Markdown
Member

borg323 commented Mar 4, 2023

Any way to customize the lc0 build parameters?

@ddobbelaere
Copy link
Copy Markdown
Contributor Author

ddobbelaere commented Mar 4, 2023

Any way to customize the lc0 build parameters?

I haven't tested it yet, but according to the docs, this could be achieved via pip install . --config-settings setup-args="option=value".

@ddobbelaere
Copy link
Copy Markdown
Contributor Author

ddobbelaere commented Mar 11, 2023

An example of how this PR can be useful for other Python packages (that can depend on the package lczero-bindings, even if it's not on PyPi (yet?)) can be found here.

Putting lczero-bindings @ git+https://github.com/LeelaChessZero/lc0.git in a requirements file or equivalent would do the trick. I hope this goes into master, so that I don't have to depend on my fork each time :).

@borg323 borg323 merged commit 6df87ad into LeelaChessZero:master Apr 18, 2023
PikaCat-OuO pushed a commit to official-pikafish/px0 that referenced this pull request Apr 26, 2023
@hwuebben
Copy link
Copy Markdown

@ddobbelaere which prerequisites does one need for this to work? Because I get this when running pip install --user git+https://github.com/LeelaChessZero/lc0.git.

backends.cc:643:1: warning: missing initializer for member ‘_typeobject::tp_                                                                                                                                                                                                cache’ [-Wmissing-field-initializers]
backends.cc:643:1: warning: missing initializer for member ‘_typeobject::tp_                                                                                                                                                                                                subclasses’ [-Wmissing-field-initializers]
backends.cc:643:1: warning: missing initializer for member ‘_typeobject::tp_                                                                                                                                                                                                weaklist’ [-Wmissing-field-initializers]
backends.cc:643:1: warning: missing initializer for member ‘_typeobject::tp_                                                                                                                                                                                                del’ [-Wmissing-field-initializers]
backends.cc:643:1: warning: missing initializer for member ‘_typeobject::tp_                                                                                                                                                                                                version_tag’ [-Wmissing-field-initializers]
backends.cc:643:1: warning: missing initializer for member ‘_typeobject::tp_                                                                                                                                                                                                finalize’ [-Wmissing-field-initializers]
backends.cc:643:1: warning: missing initializer for member ‘_typeobject::tp_                                                                                                                                                                                                vectorcall’ [-Wmissing-field-initializers]
[164/231] Linking target encoder_test
ninja: build stopped: subcommand failed.
----------------------------------------

WARNING: Discarding git+https://github.com/LeelaChessZero/lc0.git. Command error ed out with exit status 1: /usr/local/bin/python3.9 /usr/local/lib/python3.9/sit e-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_bui ld_wheel /tmp/tmp4iilmeq8 Check the logs for full command output.

ERROR: Command errored out with exit status 1: /usr/local/bin/python3.9 /usr/loc al/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prep are_metadata_for_build_wheel /tmp/tmp4iilmeq8 Check the logs for full command ou tput.

@ddobbelaere
Copy link
Copy Markdown
Contributor Author

ddobbelaere commented May 21, 2023

@hwuebben You should have a recent C++ compiler installed to build lc0 from sources. It might be a good idea to verify if that works first. Apart from that, looking at the full logs as indicated by pip is another way to proceed.

@ddobbelaere ddobbelaere deleted the python-bindings branch May 21, 2023 05:54
@ukrainec45
Copy link
Copy Markdown

Hi guys. I have a trouble building python bindings using the following command:
pip install --user .
It fails on metadata generation step producing a huge amount of C++ warnings (I guess) and the end of the output is the following:
image
Can you please assist with it?

@ukrainec45
Copy link
Copy Markdown

To my previous comment. I think I found the root cause of this:
image
The issue is with the C++ 14 standard is not supporting constructions used in the source code. I change it using Visual Studio (in backend project properties) but it doesn't make any difference, there is the same error.
image

What should I change to make it work?

@LilacPeregrine
Copy link
Copy Markdown

I'm having some trouble understanding how this works. I followed the instructions to the best of my knowledge.
(on Mac, Ventura)
When running "pip install --user ." , it just throws an error of invalid syntax.
I tried "pip install --user git+https://github.com/LeelaChessZero/lc0.git", and it seemed to download, but I already had the files?

When I run "pip show lczero-bindings", I get the same output you did, with the long GNU license.
However, trying to run "import lczero.backends" in python just leads to "ModuleNotFoundError: No module named 'lczero'".
Is there something I am missing?

@ddobbelaere
Copy link
Copy Markdown
Contributor Author

ddobbelaere commented Mar 24, 2024

@LilacPeregrine I don't really know (not a Mac user either). Here are some suggestions:

python -m venv venv
source venv/bin/activate
python -m pip install git+https://github.com/LeelaChessZero/lc0.git
python -c "import lczero.backends" # Should work
deactivate
  • Is your user installation folder of pip added to the search path of python?

Good luck. Maybe the lc0 Discord is also a better place to discuss these things further.

@LilacPeregrine
Copy link
Copy Markdown

@ddobbelaere
You appear to be correct. I have an issue with multiple python versions due to mac setup (at least initially). When installing lc0, it also appears to have somehow changed the python path for the terminal?

However, it works now, if I run it through python3.11 "name of file.py"
Thank you!

@Michal-Zablocki
Copy link
Copy Markdown

I'm experiencing the same issue as @ukrainec45

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants