Skip to content

Python bindings for Lc0 backend.#1261

Merged
mooskagh merged 5 commits intoLeelaChessZero:masterfrom
mooskagh:python
May 23, 2020
Merged

Python bindings for Lc0 backend.#1261
mooskagh merged 5 commits intoLeelaChessZero:masterfrom
mooskagh:python

Conversation

@mooskagh
Copy link
Copy Markdown
Member

@mooskagh mooskagh commented May 2, 2020

Building:

  1. Run meson.py with -Dpython_bindings
  2. After the build the file backends.cpython-38-x86_64-linux-gnu.so will appear in the build directory (in Windows, the extension should be .pyd).
  3. Put this file in the following directory structure:
(root)
|
+-- lczero/
    |
    +-- __init__.py
    +-- backends.cpython-38-x86_64-linux-gnu.so
  1. cd into (root) and start python.
  2. Do import lczero.backends and it should all work!

@mooskagh
Copy link
Copy Markdown
Member Author

mooskagh commented May 2, 2020

Example session:


In [1]: from lczero.backends import Weights, Backend, GameState

In [2]: w = Weights()
Found pb network file: ./id58612

In [3]: w.filters()
Out[3]: 128

In [4]: Backend.available_backends()
Out[4]:
['cudnn-auto',
 'cudnn',
 'cudnn-fp16',
 'opencl',
 'blas',
 'eigen',
 'random',
 'check',
 'roundrobin',
 'multiplexing',
 'demux']

In [5]: b = Backend(weights=w)
Creating backend [cudnn-auto]...
Switching to [cudnn]...
CUDA Runtime version: 10.2.0
Cudnn version: 7.6.5
Latest version of CUDA supported by the driver: 10.2.0
GPU: GeForce GTX 970
GPU memory: 3.93951 Gb
GPU clock frequency: 1253 MHz
GPU compute capability: 5.2

In [6]: g = GameState(moves=['e2e4, e7e5'])
---------------------------------------------------------------------------
LczeroException                           Traceback (most recent call last)
<ipython-input-6-cb5a9b68a873> in <module>
----> 1 g = GameState(moves=['e2e4, e7e5'])

LczeroException: Bad move: e2e4, e7e5

In [7]: g = GameState(moves=['e2e4', 'e7e5'])

In [8]: print(g.as_string())
rnbqkbnr
pppp.ppp
........
....p...
....P...
........
PPPP.PPP
RNBQKBNR KQkq[ah] (from white's eyes) Hash: 8572518730975593504


In [9]: i1 = g.as_input(b)

In [10]: i2 = GameState(fen='2R5/5kpp/4p3/p4p2/3B4/1K5N/4rNPP/8 b - - 0 29').as_input()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-10-3203972071a3> in <module>
----> 1 i2 = GameState(fen='2R5/5kpp/4p3/p4p2/3B4/1K5N/4rNPP/8 b - - 0 29').as_input()

TypeError: function missing required argument 'backend' (pos 1)

In [11]: i2 = GameState(fen='2R5/5kpp/4p3/p4p2/3B4/1K5N/4rNPP/8 b - - 0 29').as_input(b)

In [12]: bin(i1.mask(0))
Out[12]: '0b10000000000001110111100000000'

In [13]: bin(i2.mask(0))
Out[13]: '0b100001000100001100000000000000'

In [14]: o1, o2 = b.evaluate(i1, i2)

In [15]: o1.q()
Out[15]: 0.10548576712608337

In [16]: o2.q()
Out[16]: -0.9555819034576416

In [17]: zip(p1.moves(), o1.p_softmax(*p1.policy_indices()))
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-17-6c83ce4b9dff> in <module>
----> 1 zip(p1.moves(), o1.p_softmax(*p1.policy_indices()))

NameError: name 'p1' is not defined

In [18]: zip(g.moves(), o1.p_softmax(*g.policy_indices()))
Out[18]: <zip at 0x7efca0e8c340>

In [19]: list(zip(g.moves(), o1.p_softmax(*g.policy_indices())))
Out[19]:
[('b1a3', 0.0020779985934495926),
 ('b1c3', 0.047005292028188705),
 ('d1e2', 0.0033786550629884005),
 ('d1f3', 0.004004619084298611),
 ('d1g4', 0.001967670163139701),
 ('d1h5', 0.004279660061001778),
 ('e1e2', 0.0014249634696170688),
 ('f1e2', 0.007004433311522007),
 ('f1d3', 0.005651619751006365),
 ('f1c4', 0.017071455717086792),
 ('f1b5', 0.0045316447503864765),
 ('f1a6', 0.001234786119312048),
 ('g1e2', 0.007879829034209251),
 ('g1f3', 0.8301827907562256),
 ('g1h3', 0.002097159158438444),
 ('a2a3', 0.00787483248859644),
 ('a2a4', 0.004778245929628611),
 ('b2b3', 0.002658447250723839),
 ('b2b4', 0.0014852079330012202),
 ('c2c3', 0.0037954614963382483),
 ('c2c4', 0.00382616207934916),
 ('d2d3', 0.006489641964435577),
 ('d2d4', 0.006822879426181316),
 ('f2f3', 0.0023090150207281113),
 ('f2f4', 0.003144552232697606),
 ('g2g3', 0.0036341759841889143),
 ('g2g4', 0.0014319035690277815),
 ('h2h3', 0.009356374852359295),
 ('h2h4', 0.0026004489045590162)]

In [20]:

@mooskagh mooskagh requested review from Tilps and borg323 May 2, 2020 15:05
borg323
borg323 reviewed May 3, 2020
View reviewed changes
borg323
borg323 approved these changes May 21, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@borg323 borg323 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't build with python 3.7 with some link error, but this may be an issue with my setup. However python 3.8 seems to work fine. Also I get a few warnings when compiling.

borg323
borg323 reviewed May 21, 2020
View reviewed changes
meson.build
cpython = dependency('python3')
python.extension_module('backends',
[py_files + files],
include_directories: [includes],
Copy link
Copy Markdown
Member

@borg323 borg323 May 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding cpp_args: cc.get_supported_arguments(['-Wno-c99-extensions', '-Wno-pedantic', '-Wno-missing-field-initializers']), here silences a lot of the warnings, but probably not a good idea.

Copy link
Copy Markdown
Member Author

@mooskagh mooskagh May 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c99-extension that I use will appear in C++20, so I'd ignore that. Other ones probably worth fixing, but I'll do it later (clang also emits some set of warnings).

@mooskagh mooskagh merged commit ed956ed into LeelaChessZero:master May 23, 2020
@mooskagh mooskagh deleted the python branch May 23, 2020 07:03
@FrugoFruit90 FrugoFruit90 mentioned this pull request Aug 1, 2020
Open
@patrik-ha patrik-ha mentioned this pull request Sep 5, 2022
Merged
@sagebei
Copy link
Copy Markdown

sagebei commented Oct 4, 2022

How to use run MCTS simulations in the backends?

@ddobbelaere ddobbelaere mentioned this pull request Mar 4, 2023
Merged
@donaldlin30
Copy link
Copy Markdown

donaldlin30 commented Oct 2, 2023

Is there a way I can retrieve centipawn scores from the binding?

@XintianHan
Copy link
Copy Markdown

XintianHan commented Nov 1, 2023

How do I know the winner of the game?

@Hidancloud
Copy link
Copy Markdown

Hidancloud commented Nov 23, 2023

Can you guys please work a bit more with documentation and tutorials? I'm a big fan of Lc0 and there are so many great things in this repo, but there is no more or less full tutorial & explanation how and what to use here. I suppose, your solution far exceeded the amount of work that should be documented.

@psktam
Copy link
Copy Markdown

psktam commented Jan 7, 2024

@Hidancloud not sure if you already saw this but just in case looks like there's a more up-to-date solution here: #1856

If all you want to do is install the python bindings, looks like you just need to cd into the project root for this repository and run pip install --user . which will allow you to import the lczero.backends package in python.

@Iriskinn
Copy link
Copy Markdown

Iriskinn commented Aug 6, 2024 •
edited
Loading

Hi there, I get the following error while installing bindings with pip install -e . (after downloading the repo and replacing c++17 with c++20 in meson.build):

[3/9] Linking target chessboard_test.exe
  FAILED: chessboard_test.exe
  "link"  /MACHINE:x64 /OUT:chessboard_test.exe chessboard_test.exe.p/src_chess_board_test.cc.obj chessboard_test.exe.p/subprojects_googletest-release-1.10.0_googletest_src_gtest-all.cc.obj "/release" "/nologo" "/OPT:REF" "lc0_lib.lib" "/SUBSYSTEM:CONSOLE" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
  LINK : fatal error LNK1181: cannot open input file 'lc0_lib.lib'

I have Win64 platform with no CUDA or cuDNN, perform the installation in a virtual env.
What can be a problem in my case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@borg323 borg323 borg323 approved these changes

@Tilps Tilps Awaiting requested review from Tilps

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@mooskagh @sagebei @donaldlin30 @XintianHan @Hidancloud @psktam @Iriskinn @borg323