Skip to content

Added support to use NCCL with python3 and fixed nvml.dll related error under windows#5400

Open
willyd wants to merge 2 commits intoBVLC:windowsfrom
willyd:nccl-py3-win
Open

Added support to use NCCL with python3 and fixed nvml.dll related error under windows#5400
willyd wants to merge 2 commits intoBVLC:windowsfrom
willyd:nccl-py3-win

Conversation

@willyd
Copy link
Copy Markdown
Contributor

@willyd willyd commented Mar 14, 2017

WIP. This may be of interest for the master branch since the python 3 error is not OS specific. I may split this PR in two.

@willyd
Copy link
Copy Markdown
Contributor Author

willyd commented Mar 14, 2017

@GaryKT can you confirm that this PR fixes #5401 and #5347 for you as well?

@shelhamer
Copy link
Copy Markdown
Member

Closing as presumed fixed until hearing otherwise.

@shelhamer shelhamer closed this Apr 12, 2017
@willyd
Copy link
Copy Markdown
Contributor Author

willyd commented Apr 12, 2017

@shelhamer I will keep this one open since I would like that fix to go in on windows. There are two fixes in this PR:

  1. Getting nccl to correctly load the nvml.dll since it is not on PATH by default. This one is Windows specific.
  2. Getting the uid's to work on python3 since python 3 and python 2 handle strings differently. Although I have not tested this change on a UNIX platform I believe the changes could also benefit the master branch.

@willyd willyd reopened this Apr 12, 2017
@cypof
Copy link
Copy Markdown
Member

cypof commented Apr 12, 2017

I tested the first commit about uid and it works, can you remove the second commit so that we can merge?

@willyd
Copy link
Copy Markdown
Contributor Author

willyd commented Apr 12, 2017

@cypof I wrote the code in a rush so let me clean it up (I think I can come up with less changes) and then I will resubmit the first commit to master.

@willyd
Copy link
Copy Markdown
Contributor Author

willyd commented Apr 13, 2017

@cypof @shelhamer I could simplify the first commit if we use a relatively recent version of boost.

According to the docs we require 1.55 but according to the CMake build we require 1.46, I assume that the docs is more up to date than the CMake build. Am I correct?

I am asking since 1.53 handles the conversion from python strings and bytes to C++ std::string automatically. If we can assume this we only have to convert manually from C++ to python.

See this: https://github.com/boostorg/python/blob/boost-1.53.0/src/converter/builtin_converters.cpp#L381

@willyd
Copy link
Copy Markdown
Contributor Author

willyd commented Apr 14, 2017

Part of this was merge in #5527.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants