Hello,

I'd appreciate some help in using cibuildwheel in a project that I work on. Not sure if this is the right place to ask for help, so if there's a better place please let me know.

This project is an implementation of an algorithm for text clustering, named sequential Information Bottleneck (sIB). It is a product of IBM Research, and recently IBM contributed it to the open source community. The GitHub is here: https://github.com/IBM/sib.

The source code and supplementary files (setup.py and requirements.txt) are already in the repository, and I'm looking for some assistance in creating the build system.

In IBM, we used a custom-made build system for this. For creating Linux wheels, we used the manylinux dockers and a script for iterating over several python versions in each container. The wheel files were collected in a mounted shared directory.

One of the things that complicate the story is that the python code uses a cython extension and this extension needs to be built as part of the build process. In fact, there are two setup.py files - one for the wheel and one for the extension. Therefore, for every python version, a short script is called from the project root directory (For MacOS and Windows a similar script is used):

#!/bin/bash

# build the extension
cd src/sib/c_package
python setup.py build_ext --inplace

# build the wheel
cd ../../..
python setup.py sdist bdist_wheel 

I assume that this is not how the project is supposed to be built when using cibuildwheel. I looked at two examples: 1 and 2 but I'm not sure I fully understand them.

Looking at 2, if I got this right, build_wheels builds and packages the wheel file on every os mentioned in the matrix, and build_sdist packages the source code into a tar.gz file. Also, upload_pypi depends on the completion of the other two jobs since it uploads their result to the index.

What I'm not sure about is where the building of the extension should be done. Should it be a separate job before build_wheels, or part of the build_wheels job. And if it is part of build_wheels, should it be a separate run command, or invoked as part of the command python -m cibuildwheel --output-dir wheelhouse. In the latter case, I might need to change the main setup.py in some way to make he extension build part of the wheel build but I'm unfamiliar with a way to do that.

Any help with this will be appreciated.

Also, I wanted to ask why the package vcpython27 is being installed in the examples above.

Thanks.