Skip to content

FR: support dynamic CIBW_PRERELEASE_PYTHON and CIBW_FREETHREADED_SUPPORT in publish.yml #212

Closed

Description

@neutrinoceros

Rationale:
cibuildwheel itself allows the following combination of env vars

CIBW_BUILD="cp312-* cp313-*"
CIBW_PRERELEASE_PYTHONS="false"

which results in only cp312 wheels being built (since CPython 3.13 is at pre-release stage)
However, if I use OpenAstronomy/github-actions-workflows/.github/workflows/publish.yml@v1, I need to specify targets explicitly as, for instance:

targets: |
  - cp312-manylinux_x86_64
  - cp313-manylinux_x86_64
  - ...

which, if I also set CIBW_PRERELEASE_PYTHONS="false", results in the following error

cibuildwheel: No build identifiers selected: BuildSelector(build_config='cp313-manylinux_x86_64', skip_config='', requires_python=<SpecifierSet('>=3.10')>, prerelease_pythons=False, free_threaded_support=False)

This is important in case we want to set CIBW_PRERELEASE_PYTHONS dynamically, as for instance in astropy/astropy#16596

A possible workaround is to use wildcards when specifying targets, e.g.,

targets: |
  - cp312-*
  - cp313-*

but then jobs take much longer in wall time since you lose parallelism.

It seems possible to support this use case by updating target discovery so it's aware of these env vars (CIBW_PRERELEASE_PYTHON and CIBW_FREETHREADED_SUPPORT) and can dynamically skip any matching wheels as needed. However I see one difficulty to discuss/overcome
The list of versions to be ignored if CIBW_PRERELEASE_PYTHON!="true" depends on cibuildwheel's version (and indirectly on calendar), so supporting this would also require keeping the list in sync with cibuildwheel. To minimize impact of potential errors, we could choose to keep the current behavior in case CIBW_PRERELEASE_PYTHON isn't defined at all.

I'm happy to contribute a patch if this is desirable, once the matter is discussed.

Conversation originated at astropy/astropy#16596 (comment)

Activity

  1. ConorMacBride commented on Jul 7, 2024

    @ConorMacBride
    Member

    The easiest solution would be to pass the --allow-empty argument to cibuildwheel to prevent the error, however that's not possible with their GitHub Action. I've opened pypa/cibuildwheel#1922 to see if a CIBW_ALLOW_EMPTY env var could be added, and then you could also set that when you set CIBW_PRERELEASE_PYTHONS="true".

  2. neutrinoceros commented on Aug 4, 2024

    @neutrinoceros
    ContributorAuthor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions