Skip to content

Race condition when building with Meson #28820

@lesteve

Description

@lesteve

Opening this to track it in an issue rather than seeing it appear in spread-out PRs.

The error looks like this:

[61/249] Compiling Cython source sklearn/linear_model/_sgd_fast.pyx
FAILED: sklearn/linear_model/_sgd_fast.cpython-312-darwin.so.p/sklearn/linear_model/_sgd_fast.pyx.c
cython -M --fast-fail -3 '-X language_level=3' '-X boundscheck=False' '-X wraparound=False' '-X initializedcheck=False' '-X nonecheck=False' '-X cdivision=True' '-X profile=False' --include-dir /Users/runner/work/1/s/build/cp312 sklearn/linear_model/_sgd_fast.pyx -o sklearn/linear_model/_sgd_fast.cpython-312-darwin.so.p/sklearn/linear_model/_sgd_fast.pyx.c


from cython cimport floating
from libc.math cimport exp, fabs, isfinite, log, pow, INFINITY

from ..utils._typedefs cimport uint32_t
^
------------------------------------------------------------

sklearn/linear_model/_sgd_fast.pyx:9:0: relative cimport from non-package directory is not allowed

or like this:

[60/249] Compiling Cython source /Users/runner/work/1/s/sklearn/cluster/_hdbscan/_linkage.pyx
  FAILED: sklearn/cluster/_hdbscan/_linkage.cpython-312-darwin.so.p/sklearn/cluster/_hdbscan/_linkage.pyx.c
  cython -M --fast-fail -3 '-X language_level=3' '-X boundscheck=False' '-X wraparound=False' '-X initializedcheck=False' '-X nonecheck=False' '-X cdivision=True' '-X profile=False' --include-dir /Users/runner/work/1/s/build/cp312 /Users/runner/work/1/s/sklearn/cluster/_hdbscan/_linkage.pyx -o sklearn/cluster/_hdbscan/_linkage.cpython-312-darwin.so.p/sklearn/cluster/_hdbscan/_linkage.pyx.c

  Error compiling Cython file:
  ------------------------------------------------------------
  ...

  cimport numpy as cnp
  from libc.float cimport DBL_MAX

  import numpy as np
  from ...metrics._dist_metrics cimport DistanceMetric64
  ^
  ------------------------------------------------------------

  /Users/runner/work/1/s/sklearn/cluster/_hdbscan/_linkage.pyx:38:0: 'sklearn/metrics/_dist_metrics.pxd' not found

Some files __init__.py of .pxd need to be copied to the Meson build dir before the extension using cimport can be built. Since the Meson (or rather ninja) build order is based on dependencies (there is no way to tweak the build order ninja-build/ninja#232), when you are unlucky it does not happen in the right order and the file is not found. Not clear to me right now whether this is a Meson issue or not.

For some unknown reason, until now this has always happened in the no-OpenMP build. I tried to reproduce on OSX with the no-OpenMP lock-file but I did not manage to reproduce.

Occurences of this:

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions