BLD: ensure libnpymath and highway static libs use hidden visibility#26286
BLD: ensure libnpymath and highway static libs use hidden visibility#26286rgommers merged 1 commit intonumpy:mainfrom
Conversation
The effect this has is that symbols don't get re-exported as public
by Python extension modules that link with these shared libraries.
E.g., building on macOS arm64 with Clang changes the set of exported
symbols for `_multiarray_umath` from:
```
% dy2ld_info -exports build/numpy/_core/_multiarray_umath.cpython-39-darwin.so
build/numpy/_core/_multiarray_umath.cpython-39-darwin.so [arm64]:
-exports:
offset symbol
0x00112BA4 _PyInit__multiarray_umath
0x001C955C _npy_spacingf
0x001C95F8 _npy_spacing
...
<many more _npy symbols>
```
to:
```
build/numpy/_core/_multiarray_umath.cpython-311-darwin.so [arm64]:
-exports:
offset symbol
0x0010B8F8 _PyInit__multiarray_umath
```
This works for all compilers that support GNU-style
`__attribute__((visibility("hidden"))`, which both GCC and Clang do.
Note that the `libnpyrandom` static library is left alone here. Trying
to change visibility there breaks a test for CFFI, because that test
is accessing private symbols. This is clearly wrong, but explicitly
documented at
https://numpy.org/devdocs/reference/random/extending.html#cffi. So
leaving that alone here. `libnpyrandom` isn't used by SciPy anymore
and may well have zero users left, so it's not critical.
seberg
left a comment
seberg
left a comment
There was a problem hiding this comment.
LGTM, is highway part of NumPy 2 and we should backport it? OTOH, I guess it isn't really high priority and if someone ends up relying on it by accident it isn't the end of the world.
|
Highway is included in the 2.0 branch indeed: https://github.com/numpy/numpy/tree/maintenance/2.0.x/numpy/_core/src. Backporting is a toss-up I'd say. It may be useful to pull it in, since we're doing another RC. It's not critical though, and it's conceivable (if pretty unlikely) that some package is relying on private symbols being exported by some other package or extension module somehow. So I'm happy either way. |
|
Thanks for the review @seberg. Going to hit the green button on this now that it's still fresh in memory, so we can use it in SciPy once the new nightlies are available. |
2 participants
The effect this has is that symbols don't get re-exported as public by Python extension modules that link with these shared libraries. E.g., building on macOS arm64 with Clang changes the set of exported symbols for
_multiarray_umathfrom:to:
This works for all compilers that support GNU-style
__attribute__((visibility("hidden")), which both GCC and Clang do. Tested on SciPy as well, it has the same effect there. For example,scipy.special._ufuncswas re-exporting all_npy_*symbols and after this change that is no longer the case.Note that the
libnpyrandomstatic library is left alone here. Trying to change visibility there breaks a test for CFFI, because that test is accessing private symbols. This is clearly wrong, but explicitly documented athttps://numpy.org/devdocs/reference/random/extending.html#cffi. So leaving that alone here.
libnpyrandomisn't used by SciPy anymore and may well have zero users left, so it's not critical.