ENH: Support export PyArray_API and PyUFunc_API from shared libraries#22774
ENH: Support export PyArray_API and PyUFunc_API from shared libraries#22774huangqinjin wants to merge 2 commits intonumpy:mainfrom huangqinjin:export-no-import
Conversation
|
Just to make sure I understand correctly. You need to share the API table in a shared library, but are using restricted visibility for the API table symbol? |
|
Yes. The shared library is compiled with |
|
I don't have a strong opinion on whether this is useful or the best approach, but it seems fine. Could you look where |
|
The macro names are just follow the existing I have added the document, kindly help polish the language. |
charris
added
01 - Enhancement
56 - Needs Release Note.
|
This will need a release note. |
seberg
left a comment
seberg
left a comment
There was a problem hiding this comment.
I would still much prefer to call it API_ARRAY_ATTRIBUTE or even API_TABLE. The "unique symbol" is a bit underspecific name as well, but I don't feel just because it is, the new name needs to be also.
@rossbar would you have time to smoothen the docs here? The docs could use a little bit of love, but I don't have a great rephrasing idea right now.
| extension module). | ||
|
|
||
| To understand how these macros work together, please refer to | ||
| :c:func:`import_array`. |
There was a problem hiding this comment.
Good idea to just point there, thanks.
| be defined in another compilation unit. | ||
| declared to be ``extern PY_ARRAY_ATTRIBUTE void**``, so it is | ||
| expected to be defined in another compilation unit of the same | ||
| binary if ``PY_ARRAY_ATTRIBUTE`` is not defined. Or it is imported |
There was a problem hiding this comment.
I would suggest making a new bullet point where the old sentence ends:
* When the C-API is defined in another binary with reduced visibility
(e.g. ``-fvisibility=hidden``) the :c:macro:`PY_ARRAY_ATTRIBUTE` can ensure
visibility by defining it to ``__attribute__((visibility("default")))``
or ``__declspec(dllimport)``.
(or similar)
| If the single extension module not only consists of multiple | ||
| compilation units, but also spans multiple binaries, instead of | ||
| invoking ``import_array`` in every binary, you can leverage the | ||
| ``PY_ARRAY_ATTRIBUTE`` macro. Internally, the C-API is actually |
There was a problem hiding this comment.
I feel this should be clear about the hidden visibility. The details can be in the bullet points below.
|
Thanks, I'll close this. We eventually did this with a single attribute and also rewriting the docs. Sorry that this stalled, but it is now available (in 2.1 I think, or will be in 2.2). I will note that I very much discourage doing this. It is a bad idea unless you bundle and control all of the shared libraries in questions in a single application. Otherwise they might have been compiled against different NumPy versions. |
3 participants
Currently, if multiple extension modules share same initialization, and the code is insde a shared library. e.g.
boost::python::numpy::initialize(), we have toimport_array()again for each module:This PR adds two macros:
PY_ARRAY_ATTRIBUTEandPY_UFUNC_ATTRIBUTE. To address the issue above, just add the following macro definitions to every compilation unit of every extension module: