Repository navigation
BUG: overridable functions from NEP 35 do not show up in ARRAY_FUNCTIONS #27451
Description
Describe the issue:
Functions that were made dispatchable via NEP 35 (which added the like keyword argument) do not appear to be so using the np.testing.overrides API.
Taking for instance np.asarray, whose like argument is documented as
If an array-like passed in as like supports the
__array_function__protocol, the result will be defined by it
Meanwhile, np.testing.overrides.get_overridable_numpy_array_functions's docstring says
List all numpy functions overridable via
__array_function__
Since there's an API to check NEP 18 support, there should probably be one for NEP 35 too. I'm not sure wether the best route would be to extend the existing API to cover NEP 35 functions too, or if another set of functions should be added to np.testing.overrides.
Reproduce the code example:
import numpy as np
assert np.asarray in np.testing.overrides.get_overridable_numpy_array_functions()Error message:
Traceback (most recent call last):
File "/Users/clm/dev/astropy-project/coordinated/astropy/t.py", line 2, in <module>
assert np.asarray in np.testing.overrides.get_overridable_numpy_array_functions()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionErrorPython and NumPy Versions:
2.1.1
3.12.6 (main, Sep 9 2024, 21:36:32) [Clang 18.1.8 ]
Runtime Environment:
[{'numpy_version': '2.1.1',
'python': '3.12.6 (main, Sep 9 2024, 21:36:32) [Clang 18.1.8 ]',
'uname': uname_result(system='Darwin', node='kwanzaabot.home', release='23.6.0', version='Darwin Kernel Version 23.6.0: Wed Jul 31 20:48:52 PDT 2024; root:xnu-10063.141.1.700.5~1/RELEASE_ARM64_T6020', machine='arm64')},
{'simd_extensions': {'baseline': ['NEON', 'NEON_FP16', 'NEON_VFPV4', 'ASIMD'],
'found': ['ASIMDHP'],
'not_found': ['ASIMDFHM']}}]
Context for the issue:
We started discussing supporting a subset of these functions in astropy.units (astropy/astropy#17001), but I haven't found an easy systematic way to discover the whole set (in particular because we're already testing for implementation completeness of NEP 18 functions).
Activity
seberg commentedon Sep 30, 2024
Thanks for the note. This should not be very hard, I think. The
like=argument for (most, I think not all) is handled in C.They are thus slightly different (not wrapped, e.g. also don't have
_implementation).The solution may just be to add them manually (maybe with a decorator), and I suppose also add a test for that. Where the test may be the more interesting part even.
This may not be quite trivial, but should be possible to fiugre out with some pointers, I suspect.
neutrinoceros commentedon Sep 30, 2024
I can give it a try later this week.
neutrinoceros commentedon Oct 2, 2024
Good news, it seems that many such functions are already decorated with
@set_array_function_like_doc(ref gh-16935).Here are all 9 functions with this decorator
requireonesfullfromfunctionidentityloadtxtgenfromtxteyetriAnd here are every other NEP 35 function I could find (reading through gh-16935), which I'm guessing are all C-implemented
arrayasarrayasanyarrayascontiguousarrayasfortranarrayemptyzerosfromstringfromiterfromfilefrombufferarangeconveniently, this second subset still has their docstrings defined in Python and can be discovered by grepping for
(which is essentially an inlining of
@set_array_function_like_doc)I think it's possible to resolve the problem with just some plumbing in that decorator... at least for the Python-implemented part. For the C-implemented part, I'm not sure what to do. Manually writing down the list of 12 functions somewhere is clearly the easy way out, but it's also not easily discoverable or maintainable. Adding a dedicated flag to
numpy.lib.add_newdocwould provide a systematic way to do it but I generally don't like stacking argument to existing functions, even semi-private ones. How about tweakingadd_newdocso it automatically performs thestr.replaceand registers the function upon dedecting the substring"${ARRAY_FUNCTION_LIKE}"?seberg commentedon Oct 2, 2024
Not sure that modifying
add_newdocis worth it, maybe we just add it as another line in the same place? But if you think it is, also fine.neutrinoceros commentedon Oct 3, 2024
@seberg I opened gh-27490 with my initial approach. Let me know what you think !
API: register NEP 35 functions as array_functions (numpy#27490)