@@ -6,7 +6,7 @@ NEP 31 — Context-local and global overrides of the NumPy API
66:Author: Ralf Gommers <
[email protected] >
77:Status: Draft
88:Type: Standards Track
9- :Created: 2019-07-31
9+ :Created: 2019-08-22
1010
1111
1212Abstract
@@ -16,19 +16,27 @@ This NEP proposes to make all of NumPy's public API overridable via a backend
1616mechanism, using a library called ``uarray `` `[1] `_
1717
1818``uarray `` provides global and context-local overrides, as well as a dispatch
19- mechanism similar to NEP-18 `[2] `_. This NEP proposes to supercede NEP-18,
20- and is intended as a comprehensive resolution to NEP-22 `[3] `_.
19+ mechanism similar to NEP-18 `[2] `_. First experiences with ``__array_function__ ``
20+ show that it is necessary to be able to override NumPy functions that
21+ *do not take an array-like argument *, and hence aren't overridable via
22+ ``__array_function__ ``. The most pressing need is array creation and coercion
23+ functions - see e.g. NEP-30 `[9] `_.
24+
25+ This NEP proposes to allow, in an opt-in fashion, overriding any part of the NumPy API.
26+ It is intended as a comprehensive resolution to NEP-22 `[3] `_, and obviates the need to
27+ add an ever-growing list of new protocols for each new type of function or object that needs
28+ to become overridable.
2129
2230Motivation and Scope
2331--------------------
2432
25- The motivation behind this library is manifold : First, there have been several attempts to allow
33+ The motivation behind `` uarray `` is manyfold : First, there have been several attempts to allow
2634dispatch of parts of the NumPy API, including (most prominently), the ``__array_ufunc__ `` protocol
2735in NEP-13 `[4] `_, and the ``__array_function__ `` protocol in NEP-18 `[2] `_, but this has shown the
28- need for further protocols to be developed, including a protocol for coercion. `[5] `_. The reasons
36+ need for further protocols to be developed, including a protocol for coercion (see `[5] `_) . The reasons
2937these overrides are needed have been extensively discussed in the references, and this NEP will not
3038attempt to go into the details of why these are needed. Another pain point requiring yet another
31- protocol is the duck-array protocol. `[9] `_
39+ protocol is the duck-array protocol (see `[9] `_).
3240
3341This NEP takes a more holistic approach: It assumes that there are parts of the API that need to be
3442overridable, and that these will grow over time. It provides a general framework and a mechanism to
@@ -39,41 +47,51 @@ functions that can be easily expressed in terms of others, as well as a reposito
3947that help in the implementation of duck-arrays that most duck-arrays would require.
4048
4149The third is the existence of actual, third party dtype packages, and
42- their desire to blend into the NumPy ecosystem. `[6] `_. This is a separate
50+ their desire to blend into the NumPy ecosystem (see `[6] `_) . This is a separate
4351issue compared to the C-level dtype redesign proposed in `[7] `_, it's about
4452allowing third-party dtype implementations to work with NumPy, much like third-party array
4553implementations.
4654
4755This NEP proposes the following: That ``unumpy `` `[8] `_ becomes the recommended override mechanism
48- for the NumPy API.
56+ for the parts of the NumPy API not yet covered by ``__array_function__ `` or ``__array_ufunc__ ``,
57+ and that ``uarray `` is vendored into a new namespace within NumPy to give users and downstream dependencies
58+ access to these overrides. This vendoring mechanism is similar to what SciPy decided to do for
59+ making ``scipy.fft `` overridable (see `[10] `_).
4960
5061
5162Detailed description
5263--------------------
5364
54- This section will not attempt to explain the specifics or the mechanism of ``uarray ``,
55- that is explained in the ``uarray `` documentation. `[1] `_ However, the NumPy community
56- will have input into the design of ``uarray ``, and any backward-incompatible changes
57- will be discussed on the mailing list.
65+ _Note that this section will not attempt to explain the specifics or the mechanism of ``uarray ``,_
66+ _that is explained in the ``uarray `` documentation. `[1] `_ However, the NumPy community _
67+ _will have input into the design of ``uarray ``, and any backward-incompatible changes _
68+ _will be discussed on the mailing list._
69+
70+ The way we propose the overrides will be used by end users is::
71+
72+ from numpy import unumpy
73+ TODO
5874
59- The first goal of this NEP is as follows: To complete an overridable version of NumPy,
60- called `` unumpy `` ` [8] `_, the implementation of which is already underway. Again, `` unumpy ``
61- will not be explained here, the reader should refer to its documentation for this purpose.
75+ And a library that implements a NumPy-like API will use it like::
76+
77+ TODO: example corresponding to NEP 30 `duckarray`
6278
6379The only change this NEP proposes at its acceptance, is to make ``unumpy `` the officially recommended
64- way to override NumPy. ``unumpy `` will remain a separate repository/package, and will be developed
80+ way to override NumPy. ``unumpy `` will remain a separate repository/package (which we propose to vendor
81+ to avoid a hard dependency, and use the separate ``unumpy `` package only if it is installed)
82+ rather than depend on for the time being), and will be developed
6583primarily with the input of duck-array authors and secondarily, custom dtype authors, via the usual
6684GitHub workflow. There are a few reasons for this:
6785
6886* Faster iteration in the case of bugs or issues.
6987* Faster design changes, in the case of needed functionality.
70- * Lower likelihood to be stuck with a bad design decision .
88+ * `` unumpy `` will work with older versions of NumPy as well .
7189* The user and library author opt-in to the override process,
7290 rather than breakages happening when it is least expected.
7391 In simple terms, bugs in ``unumpy `` mean that ``numpy `` remains
7492 unaffected.
75- * The upgrade pathway to NumPy 2.0 becomes simpler, requiring just
76- a backend change, and allowing both to exist side-by-side.
93+
94+ ** FIXME: this section doesn't match the proposal. in the abstract and motivation anymore. **
7795
7896Once maturity is achieved, ``unumpy `` be moved into the NumPy organization,
7997and NumPy will become the reference implementation for ``unumpy ``.
@@ -150,8 +168,13 @@ There are no backward incompatible changes proposed in this NEP.
150168Alternatives
151169------------
152170
153- The current alternative to this problem, already implemented, is a
154- combination of NEP-18 and NEP-13.
171+ The current alternative to this problem is NEP-30 plus adding more protocols
172+ (not yet specified) in addition to it. Even then, some parts of the NumPy
173+ API will remain non-overridable, so it's a partial alternative.
174+
175+ The main alternative to vendoring ``unumpy `` is to simply move it into NumPy
176+ completely and not distribute it as a separate package. This would also achieve
177+ the proposed goals, however we prefer to keep it a separate package for now.
155178
156179
157180Discussion
@@ -204,6 +227,11 @@ References and Footnotes
204227
205228[9] NEP 30 — Duck Typing for NumPy Arrays - Implementation: https://www.numpy.org/neps/nep-0030-duck-array-protocol.html
206229
230+ .. _[10] :
231+
232+ [10] http://scipy.github.io/devdocs/fft.html#backend-control
233+
234+
207235Copyright
208236---------
209237
0 commit comments