-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Description
Current blockwise_meta/blockwise implementation defaults to dtype construction of Dask.Array when an non-trivial exception is raised when computing _meta. This is suboptimal for two reasons:
- It's difficult to trace that the defaulting happened within
blockwisefor somewhat complex user code; - Exceptions that come from the array compute task (with NumPy, CuPy, etc.) are not raised to the user during graph construction.
Ideally, we would handle all possible exceptions, with all remaining ones assumed to be user's mistake, such as passing casting to cupy.einsum (which isn't supported), as discussed in #4914.
Given that _meta now exists, Dask has become a universal scheduler, which probably means we can't handle every single possible exception, but we should at least allow developers to know about such cases, as well as letting users know they possibly made a mistake. We should therefore let most exceptions pass back to the user in blockwise.
This is loosely related to #4951, where we wish to allow users/developers be alerted of conditions that Dask can't handle and should probably be better handled by the library which Dask is using as a backend through __array_function__.