-
-
Notifications
You must be signed in to change notification settings - Fork 1k
Support sparse matrices with more than 2^31-1 elements #3513
Copy link
Copy link
Open
Labels
Description
Currently there are a number of places in the codebase where the dtype for indptr and indices are hardcoded to 'i' (i.e., int32), for example:
cupy/cupyx/scipy/sparse/compressed.py
Lines 330 to 331 in 22e270f
| indices = cupy.array(x.indices, dtype='i') | |
| indptr = cupy.array(x.indptr, dtype='i') |
But they shouldn't be. In SciPy, they use a helper function
get_index_dtype() to determine this:https://github.com/scipy/scipy/blob/f4b5605031f738bc87ae4e193d614d525b98ffba/scipy/sparse/compressed.py#L65-L71
I think this change shouldn't be hard, but it is worth a standalone PR with sufficient test coverage added.
For the short term, I'd say let's encourage PR authors to take this into account, so we can make the changes gradually. In particular, when a new ElementwiseKernel or ReductionKernel is added, we just need to use template types T as oppose to hardcoded int32.
Reactions are currently unavailable