Skip to content

Support sparse matrices with more than 2^31-1 elements #3513

@leofang

Description

@leofang

Currently there are a number of places in the codebase where the dtype for indptr and indices are hardcoded to 'i' (i.e., int32), for example:

indices = cupy.array(x.indices, dtype='i')
indptr = cupy.array(x.indptr, dtype='i')

But they shouldn't be. In SciPy, they use a helper function get_index_dtype() to determine this:
https://github.com/scipy/scipy/blob/f4b5605031f738bc87ae4e193d614d525b98ffba/scipy/sparse/compressed.py#L65-L71
I think this change shouldn't be hard, but it is worth a standalone PR with sufficient test coverage added.

For the short term, I'd say let's encourage PR authors to take this into account, so we can make the changes gradually. In particular, when a new ElementwiseKernel or ReductionKernel is added, we just need to use template types T as oppose to hardcoded int32.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions