Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

FlatTo1D in copy Tensor is a problem for huge array! #9304

@starimpact

Description

@starimpact

in ndarray Copy function:

        mshadow::Copy(to->FlatTo1D<cpu, DType>(),
                      from.FlatTo1D<cpu, DType>());

FlatTo1D will cause a int of Shape overflow problem. Because the Shape value type is index_t which range is on int. Sometimes, we need to handle a very big data, which size will exceed the range of int.
So... I suggest the following code:

        mshadow::Copy(to->FlatTo2D<cpu, DType>(),
                      from.FlatTo2D<cpu, DType>());

yes....just convert to 2D shape, we will be able to handle much bigger very big array data.

how about it?
@piiswrong

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions