Skip to content

Wrapper subclass serialization should preserve all _make_wrapper_subclass() args #125622

@jbschlosser

Description

@jbschlosser

Related: #125583 pointed out that dtype was not being preserved for wrapper subclasses through multiprocessing serialization (or serialization in general).

When implementing the dtype fix, I realized that there are other args not being passed to _make_wrapper_subclass() when rebuilding subclass instances after serialization:

pytorch/torch/_utils.py

Lines 352 to 363 in 3827810

def _rebuild_wrapper_subclass(
cls, dtype, size, stride, storage_offset, layout, device, requires_grad
):
return torch.Tensor._make_wrapper_subclass( # type: ignore[attr-defined]
cls,
size,
strides=stride,
storage_offset=storage_offset,
layout=layout,
device=device,
requires_grad=requires_grad,
)

Signature:

_make_wrapper_subclass(
    PyObject* cls,
    SymIntArrayRef size,
    SymIntArrayRef? strides=None,
    SymInt? storage_offset=None,
    MemoryFormat? memory_format=None,  # not preserved
    ScalarType dtype=None,
    Layout layout=torch.strided,
    Device device=None,
    bool pin_memory=False,  # not preserved
    bool requires_grad=False,
    c10::string_view? dispatch_sizes_strides_policy=None,  # not preserved
    bool dispatch_device=False,  # not preserved
    bool dispatch_layout=False,  # not preserved
    DispatchKeySet _extra_dispatch_keys=None  # not preserved
)

Of those not passed, pin_memory() status can be queried by is_pinned(), but the others are not directly queryable to my knowledge. For full correctness, all these args should be preserved through serialization. @albanD has pointed out that this requires tricky changes to the serialization format.

cc @mruberry @mikaylagawarecki @msaroufim @ezyang @albanD

Metadata

Metadata

Assignees

No one assigned

    Labels

    module: serializationIssues related to serialization (e.g., via pickle, or otherwise) of PyTorch objectstensor subclassRelated to tensor subclassestriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions