Skip to content

PicklingError on multiprocessing.pool "Can't pickle <class 'zstd.ZstdError' #248

@hub-il

Description

@hub-il

Using the zstandard module to compress and decompress data in a multiprocessing context. If there is a zstd.ZstdError thrown when running in a pool such as:

zstd.ZstdError: decompression error: did not decompress full frame

It results in this error:

multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x74ccabe98bb0>'. Reason: 'PicklingError("Can't pickle <class 'zstd.ZstdError'>: import of module 'zstd' failed")'

Seems that there is something in the zstd.ZstdError class that can't be pickled by the multiprossing.pool.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions