Skip to content

zstd decompression using "--patch-from" cost memory too high (when the dict is a big file) #3324

@rongekuta

Description

@rongekuta

Is your feature request related to a problem? Please describe.

zstd -d -M128MB --patch-from=d:/code-shared/apks/wzry_old.apk d:/code-shared/apks/wzry.diff -o d:/code-shared/apks/wzry_new_d.apk

when i use zstd (--patch-from) to decompress a patch file using a big dict, i find "-M#" option is not valid, the big dict is load to memory all, and cost too many memory. i search code for reason, and find:
https://github.com/facebook/zstd/blob/v1.5.2/programs/fileio.c#L2019

these code cause "-M" memory limit is not work under "--patch-from" mode

Describe the solution you'd like
decompression under a big dict with "--patch-from" is also be limited memory

Describe alternatives you've considered
the big dict also be cache from file system, not load to memory totally

Additional context

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions