Skip to content

Passthrough on short files fails #3210

@cgbur

Description

@cgbur

Describe the bug
When trying to use the decompression passed through functionality on files of 0, 1, or 2 bytes the decompressor fails to read enough bytes to understand the context and quits instead of just copying the bytes to the output. This means if you have small files and are trying to use zstdcat for all of them, the very small files will fail and not be written to the output.

To Reproduce

echo | zstd -df
echo 1 | zstd -df
echo 12 | zstd -df
echo 123 | zstd -df
echo | zstdcat
echo 1 | zstdcat
echo 12 | zstdcat
echo 123 | zstdcat

output

zstd: /*stdin*\: unknown header 
zstd: /*stdin*\: unknown header 
zstd: /*stdin*\: unknown header 
123

Expected behavior
Zstd and gzip support decompression passthrough where the bytes will simply be copied to the output as-is when the following conditions are met:

  1. if a file is present that is not in a known format
  2. the -f flag is present
  3. the output will be written to stdout

example

echo hello there | zstd -df
hello there

gzip on the failure cases produces the correct output

echo | gzip -df
echo 1 | gzip -df
echo 12 | gzip -df
echo 123 | gzip -df

1
12
123

Screenshots and charts
Linux x86 - self installed from repo

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions