Skip to content

decompress to block device using -o is broken #3583

@georgmu

Description

@georgmu

Describe the bug

When decompressing with "-o" to a block device, the output does not match the source file

To Reproduce
The following script shows the problem:

#!/bin/bash

# via this command, it is possible to specify the zstd binary to use, defaulting to zstd from PATH
ZSTD=${1:-zstd}

# generate our testfile
echo 123456789 > testfile
truncate -s 1M testfile

rm -f testfile.zst
"${ZSTD}" testfile -o testfile.zst

# generate a target file and loop-mount it
dd if=/dev/urandom of=target bs=1M count=1 2>/dev/null

LOOPDEV="$(losetup -f)"
losetup "$LOOPDEV" target

# these are the same, all other sha256 outputs should be the same as well
echo "expected sums:"
sha256sum testfile
"${ZSTD}" -d < testfile.zst | sha256sum

echo

"${ZSTD}" -d testfile.zst -o "$LOOPDEV"
# this one fails
echo
echo "using -o:"
sha256sum "$LOOPDEV"

echo
hexdump -C target | head -n 5
echo
"${ZSTD}" -d < testfile.zst > "$LOOPDEV"
# now it is correct:
echo "using pipe:"
sha256sum "$LOOPDEV"

echo
hexdump -C target | head -n 5

losetup -d "$LOOPDEV"

Sample output:

testfile             :  0.01%   ( 1.000 MiB =>     63 B, testfile.zst)         
expected sums:
a29e830fc0eede27937b8cb0d7336c2c3fcdeedd7c6207923c3e31dec0541537  testfile
a29e830fc0eede27937b8cb0d7336c2c3fcdeedd7c6207923c3e31dec0541537  -

testfile.zst        : 1048576 bytes                                            

using -o:
e074889058caa6266c163e27d1c5990d3810b39927616732102667876c7cefc1  /dev/loop0

00000000  31 32 33 34 35 36 37 38  39 0a 00 00 00 00 00 00  |123456789.......|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00008000  63 4e 0f 16 5c e4 09 2b  a8 0f 3a 55 1e 14 52 a5  |cN..\..+..:U..R.|
00008010  ed 86 59 95 d2 b1 66 93  a4 36 2d 2c 0e 97 b0 a7  |..Y...f..6-,....|

using pipe:
a29e830fc0eede27937b8cb0d7336c2c3fcdeedd7c6207923c3e31dec0541537  /dev/loop0

00000000  31 32 33 34 35 36 37 38  39 0a 00 00 00 00 00 00  |123456789.......|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00100000

Expected behavior
-o should generate the same output as piping to a device

Desktop (please complete the following information):

  • OS: Linux, Fedaora 38, Debian 11
  • tested against dev branch, releases 1.4.8 (debian) and 1.5.4 (fedora)

Additional context
The problem seems to be seeks in the output. This is fine for files, which are generated (seek will generate zeroes in the output file). For block devices, seek must not be used (because it keeps the file contents as is instead of zeroing them as expected)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions