-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Closed
Description
Describe the bug
When decompressing with "-o" to a block device, the output does not match the source file
To Reproduce
The following script shows the problem:
#!/bin/bash
# via this command, it is possible to specify the zstd binary to use, defaulting to zstd from PATH
ZSTD=${1:-zstd}
# generate our testfile
echo 123456789 > testfile
truncate -s 1M testfile
rm -f testfile.zst
"${ZSTD}" testfile -o testfile.zst
# generate a target file and loop-mount it
dd if=/dev/urandom of=target bs=1M count=1 2>/dev/null
LOOPDEV="$(losetup -f)"
losetup "$LOOPDEV" target
# these are the same, all other sha256 outputs should be the same as well
echo "expected sums:"
sha256sum testfile
"${ZSTD}" -d < testfile.zst | sha256sum
echo
"${ZSTD}" -d testfile.zst -o "$LOOPDEV"
# this one fails
echo
echo "using -o:"
sha256sum "$LOOPDEV"
echo
hexdump -C target | head -n 5
echo
"${ZSTD}" -d < testfile.zst > "$LOOPDEV"
# now it is correct:
echo "using pipe:"
sha256sum "$LOOPDEV"
echo
hexdump -C target | head -n 5
losetup -d "$LOOPDEV"
Sample output:
testfile : 0.01% ( 1.000 MiB => 63 B, testfile.zst)
expected sums:
a29e830fc0eede27937b8cb0d7336c2c3fcdeedd7c6207923c3e31dec0541537 testfile
a29e830fc0eede27937b8cb0d7336c2c3fcdeedd7c6207923c3e31dec0541537 -
testfile.zst : 1048576 bytes
using -o:
e074889058caa6266c163e27d1c5990d3810b39927616732102667876c7cefc1 /dev/loop0
00000000 31 32 33 34 35 36 37 38 39 0a 00 00 00 00 00 00 |123456789.......|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00008000 63 4e 0f 16 5c e4 09 2b a8 0f 3a 55 1e 14 52 a5 |cN..\..+..:U..R.|
00008010 ed 86 59 95 d2 b1 66 93 a4 36 2d 2c 0e 97 b0 a7 |..Y...f..6-,....|
using pipe:
a29e830fc0eede27937b8cb0d7336c2c3fcdeedd7c6207923c3e31dec0541537 /dev/loop0
00000000 31 32 33 34 35 36 37 38 39 0a 00 00 00 00 00 00 |123456789.......|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00100000
Expected behavior
-o should generate the same output as piping to a device
Desktop (please complete the following information):
- OS: Linux, Fedaora 38, Debian 11
- tested against dev branch, releases 1.4.8 (debian) and 1.5.4 (fedora)
Additional context
The problem seems to be seeks in the output. This is fine for files, which are generated (seek will generate zeroes in the output file). For block devices, seek must not be used (because it keeps the file contents as is instead of zeroing them as expected)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels