Skip to content

Block Checksum calculation is incorrect #223

@scunningham

Description

@scunningham

When block checksum flag is enabled, the library calculates the xx32 hash of the original source data and appends the little endian as a block trailer.

https://github.com/pierrec/lz4/blob/v4/internal/lz4stream/block.go#L249

Per the spec, the hash should be calculated on the compressed block, not the src uncompressed data.

Block checksum flag

If this flag is set, each data block will be followed by a 4-bytes checksum, calculated by using the xxHash-32 algorithm on the raw (compressed) data block. The intention is to detect data corruption (storage or transmission errors) immediately, before decoding. Block checksum usage is optional.

As expected, the hash check fails on a file generated by this library with the option enabled:

$ lz4cat /tmp/busted.lz4
Error 34 : Decompression error : ERROR_blockChecksum_invalid

Note that the bug is masked if a block is not compressed and the high bit is set on the frame size. In that case, hashing the source and the compressed output are equivalent.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions