cksum/hashsum: merge digest computation & various improvements #9135

RenjiSann · 2025-11-03T23:48:49Z

This PR adds several improvements:

Introduce the AlgoKind enum to stop relying on string comparison for checking algorithm behaviors
Introduce the SizedAlgokind enum to simply represent an algorithm with a given size.
cksum sanitization for BLAKE2b's length
Most importantly: Merging the digest computation for cksum and hashsum, so we don't have different implementations anymore. The only difference now happens in cli interpretation, but the underlying is now the same on both sides

codspeed-hq · 2025-11-04T00:08:22Z

CodSpeed Performance Report

Merging #9135 will improve performances by ×510

_{Comparing RenjiSann:cksum-reworks (6d6a991) with main (b0f41e7)}

Summary

⚡ 6 improvements
✅ 120 untouched
⏩ 6 skipped¹

Benchmarks breakdown

	Benchmark	`BASE`	`HEAD`	Change
⚡	`cksum_blake3`	95,717.6 µs	187.2 µs	×510
⚡	`cksum_crc`	17.7 ms	17.2 ms	+2.99%
⚡	`cksum_crc32b`	14.1 ms	13.6 ms	+3.65%
⚡	`cksum_default`	17.7 ms	17.2 ms	+2.88%
⚡	`cksum_multiple_files`	26.5 ms	25.8 ms	+2.83%
⚡	`cksum_raw_output`	17.7 ms	17.2 ms	+2.85%

6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

github-actions · 2025-11-04T00:12:38Z

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)

github-actions · 2025-11-04T01:59:13Z

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)

github-actions · 2025-11-06T16:03:19Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

github-actions · 2025-11-06T19:40:21Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

sylvestre

Windows builds are failing

RenjiSann · 2025-11-09T15:27:18Z

Windows builds are failing

Yes, this is being discussed in #9168, I will move the PR back to draft until it's fixed

github-actions · 2025-11-09T15:47:28Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

github-actions · 2025-11-12T16:04:29Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

github-actions · 2025-11-14T14:26:54Z

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

github-actions · 2025-11-14T15:01:31Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

github-actions · 2025-11-14T17:08:16Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

github-actions · 2025-11-18T01:37:33Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)

github-actions · 2025-11-18T06:47:14Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)

…anup

sylvestre · 2025-11-18T10:42:10Z

src/uucore/src/lib/features/checksum/mod.rs

+            (algo @ (ak::Sha2 | ak::Sha3), None) => {
+                Err(ChecksumError::LengthRequiredForSha(algo.to_lowercase().into()).into())
+            }
+            // [`calculate_blake2b_length`] expects a length in bits but we


Comment says 'expects a length in bits' but the multiplication by 8 suggests the input is in bytes ?

Yes, l is a length in bytes and we multiply it to give the length in bits to the function

src/uucore/src/lib/features/checksum/validate.rs

sylvestre · 2025-11-18T10:43:57Z

src/uucore/src/lib/features/checksum/validate.rs

+    let mut digest = algo.create_digest();
    let (calculated_checksum, _) =
-        digest_reader(&mut digest, &mut file_reader, opts.binary, algo.bits).unwrap();
+        digest_reader(&mut digest, &mut file_reader, false, algo.bitlen()).unwrap();


maybe add a comment why it is now "false" ? :)
(maybe we should also improve the function profile not to use a bool)

maybe we should also improve the function profile not to use a bool

I plan on doing that at some point, but I reserve that to a later patch, once #9168 is properly determined

src/uucore/src/lib/features/checksum/compute.rs

sylvestre · 2025-11-18T10:46:02Z

tests/by-util/test_hashsum.rs


+// This test is currently disabled on windows
 #[test]
+#[cfg_attr(windows, ignore = "Discussion is in #9168")]


very interesting issue

sylvestre · 2025-11-18T10:47:24Z

src/uu/hashsum/src/hashsum.rs


-        let opts = ChecksumOptions {
-            binary,
+        let opts = ChecksumValidateOptions {


maybe we could remove it for validate_opts ?

Not sure I understand this comment

github-actions · 2025-11-18T10:49:55Z

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)

github-actions · 2025-11-18T17:03:12Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

github-actions · 2025-11-18T17:38:35Z

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)

github-actions · 2025-11-19T13:32:41Z

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)

RenjiSann force-pushed the cksum-reworks branch 2 times, most recently from db8fb3c to 36c93eb Compare November 4, 2025 01:33

RenjiSann force-pushed the cksum-reworks branch from 36c93eb to e449a04 Compare November 6, 2025 14:25

RenjiSann changed the title ~~cksum: Various improvements~~ cksum/hashsum: merge digest computation & various improvements Nov 6, 2025

RenjiSann marked this pull request as ready for review November 6, 2025 17:20

RenjiSann requested a review from sylvestre November 6, 2025 17:20

RenjiSann force-pushed the cksum-reworks branch from 85549e7 to cc69557 Compare November 6, 2025 17:28

sylvestre requested changes Nov 8, 2025

View reviewed changes

RenjiSann marked this pull request as draft November 9, 2025 15:27

RenjiSann force-pushed the cksum-reworks branch from 57a9317 to 274749c Compare November 9, 2025 15:27

RenjiSann force-pushed the cksum-reworks branch from 274749c to 6375a7a Compare November 12, 2025 15:36

RenjiSann force-pushed the cksum-reworks branch from 6375a7a to 7828fb1 Compare November 14, 2025 14:04

RenjiSann force-pushed the cksum-reworks branch from 7828fb1 to 6e191dd Compare November 14, 2025 14:39

RenjiSann force-pushed the cksum-reworks branch from 6e191dd to 1c9a7af Compare November 14, 2025 16:34

RenjiSann marked this pull request as ready for review November 14, 2025 23:04

RenjiSann force-pushed the cksum-reworks branch from 1c9a7af to 6e6b545 Compare November 18, 2025 01:18

RenjiSann requested a review from sylvestre November 18, 2025 01:18

sylvestre force-pushed the cksum-reworks branch from 6e6b545 to b4671a1 Compare November 18, 2025 06:26

RenjiSann added 6 commits November 18, 2025 11:30

test(cksum): Add tests for BLAKE2b --length sanitization

8c9bb16

util(cksum): Fix BLAKE2b --length sanitization

885082c

checksum: Introduce AlgoKind enum to rely less on string comparison

9b6dc96

checksum: Introduce SizedAlgoKind to improve representation and cle…

413e71d

…anup

checksum: Always get bit length from the SizedAlgoKind

923baae

checksum: Get rid of detect_algo and child functions

68dc6fd

RenjiSann force-pushed the cksum-reworks branch from b4671a1 to 5956f67 Compare November 18, 2025 10:30

sylvestre reviewed Nov 18, 2025

View reviewed changes

src/uucore/src/lib/features/checksum/validate.rs Outdated Show resolved Hide resolved

sylvestre reviewed Nov 18, 2025

View reviewed changes

src/uucore/src/lib/features/checksum/compute.rs Outdated Show resolved Hide resolved

sylvestre reviewed Nov 18, 2025

View reviewed changes

RenjiSann force-pushed the cksum-reworks branch from 5956f67 to 876c21c Compare November 18, 2025 16:44

checksum: Create a sub-module for checksum validation

0b72893

RenjiSann force-pushed the cksum-reworks branch from 876c21c to 3ce7c77 Compare November 18, 2025 17:19

RenjiSann added 3 commits November 19, 2025 13:37

checksum: Move cksum computation to uucore::checksum

3849d91

checksum: Adapt checksum computation to hashsum

c97ce1b

checksum: fix binary flag on windows, ignore test

6d6a991

RenjiSann force-pushed the cksum-reworks branch from 3ce7c77 to 6d6a991 Compare November 19, 2025 13:13

RenjiSann mentioned this pull request Nov 20, 2025

cksum: Fix GNU test cksum-base64-untagged #9344

Merged

sylvestre merged commit 643a72c into uutils:main Nov 20, 2025
127 checks passed

RenjiSann deleted the cksum-reworks branch November 20, 2025 09:55

moonfruit mentioned this pull request Dec 15, 2025

uutils-selected 0.5.0 moonfruit/homebrew-tap#387

Closed

Uh oh!

cksum/hashsum: merge digest computation & various improvements #9135

cksum/hashsum: merge digest computation & various improvements #9135

Uh oh!

Conversation

RenjiSann commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed-hq bot commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging #9135 will improve performances by ×510

Summary

Benchmarks breakdown

Footnotes

Uh oh!

github-actions bot commented Nov 4, 2025

Uh oh!

github-actions bot commented Nov 4, 2025

Uh oh!

github-actions bot commented Nov 6, 2025

Uh oh!

github-actions bot commented Nov 6, 2025

Uh oh!

sylvestre left a comment

Choose a reason for hiding this comment

Uh oh!

RenjiSann commented Nov 9, 2025

Uh oh!

github-actions bot commented Nov 9, 2025

Uh oh!

github-actions bot commented Nov 12, 2025

Uh oh!

github-actions bot commented Nov 14, 2025

Uh oh!

github-actions bot commented Nov 14, 2025

Uh oh!

github-actions bot commented Nov 14, 2025

Uh oh!

github-actions bot commented Nov 18, 2025

Uh oh!

github-actions bot commented Nov 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 18, 2025

Uh oh!

github-actions bot commented Nov 18, 2025

Uh oh!

github-actions bot commented Nov 18, 2025

Uh oh!

github-actions bot commented Nov 19, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

RenjiSann commented Nov 3, 2025 •

edited

Loading

codspeed-hq bot commented Nov 4, 2025 •

edited

Loading