base64 implementation almost 8x slower than GNU for some large files #8574
Description
I benchmarked the uutils version against the GNU version of base64 using hyperfine and noticed that there was a significant difference in performance for some large files.
- OS release: Ubuntu Questing Quokka (development branch) 25.10
- GNU coreutils version:
9.5-1ubuntu2+0.0.0~ubuntu20 - uutils coreutils version:
0.1.0
Reproduce
Generate two large files:
oneline_4G.txt- a single line file consisting of a sequence of A's (4GB in size)
dd if=/dev/zero bs=1M count=4096 status=progress | tr '\0' 'A' |
head -c 4294967295 > oneline_4G.txt && echo >> oneline_4G.txttruncated_4G_file- a truncated file (4GB in size)
truncate -s 4G truncated_4G_fileRun hyperfine on oneline_4G.txt:
hyperfine 'base64 oneline_4G.txt' 'coreutils base64 oneline_4G.txt' -N --warmup 3Output
Benchmark 1: base64 oneline_4G.txt
Time (mean ± σ): 2.461 s ± 0.037 s [User: 2.067 s, System: 0.394 s]
Range (min … max): 2.416 s … 2.525 s 10 runs
Benchmark 2: coreutils base64 oneline_4G.txt
Time (mean ± σ): 19.556 s ± 0.571 s [User: 17.923 s, System: 1.632 s]
Range (min … max): 18.475 s … 20.434 s 10 runs
Summary
base64 oneline_4G.txt ran
7.95 ± 0.26 times faster than coreutils base64 oneline_4G.txtRun hyperfine on truncated_4G_file:
hyperfine 'base64 truncated_4G_file' 'coreutils base64 truncated_4G_file' -N --warmup 3Output
Benchmark 1: base64 truncated_4G_file
Time (mean ± σ): 2.449 s ± 0.036 s [User: 2.044 s, System: 0.405 s]
Range (min … max): 2.405 s … 2.505 s 10 runs
Benchmark 2: coreutils base64 truncated_4G_file
Time (mean ± σ): 19.451 s ± 0.493 s [User: 17.829 s, System: 1.621 s]
Range (min … max): 18.612 s … 20.271 s 10 runs
Summary
base64 truncated_4G_file ran
7.94 ± 0.23 times faster than coreutils base64 truncated_4G_file
Reference Machine
- OS: Ubuntu Questing Quokka (development branch) 25.10
- Hardware Model: Lenovo Legion Slim 5 16IRH8
- Processor: 13th Gen Intel® Core™ i7-13620H × 16
- Memory: DDR5 16.0 GiB @ 5600 MT/s