Skip to content

Comments

feat(bindings/go): add benchmark#6341

Merged
Xuanwo merged 1 commit intomainfrom
benchmark-go-binding
Jun 28, 2025
Merged

feat(bindings/go): add benchmark#6341
Xuanwo merged 1 commit intomainfrom
benchmark-go-binding

Conversation

@yuchanns
Copy link
Member

@yuchanns yuchanns commented Jun 28, 2025 •
edited
Loading

  • Add 'bench' target to Go bindings Makefile for running performance benchmarks
  • Enhance benchmark tests to compare OpenDAL performance against AWS S3 SDK
  • Integrate benchmark execution into CI pipeline for S3 service testing (minio only)
  • Update help documentation to include new benchmark target usage examples

Benchmark Result
Diff with benchstat

benchstat aws.txt opendal.txt
goos: linux
goarch: amd64
pkg: opendal_test
cpu: AMD EPYC 7763 64-Core Processor                
             │    aws.txt    │             opendal.txt              │
             │    sec/op     │    sec/op      vs base               │
Write/4KiB      3.096m ± 20%    3.606m ± 71%  +16.47% (p=0.015 n=6)
Write/256KiB    5.969m ± 10%    5.983m ±  5%        ~ (p=1.000 n=6)
Write/4MiB      29.10m ±  1%    18.65m ±  5%  -35.92% (p=0.002 n=6)
Write/16MiB     97.94m ±  1%    53.87m ±  3%  -45.00% (p=0.002 n=6)
Read/4KiB       751.0µ ±  1%   1217.0µ ±  1%  +62.04% (p=0.002 n=6)
Read/256KiB     1.656m ±  1%    1.620m ±  1%   -2.18% (p=0.002 n=6)
Read/4MiB      10.434m ± 13%    4.851m ±  2%  -53.51% (p=0.002 n=6)
Read/16MiB      25.80m ±  5%    14.66m ±  2%  -43.19% (p=0.002 n=6)
geomean         8.050m          6.461m        -19.73%

             │    aws.txt    │             opendal.txt             │
             │     B/op      │     B/op      vs base               │
Write/4KiB      61936.0 ± 2%     384.0 ± 0%  -99.38% (p=0.002 n=6)
Write/256KiB    90581.5 ± 2%     384.0 ± 0%  -99.58% (p=0.002 n=6)
Write/4MiB      92246.5 ± 1%     384.0 ± 0%  -99.58% (p=0.002 n=6)
Write/16MiB     99849.0 ± 3%     384.0 ± 0%  -99.62% (p=0.002 n=6)
Read/4KiB      34.171Ki ± 0%   4.648Ki ± 0%  -86.40% (p=0.002 n=6)
Read/256KiB    1181.5Ki ± 0%   256.7Ki ± 0%  -78.28% (p=0.002 n=6)
Read/4MiB      20.155Mi ± 0%   4.001Mi ± 0%  -80.15% (p=0.002 n=6)
Read/16MiB      97.90Mi ± 0%   16.00Mi ± 0%  -83.66% (p=0.002 n=6)
geomean         500.3Ki        14.12Ki       -97.18%

             │   aws.txt   │            opendal.txt            │
             │  allocs/op  │ allocs/op   vs base               │
Write/4KiB     312.00 ± 0%   10.00 ± 0%  -96.79% (p=0.002 n=6)
Write/256KiB   312.00 ± 0%   10.00 ± 0%  -96.79% (p=0.002 n=6)
Write/4MiB     320.00 ± 0%   10.00 ± 0%  -96.88% (p=0.002 n=6)
Write/16MiB    322.00 ± 0%   10.00 ± 0%  -96.89% (p=0.002 n=6)
Read/4KiB      344.00 ± 0%   15.00 ± 0%  -95.64% (p=0.002 n=6)
Read/256KiB    365.00 ± 0%   15.00 ± 0%  -95.89% (p=0.002 n=6)
Read/4MiB      388.00 ± 0%   15.50 ± 3%  -96.01% (p=0.002 n=6)
Read/16MiB     397.50 ± 0%   16.00 ± 6%  -95.97% (p=0.002 n=6)
geomean         343.6        12.40       -96.39%

Analyzation by Claude-Sonnet-4

Overall Performance

  • Execution time: OpenDAL is 19.73% faster on average
  • Memory usage: OpenDAL uses 97.18% less memory
  • Memory allocations: OpenDAL has 96.39% fewer allocations

Execution Time Analysis

Write Operations:

  • 4KiB files: OpenDAL is 16.47% slower
  • 256KiB files: Performance is equivalent
  • 4MiB files: OpenDAL is 35.92% faster
  • 16MiB files: OpenDAL is 45.00% faster

Read Operations:

  • 4KiB files: OpenDAL is 62.04% slower
  • 256KiB files: OpenDAL is 2.18% faster
  • 4MiB files: OpenDAL is 53.51% faster
  • 16MiB files: OpenDAL is 43.19% faster

Memory Efficiency

Memory Usage (B/op):

  • Write operations: 99.38% - 99.62% reduction across all file sizes
  • Read operations: 78.28% - 86.40% reduction across all file sizes

Memory Allocations (allocs/op):

  • Write operations: 96.79% - 96.89% reduction
  • Read operations: 95.64% - 96.01% reduction

Performance Characteristics

  1. File size correlation: OpenDAL performance advantage increases with file size
  2. Small file overhead: AWS SDK performs better for files ≤4KiB
  3. Memory efficiency: OpenDAL demonstrates consistent memory optimization across all scenarios
  4. Scalability: OpenDAL shows better performance scaling for larger payloads

@yuchanns yuchanns force-pushed the benchmark-go-binding branch 4 times, most recently from 8d1fb17 to b75449f Compare June 28, 2025 10:56
@yuchanns
feat(bindings/go): add comprehensive benchmark testing with Makefile …
ccf0501 
…target and CI integration

- Add 'bench' target to Go bindings Makefile for running performance benchmarks
- Enhance benchmark tests to compare OpenDAL performance against AWS S3 SDK
- Integrate benchmark execution into CI pipeline for S3 service testing
- Update help documentation to include new benchmark target usage examples
@yuchanns yuchanns force-pushed the benchmark-go-binding branch from b75449f to ccf0501 Compare June 28, 2025 11:21
@yuchanns yuchanns changed the title feat(bindings/go): add comprehensive benchmark testing with Makefile … feat(bindings/go): add benchmark Jun 28, 2025
@yuchanns yuchanns marked this pull request as ready for review June 28, 2025 11:26
@yuchanns yuchanns requested a review from Xuanwo as a code owner June 28, 2025 11:26
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jun 28, 2025
@yuchanns
Copy link
Member Author

yuchanns commented Jun 28, 2025

cc @Zheaoli and @Xuanwo

@dosubot dosubot bot added the releases-note/feat The PR implements a new feature or has a title that begins with "feat" label Jun 28, 2025
Xuanwo
Xuanwo approved these changes Jun 28, 2025
View reviewed changes
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for building this!

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jun 28, 2025
@Xuanwo Xuanwo merged commit ef98b85 into main Jun 28, 2025
72 checks passed
@Xuanwo Xuanwo deleted the benchmark-go-binding branch June 28, 2025 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Xuanwo Xuanwo Xuanwo approved these changes

Assignees

No one assigned

Labels

lgtm This PR has been approved by a maintainer releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@yuchanns @Xuanwo