Skip to content

Conversation

@chungquantin
Copy link
Contributor

@chungquantin chungquantin commented Jun 17, 2025

Description

PR introduces a new feature to benchmark multiple pallets and exclude extrinsics from benchmarking. Applying changes made to frame-omni-bencher v0.11.1 in polkadot-sdk: paritytech/polkadot-sdk#8265

  • Add --pallets option to selectively run multiple pallets. In the past we only had --pallet to run a single one.
  • Add --exclude-extrinsics [pallet::extrinsic] to add (Pallet,Extrinsic) tuples that should be excluded.
  • Allow selecting multiple pallets interactively.
  • Allow selecting extrinsics of selected pallets interactively.
  • If multiple pallets selected, we don't allow user to interactively update the weight output because weights can't be generated if multiple pallets selected.

How to test the feature?

Run the below command to test:

pop bench pallet --runtime=tests/runtimes/base_parachain_benchmark.wasm --pallets=pallet_timestamp,pallet_sudo --extrinsic=on_finalize,set,sudo --steps=50 --genesis-builder=runtime
Screenshot 2025-06-24 at 20 57 16 Screenshot 2025-06-24 at 20 57 33

@chungquantin chungquantin marked this pull request as ready for review June 24, 2025 13:53
@chungquantin chungquantin changed the title feat(bench): support benchmarking multiple pallets feat(bench/pallet): support benchmarking multiple pallets Jun 24, 2025
@chungquantin chungquantin requested a review from AlexD10S June 24, 2025 14:13
Copy link
Collaborator

@AlexD10S AlexD10S left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!
Unit tests failing are not related to this PR. I fixed it in #548 once is merge into main you can rebase and merge this one

@chungquantin chungquantin force-pushed the chungquantin/feat-multiple_pallets_benchmarking branch from e072fe9 to 4fb13c0 Compare June 26, 2025 02:24
@chungquantin chungquantin requested a review from AlexD10S June 26, 2025 02:24
Copy link
Collaborator

@AlexD10S AlexD10S left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codecov
Copy link

codecov bot commented Jun 30, 2025

Codecov Report

Attention: Patch coverage is 83.67347% with 48 lines in your changes missing coverage. Please review.

Project coverage is 79.36%. Comparing base (92b9d02) to head (1030a4d).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
crates/pop-cli/src/commands/bench/pallet.rs 83.67% 21 Missing and 27 partials ⚠️
@@            Coverage Diff             @@
##             main     #547      +/-   ##
==========================================
+ Coverage   79.34%   79.36%   +0.01%     
==========================================
  Files         106      106              
  Lines       25828    25979     +151     
  Branches    25828    25979     +151     
==========================================
+ Hits        20494    20619     +125     
- Misses       3047     3073      +26     
  Partials     2287     2287              
Files with missing lines Coverage Δ
crates/pop-cli/src/common/bench.rs 83.54% <ø> (ø)
crates/pop-cli/src/commands/bench/pallet.rs 80.23% <83.67%> (+0.52%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chungquantin chungquantin requested a review from AlexD10S June 30, 2025 10:52
Copy link
Collaborator

@AlexD10S AlexD10S left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thanks for fixing it

@chungquantin chungquantin merged commit 0adf9c1 into main Jun 30, 2025
37 of 38 checks passed
@chungquantin chungquantin deleted the chungquantin/feat-multiple_pallets_benchmarking branch June 30, 2025 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants