Skip to content

refactor(load): simulator contract#4181

Merged
maru-ava merged 11 commits intomasterfrom
refactor-load-simulator
Aug 28, 2025
Merged

refactor(load): simulator contract#4181
maru-ava merged 11 commits intomasterfrom
refactor-load-simulator

Conversation

@RodrigoVillar
Copy link
Copy Markdown
Contributor

@RodrigoVillar RodrigoVillar commented Aug 18, 2025 •
edited
Loading

Why this should be merged

While researching past transactions on the C-Chain (for the month of July 2025), the following trends became clear:

  • Most C-Chain transactions were utilizing more gas than that of the load tests
  • Most C-Chain transactions were larger in size (via calldata length) than that of the load tests
  • The majority of block processing time was spent on trie operations rather than block execution

While updating the load simulator contract to account for the trends above, I identified the following issues:

  1. Several of the methods in the load simulator contract are not relevant in the context of load testing. For example, simulateMemory(), while not computationally free, does not produce the type of load relevant for projects like Firewood (i.e. no database r/w operations).
  2. The methods of the simulator contract do not have documentation on their gas-usage/calldata lengths. This makes it difficult to create load tests of different profiles (e.g. load tests that target Firewood vs load tests that target SAE).

This PR replaces the EVMSimulatorContract contract with LoadSimulator, a contract with computationally intensive methods targeted towards load testing. These methods are also documented in terms of their gas-usage and calldata lengths, making it possible to create load tests of different profiles in future PRs.

How this works

  • Replaces EVMLoadSimulator with LoadSimulator contract
  • Adjusted test weights in NewRandomTest() to reflect the trends mentioned above
  • Updates load/README.md to account for changes

How this was tested

CI + ran load test locally

Need to be documented in RELEASES.md?

No

@RodrigoVillar RodrigoVillar self-assigned this Aug 18, 2025
@RodrigoVillar RodrigoVillar added the testing This primarily focuses on testing label Aug 18, 2025
RodrigoVillar
RodrigoVillar commented Aug 19, 2025
View reviewed changes
Comment thread tests/load/tests.go
@RodrigoVillar RodrigoVillar marked this pull request as ready for review August 19, 2025 13:48
Copilot AI review requested due to automatic review settings August 19, 2025 13:48
Copilot AI reviewed Aug 19, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the load testing infrastructure by replacing the EVMLoadSimulator contract with a new LoadSimulator contract designed for more realistic and better-documented load testing scenarios. The changes align load tests with observed C-Chain usage patterns from July 2025, focusing on computationally intensive operations with documented gas costs and calldata lengths.

  • Replaces old simulator contract with streamlined, documented methods
  • Adjusts test weights to reflect real C-Chain transaction frequency and computational intensity
  • Updates documentation to reflect the new test suite composition

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/load/tests.go Updates test definitions and weights based on C-Chain analysis
tests/load/contracts/LoadSimulator.sol New contract with documented gas costs for targeted load testing
tests/load/contracts/LoadSimulator.bindings.go Generated Go bindings for the new LoadSimulator contract
tests/load/contracts/EVMLoadSimulator.sol Removes old simulator contract
tests/load/contracts/EVMLoadSimulator.bindings.go Removes old contract bindings
tests/load/README.md Updates documentation to reflect new test types and weightings

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread tests/load/tests.go
Comment thread tests/load/tests.go
maru-ava
maru-ava reviewed Aug 26, 2025
View reviewed changes
Comment thread tests/load/contracts/LoadSimulator.sol
Comment thread tests/load/contracts/LoadSimulator.sol Outdated
Comment thread tests/load/contracts/LoadSimulator.sol Outdated
Comment thread tests/load/contracts/LoadSimulator.sol Outdated
Comment thread tests/load/tests.go
Comment thread tests/load/tests.go

weightedTests := []WeightedTest{
{
// minimum gas used: 21_000
Copy link
Copy Markdown
Contributor

@maru-ava maru-ava Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(No action required) Any value in exporting the expected minimum gas used as a test metric for comparative purposes (i.e. being able to eyeball that the actual is always greater than equal to the expected minimum)?

Copy link
Copy Markdown
Contributor

@maru-ava maru-ava Aug 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I say no action is required, but I'm asking a question, I would like to see an answer before resolution of the comment.

Comment thread tests/load/contracts/LoadSimulator.sol Outdated
Comment thread tests/load/tests.go Outdated
@RodrigoVillar RodrigoVillar requested a review from maru-ava August 27, 2025 18:50
@maru-ava maru-ava added this pull request to the merge queue Aug 28, 2025
Merged via the queue into master with commit 40fbcd5 Aug 28, 2025
36 checks passed
@maru-ava maru-ava deleted the refactor-load-simulator branch August 28, 2025 04:40
@github-project-automation github-project-automation bot moved this to Done 🎉 in avalanchego Aug 28, 2025
joshua-kim added a commit that referenced this pull request Sep 3, 2025
@joshua-kim
Squashed commit of the following:
74d60c2 
commit 66ca7dc
Author: rodrigo <[email protected]>
Date:   Tue Sep 2 16:49:55 2025 -0400

    feat(load): add firewood flag (#4235)

commit 274541b
Author: Suyan Qu <[email protected]>
Date:   Tue Sep 2 14:10:48 2025 -0500

    feat: add parameters to disable metrics (#4214)

commit e03af84
Author: aaronbuchwald <[email protected]>
Date:   Tue Sep 2 12:30:07 2025 -0400

    Add timeout parameter to C-Chain re-execution jobs (#4223)

commit 0e20485
Author: aaronbuchwald <[email protected]>
Date:   Tue Sep 2 11:58:29 2025 -0400

    Comment out schedule trigger for re-execution on w/container (#4234)

    Signed-off-by: aaronbuchwald <[email protected]>
    Co-authored-by: Copilot <[email protected]>

commit 847eba1
Author: aaronbuchwald <[email protected]>
Date:   Fri Aug 29 14:38:11 2025 -0400

    Add back empty schedule entry for reexecute w/ container job (#4230)

commit a958b8a
Author: aaronbuchwald <[email protected]>
Date:   Fri Aug 29 12:56:06 2025 -0400

    Add newline to workflow dispatch (#4229)

    Signed-off-by: aaronbuchwald <[email protected]>

commit 7ec2258
Author: aaronbuchwald <[email protected]>
Date:   Thu Aug 28 17:11:38 2025 -0400

    Push benchmark re-execute results on master workflow dispatch (#4224)

commit 34f983e
Author: aaronbuchwald <[email protected]>
Date:   Thu Aug 28 15:33:12 2025 -0400

    Disambiguate source vs exec variable names in reexecute tasks (#4200)

    Signed-off-by: aaronbuchwald <[email protected]>
    Co-authored-by: Copilot <[email protected]>

commit 99578a2
Author: aaronbuchwald <[email protected]>
Date:   Thu Aug 28 12:52:31 2025 -0400

    Write grafana link to logs and github step summary (#4219)

commit 814300c
Author: aaronbuchwald <[email protected]>
Date:   Thu Aug 28 12:37:05 2025 -0400

    Remove firewood entry from PR triggers due to flakes (#4227)

commit 40fbcd5
Author: rodrigo <[email protected]>
Date:   Thu Aug 28 00:24:54 2025 -0400

    refactor(load): simulator contract (#4181)

commit 6195e1f
Author: rodrigo <[email protected]>
Date:   Wed Aug 27 17:31:51 2025 -0400

    chore: remove unzip mention (#4226)

commit 59e88f3
Author: aaronbuchwald <[email protected]>
Date:   Wed Aug 27 11:17:27 2025 -0400

    Remove schedule trigger for w/ container job that evaluates to empty matrix (#4218)

commit c2563d1
Author: Stephen Buttolph <[email protected]>
Date:   Tue Aug 26 19:07:47 2025 -0400

    Update versions for v1.13.5 (#4217)

commit a0ccd66
Author: aaronbuchwald <[email protected]>
Date:   Tue Aug 26 12:34:54 2025 -0400

    Add support for passing config and predefined configs to VM re-execution tests (#4180)

commit cc3242f
Author: Joshua Kim <[email protected]>
Date:   Mon Aug 25 18:49:28 2025 -0400

    Dynamically update mempool gossip request rate limit (#4162)

    Signed-off-by: Joshua Kim <[email protected]>
    Co-authored-by: Stephen Buttolph <[email protected]>

commit f2e3273
Author: Draco <[email protected]>
Date:   Mon Aug 25 15:00:56 2025 -0400

    Add ability to create zstd compressor with compression level (#4203)

commit 441f441
Author: Joshua Kim <[email protected]>
Date:   Mon Aug 25 12:11:07 2025 -0400

    Remove buf lint action (#4189)

    Signed-off-by: Joshua Kim <[email protected]>

commit 4bcb221
Author: Stephen Buttolph <[email protected]>
Date:   Sat Aug 23 15:43:06 2025 -0400

    Update block + validator + pgo checkpoints to 2025-08-23 (#4205)

commit b18ffc1
Author: rodrigo <[email protected]>
Date:   Fri Aug 22 16:34:53 2025 -0400

    Add s5cmd progress bar (#4204)

commit 2100bee
Author: Sam Liokumovich <[email protected]>
Date:   Fri Aug 22 11:52:31 2025 -0400

    Rename Engine Types (#4193)

    Signed-off-by: Sam Liokumovich <[email protected]>
    Co-authored-by: Copilot <[email protected]>

commit 33727a8
Author: Joshua Kim <[email protected]>
Date:   Fri Aug 22 11:00:12 2025 -0400

    Count throttled requests as hits (#4199)

    Signed-off-by: Joshua Kim <[email protected]>

commit b939be4
Author: Draco <[email protected]>
Date:   Thu Aug 21 14:22:54 2025 -0400

    fix: blockdb file eviction race issue (#4186)

commit 778ccfe
Author: aaronbuchwald <[email protected]>
Date:   Thu Aug 21 11:40:03 2025 -0400

    Add config option for AWS S3 read only credential duration (#4192)

commit ae41355
Author: Stephen Buttolph <[email protected]>
Date:   Wed Aug 20 16:46:49 2025 -0400

    Add redundant import alias linting (#4191)

    Signed-off-by: Stephen Buttolph <[email protected]>
    Co-authored-by: Copilot <[email protected]>

commit a3b5c6a
Author: Stephen Buttolph <[email protected]>
Date:   Wed Aug 20 10:53:24 2025 -0400

    Make Draco the codeowner of the blockdb (#4187)

commit a24ac68
Author: queryfast <[email protected]>
Date:   Wed Aug 20 22:18:21 2025 +0800

    refactor: replace []byte(fmt.Sprintf) with fmt.Appendf (#4161)

    Signed-off-by: queryfast <[email protected]>

commit 7aa6a17
Author: Sam Liokumovich <[email protected]>
Date:   Tue Aug 19 14:39:40 2025 -0400

    Rename height field to numBlocks (#4184)

commit 7d7e1fe
Author: aaronbuchwald <[email protected]>
Date:   Tue Aug 19 13:24:59 2025 -0400

    Add optional step to archive post-reexecution state to S3 (#4172)

    Signed-off-by: aaronbuchwald <[email protected]>
    Co-authored-by: Copilot <[email protected]>

commit ebe0558
Author: aaronbuchwald <[email protected]>
Date:   Tue Aug 19 12:11:34 2025 -0400

    Change cache path to tmp included in gitignore (#4183)

commit e5593be
Author: Draco <[email protected]>
Date:   Tue Aug 19 12:01:43 2025 -0400

    Block Database (#4027)

commit 940b96f
Author: Sam Liokumovich <[email protected]>
Date:   Tue Aug 19 11:36:37 2025 -0400

    Storage Component For Simplex (#4122)

    Signed-off-by: Sam Liokumovich <[email protected]>

commit 6d7e2dc
Author: Nicolas Arnedo Villanueva <[email protected]>
Date:   Tue Aug 19 16:59:58 2025 +0200

    `config/config.md:` Added Env Variable representation of flags + improved UI design (#4110)

    Signed-off-by: Meaghan FitzGerald <[email protected]>
    Signed-off-by: Nicolas Arnedo Villanueva <[email protected]>
    Co-authored-by: Meaghan FitzGerald <[email protected]>
    Co-authored-by: Stephen Buttolph <[email protected]>

commit 81f13b2
Author: Draco <[email protected]>
Date:   Mon Aug 18 13:59:43 2025 -0400

    feat: add eviction callback in LRU cache (#4088)

commit 4f5acfc
Author: Jonathan Oppenheimer <[email protected]>
Date:   Mon Aug 18 13:16:44 2025 -0400

    Migrate predicate package from evm repos (#4147)

    Signed-off-by: Jonathan Oppenheimer <[email protected]>
    Co-authored-by: Copilot <[email protected]>
    Co-authored-by: Stephen Buttolph <[email protected]>
    Co-authored-by: Joshua Kim <[email protected]>

commit 335e79f
Author: Kendra Karol Sevilla <[email protected]>
Date:   Mon Aug 18 18:45:52 2025 +0200

    chore: fix typo (#4179)

    Signed-off-by: Kendra Karol Sevilla <[email protected]>

commit 7275b02
Author: yinwenyu6 <[email protected]>
Date:   Mon Aug 18 22:29:03 2025 +0800

    chore: fix function name (#4178)

    Signed-off-by: yinwenyu6 <[email protected]>

commit 3b0c595
Author: yacovm <[email protected]>
Date:   Mon Aug 18 16:28:29 2025 +0200

    Fix typo in comment - PChainHeight context (#4176)

    Signed-off-by: Yacov Manevich <[email protected]>

commit 96f30d1
Author: rodrigo <[email protected]>
Date:   Fri Aug 15 02:15:44 2025 -0400

    feat(load): add token test (#4171)

commit e285ce0
Author: Sam Liokumovich <[email protected]>
Date:   Thu Aug 14 13:52:41 2025 -0400

    Use EmptyVoteMetadata in Simplex Proto Messages (#4174)

commit 5c72544
Author: aaronbuchwald <[email protected]>
Date:   Wed Aug 13 10:34:58 2025 -0400

    Move C-Chain benchmark to custom action and add ARC + GH runner triggers (#4165)

commit 3b0f8d4
Author: rodrigo <[email protected]>
Date:   Tue Aug 5 20:14:38 2025 -0400

    refactor(load): remove context from test interface (#4157)

commit a893a61
Author: Juan Leon <[email protected]>
Date:   Tue Aug 5 14:36:59 2025 -0400

    Add @joshua-kim as CODEOWNER to testing-related packages (#4118)

    Signed-off-by: Juan Leon <[email protected]>

commit be28a8b
Author: Galoretka <[email protected]>
Date:   Mon Aug 4 22:39:41 2025 +0300

    chore: fix a typo in gossip,go (#4154)

    Signed-off-by: Galoretka <[email protected]>

commit b876d78
Author: aaronbuchwald <[email protected]>
Date:   Mon Aug 4 11:58:22 2025 -0400

    Separate re-execution job params for PR from schedule (#4151)

commit 752e12f
Author: Stephen Buttolph <[email protected]>
Date:   Fri Aug 1 16:23:01 2025 -0400

    Update coreth to v0.15.3-rc.5 (#4153)

commit 3ba5246
Author: Joshua Kim <[email protected]>
Date:   Fri Aug 1 14:59:24 2025 -0400

    fix metrics tests (#4146)

    Signed-off-by: Joshua Kim <[email protected]>

commit 0cb887b
Author: Afounso Souza <[email protected]>
Date:   Fri Aug 1 16:37:53 2025 +0200

    Typo fix (#4150)

    Signed-off-by: Afounso Souza <[email protected]>

commit 110807a
Author: rodrigo <[email protected]>
Date:   Thu Jul 31 22:06:40 2025 -0400

    docs: load (#4132)

commit 24a051a
Author: Jonathan Oppenheimer <[email protected]>
Date:   Thu Jul 31 19:06:15 2025 -0400

    uplift: Add combined metrics package from evm repositories (#4135)

    Signed-off-by: Jonathan Oppenheimer <[email protected]>
    Co-authored-by: Stephen Buttolph <[email protected]>

commit d9b512e
Author: rodrigo <[email protected]>
Date:   Thu Jul 31 11:52:39 2025 -0400

    Parameterize values in transfer tests (#4144)

commit 6947e4c
Author: rodrigo <[email protected]>
Date:   Wed Jul 30 12:27:45 2025 -0400

    feat(load): add trie stress test (#4137)

Signed-off-by: Joshua Kim <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@maru-ava maru-ava maru-ava approved these changes

@joshua-kim joshua-kim Awaiting requested review from joshua-kim

+1 more reviewer

@Elvis339 Elvis339 Elvis339 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@RodrigoVillar RodrigoVillar

Labels

testing This primarily focuses on testing

Projects

avalanchego
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@RodrigoVillar @Elvis339 @maru-ava