[performance_meter] Add performance_meter tests#16842
Merged
Blueve merged 14 commits intosonic-net:masterfrom Feb 25, 2025
Merged
[performance_meter] Add performance_meter tests#16842Blueve merged 14 commits intosonic-net:masterfrom
Blueve merged 14 commits intosonic-net:masterfrom
Conversation
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Blueve
reviewed
Feb 10, 2025
| # Timing will only start after the first blocking part of operation | ||
| # is over. The op should make sure op is started correctly and ended | ||
| # correctly. If either part is unsuccessful, op should yeild False and | ||
| # log the error, otherwise yielding True is expected. |
Collaborator
There was a problem hiding this comment.
In short, the op defines the procedure:
- OP - Do something
- OP - yield or return
- Start timer and success criteria checker
- OP - Do something that we want to measure
Is this statement right?
Collaborator
|
Can you also link the HLD to this PR's description? |
Blueve
reviewed
Feb 10, 2025
Blueve
reviewed
Feb 10, 2025
Blueve
reviewed
Feb 10, 2025
| # take all results that passed op precheck. | ||
|
|
||
|
|
||
| def random_success_20_perc(duthost, **kwarg): |
Collaborator
There was a problem hiding this comment.
This is a sample implementation, right?
If user use this one, their test will got ~20% passing rate
Blueve
reviewed
Feb 10, 2025
| # process results of all runs, so there could be a success criteria | ||
| # stats function, named with a "_stats" suffix, taking the same | ||
| # variables as its single run version, like "bgp_up_stats". It will | ||
| # take all results that passed op precheck. |
Collaborator
There was a problem hiding this comment.
The _stats is special, I saw the example below:
def random_success_20_perc_stats(passed_op_precheck, **kwarg):
Can you explain why it need **kwarg?
And can you extend the sample method to output the actual success rate? Then it could be more clear.
Contributor
Author
There was a problem hiding this comment.
If we define a mean, or a p99, stuff like that, they will be passed to function through the kwarg
Blueve
reviewed
Feb 10, 2025
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Blueve
reviewed
Feb 25, 2025
| ############################ | ||
| parser.addoption("--dpu-pattern", action="store", default="all", help="dpu host name") | ||
|
|
||
| ################################# |
Collaborator
There was a problem hiding this comment.
Make this configurable in next PR
Blueve
approved these changes
Feb 25, 2025
nissampa
pushed a commit
to nissampa/sonic-mgmt_dpu_test
that referenced
this pull request
Mar 4, 2025
What is the motivation for this PR? To better understand the performance of certain operation on different hwsku, and make sure there are no unexpected performance changes. How did you do it? Add new test. User can write config file and code will perform and measure the performance of different operations. How did you verify/test it? Run test on different testbeds.
nnelluri-cisco
pushed a commit
to nnelluri-cisco/sonic-mgmt
that referenced
this pull request
Mar 15, 2025
What is the motivation for this PR? To better understand the performance of certain operation on different hwsku, and make sure there are no unexpected performance changes. How did you do it? Add new test. User can write config file and code will perform and measure the performance of different operations. How did you verify/test it? Run test on different testbeds.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Add performance meter tests
hld: #15356
Summary:
Fixes # (issue)
Type of change
Back port request
Approach
What is the motivation for this PR?
To better understand the performance of certain operation on different hwsku, and make sure there are no unexpected performance changes.
How did you do it?
Add new test. User can write config file and code will perform and measure the performance of different operations.
How did you verify/test it?
Run test on different testbeds.
Any platform specific information?
Supported testbed topology if it's a new test case?
Documentation