SAI validation by opcoder0 · Pull Request #15324 · sonic-net/sonic-mgmt

opcoder0 · 2024-11-01T06:53:54Z

Description of PR

Introduces a new method to access SONiC Redis database instance using gNMI instead of relying on sonic-db-cli.
It interacts with sonic-gnmi using standard gNMI API (get, set, subscribe) to monitor DB changes instead of using the traditional wait_until which performs sleep->check cycle.
Adds changes to perform SAI validation for ACL tests.
Uses this library to monitor STATE_DB changes using the new notification library in test_bfd.py::test_bfd_basic.

Summary:
Fixes # Not applicable

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Back port request

Approach

What is the motivation for this PR?

Motivation is to add a simple and easy to use library that allows tests (where applicable) to perform SAI validation and allow tests to use event notification-based method to monitor DB changes.

How did you do it?

Refer to description above.

How did you verify/test it?

Verified ACL tests (on t0) and BFD basic tests (on t1) testbeds.

Any platform specific information?

No

Supported testbed topology if it's a new test case?

NA

Documentation

TBD.

- Key changes detected

of N unique keys while waiting

mssonicbld · 2025-05-16T06:18:33Z

/azp run

azure-pipelines · 2025-05-16T06:18:46Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-05-20T00:53:18Z

/azp run

azure-pipelines · 2025-05-20T00:53:29Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-05-20T06:32:13Z

/azp run

azure-pipelines · 2025-05-20T06:32:24Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-05-23T01:05:21Z

/azp run

azure-pipelines · 2025-05-23T01:05:32Z

Azure Pipelines successfully started running 1 pipeline(s).

honllum · 2025-05-26T19:53:01Z

Hi @qiluo-msft ,
It seems with this commit, running test will throw away any uncommitted diff (similar to the effect of 'git stash') in local WS.

opcoder0 · 2025-05-27T00:47:36Z

Hi @qiluo-msft , It seems with this commit, running test will throw away any uncommitted diff (similar to the effect of 'git stash') in local WS.

Hi @honllum - I checked by running tests in my local environment from current HEAD (2aafdaf2) on master and found things are working as expected. I didn't see my uncommitted files being removed / stashed. Can you please re-check. If the issue persists, please open an issue with all the details (environment details, test that was executed, logs etc.).

Description of PR Introduces a new method to access SONiC Redis database instance using redis-py native Python library instead of relying on sonic-db-cli. It also uses the Redis' pub-sub infrastructure to listen to key-space notifications to monitor DB changes instead of using the traditional wait_until which performs sleep->check cycle. Adds changes to perform SAI validation for ACL tests. Uses this library to monitor STATE_DB changes using the new notification library in test_bfd.py::test_bfd_basic. Approach What is the motivation for this PR? Motivation is to add a simple and easy to use library that allows tests (where applicable) to perform SAI validation and allow tests to use event notification-based method to monitor DB changes. How did you do it? Refer to description above. How did you verify/test it? Verified ACL tests (on t0) and BFD basic tests (on t1) testbeds.

JibinBao · 2025-07-10T11:06:50Z

Hi @opcoder0 ,
It looks like the PR will cause running ptfhost.shell() and duthost.shell() stuck.
Can you please check the ticket: #19519

Description of PR Introduces a new method to access SONiC Redis database instance using redis-py native Python library instead of relying on sonic-db-cli. It also uses the Redis' pub-sub infrastructure to listen to key-space notifications to monitor DB changes instead of using the traditional wait_until which performs sleep->check cycle. Adds changes to perform SAI validation for ACL tests. Uses this library to monitor STATE_DB changes using the new notification library in test_bfd.py::test_bfd_basic. Approach What is the motivation for this PR? Motivation is to add a simple and easy to use library that allows tests (where applicable) to perform SAI validation and allow tests to use event notification-based method to monitor DB changes. How did you do it? Refer to description above. How did you verify/test it? Verified ACL tests (on t0) and BFD basic tests (on t1) testbeds. Signed-off-by: opcoder0 <[email protected]>

Description of PR Introduces a new method to access SONiC Redis database instance using redis-py native Python library instead of relying on sonic-db-cli. It also uses the Redis' pub-sub infrastructure to listen to key-space notifications to monitor DB changes instead of using the traditional wait_until which performs sleep->check cycle. Adds changes to perform SAI validation for ACL tests. Uses this library to monitor STATE_DB changes using the new notification library in test_bfd.py::test_bfd_basic. Approach What is the motivation for this PR? Motivation is to add a simple and easy to use library that allows tests (where applicable) to perform SAI validation and allow tests to use event notification-based method to monitor DB changes. How did you do it? Refer to description above. How did you verify/test it? Verified ACL tests (on t0) and BFD basic tests (on t1) testbeds. Signed-off-by: Aharon Malkin <[email protected]>

Description of PR Introduces a new method to access SONiC Redis database instance using redis-py native Python library instead of relying on sonic-db-cli. It also uses the Redis' pub-sub infrastructure to listen to key-space notifications to monitor DB changes instead of using the traditional wait_until which performs sleep->check cycle. Adds changes to perform SAI validation for ACL tests. Uses this library to monitor STATE_DB changes using the new notification library in test_bfd.py::test_bfd_basic. Approach What is the motivation for this PR? Motivation is to add a simple and easy to use library that allows tests (where applicable) to perform SAI validation and allow tests to use event notification-based method to monitor DB changes. How did you do it? Refer to description above. How did you verify/test it? Verified ACL tests (on t0) and BFD basic tests (on t1) testbeds. Signed-off-by: Guy Shemesh <[email protected]>

opcoder0 added 23 commits August 22, 2024 06:42

Initial commit with draft design - in progress

96094a0

Remove test category column and fix formatting

30727ce

Add everflow; fix description

abb1b88

Add tests/fdb, tests/fib

65d2e0e

Update design document; Write pubsub based sonic database access library

e0fbbbf

Merge branch 'master' into feat/sai-validation

f4bd2d7

Add logging to test bfd and sonic db library

9e510de

Merge branch 'master' into feat/sai-validation

7b41e62

Add wait_for_n_keys; update usage in bfd and acl

25ab88d

Basic working:

c3624dc

- Key changes detected

Add to design doc; Keep count

7fd5761

of N unique keys while waiting

refactor code to count rules

a3767a0

Reduce logging; Add SAI validation to TestIncrementalAcl

a1a3178

add log to test_bfd.py and make basic test_basic_new

da86d65

Add new subscription based API for watching Sonic DB keys

b2b410f

Merge branch 'master' into feat/sai-validation

c7e1adc

Fix failures; Move changes to test_bfd.py; remove test_bfd_new.py;

d7d0ea8

Add socat files

fc76650

remove test_bfd_new.py

778acb1

Expose redis via db fixtures; Assert ACL test;

e456688

Merge branch 'master' into feat/sai-validation

7a55ec3

Remove vs config files from commit

c51dc9f

Remove extra debug log entry

6cbabd6

opcoder0 requested review from xwjiang-ms and yutongzhang-microsoft November 1, 2024 07:01

opcoder0 added 2 commits November 1, 2024 07:08

Fix merge issues in conftest.py

b1fa4e9

Remove debug log entries from test_acl.py

6548a75

opcoder0 marked this pull request as ready for review November 4, 2024 05:44

opcoder0 requested review from wangxin and yxieca as code owners November 4, 2024 05:44

opcoder0 added 2 commits May 16, 2025 02:39

Merge remote-tracking branch 'upstream/master' into feat/sai-validation

48683cc

Remove _pb2.py files; Add logic to generate them at runtime

8598dc9

qiluo-msft previously approved these changes May 16, 2025

View reviewed changes

Merge branch 'master' into feat/sai-validation

a079448

opcoder0 dismissed qiluo-msft’s stale review via a079448 May 20, 2025 00:53

Merge branch 'master' into feat/sai-validation

a950b7d

wangxin approved these changes May 22, 2025

View reviewed changes

qiluo-msft approved these changes May 22, 2025

View reviewed changes

qiluo-msft enabled auto-merge (squash) May 22, 2025 18:03

Merge branch 'master' into feat/sai-validation

1bf6e00

qiluo-msft merged commit 98a8487 into sonic-net:master May 23, 2025
17 checks passed

AharonMalkin mentioned this pull request May 27, 2025

Add BFD state checker with wait until function instead of a static wait. #18550

Closed

11 tasks

JibinBao mentioned this pull request Jul 10, 2025

Bug: Sometimes running ptfhost.shell() or duthost.shell() will be stuck when running test_bfd_basic #19519

Open

aronovic mentioned this pull request Jul 30, 2025

Bug:test_bfd_basic failure gRPC Error during GetRequest: StatusCode.UNAVAILABLE #19940

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SAI validation#15324

SAI validation#15324
qiluo-msft merged 64 commits intosonic-net:masterfrom
opcoder0:feat/sai-validation

opcoder0 commented Nov 1, 2024 •

edited

Loading

Uh oh!

mssonicbld commented May 16, 2025

Uh oh!

azure-pipelines bot commented May 16, 2025

Uh oh!

mssonicbld commented May 20, 2025

Uh oh!

azure-pipelines bot commented May 20, 2025

Uh oh!

mssonicbld commented May 20, 2025

Uh oh!

azure-pipelines bot commented May 20, 2025

Uh oh!

mssonicbld commented May 23, 2025

Uh oh!

azure-pipelines bot commented May 23, 2025

Uh oh!

Uh oh!

honllum commented May 26, 2025 •

edited

Loading

Uh oh!

opcoder0 commented May 27, 2025

Uh oh!

JibinBao commented Jul 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

opcoder0 commented Nov 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

mssonicbld commented May 16, 2025

Uh oh!

azure-pipelines bot commented May 16, 2025

Uh oh!

mssonicbld commented May 20, 2025

Uh oh!

azure-pipelines bot commented May 20, 2025

Uh oh!

mssonicbld commented May 20, 2025

Uh oh!

azure-pipelines bot commented May 20, 2025

Uh oh!

mssonicbld commented May 23, 2025

Uh oh!

azure-pipelines bot commented May 23, 2025

Uh oh!

Uh oh!

honllum commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

opcoder0 commented May 27, 2025

Uh oh!

JibinBao commented Jul 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

opcoder0 commented Nov 1, 2024 •

edited

Loading

honllum commented May 26, 2025 •

edited

Loading