Fix fixture invocation order in qos_sai_base.py to prevent teardown failure. by vivekverma-arista · Pull Request #17180 · sonic-net/sonic-mgmt

vivekverma-arista · 2025-02-26T15:20:04Z

Description of PR

Summary: Fix fixture invocation order in qos_sai_base.py to prevent teardown failure.
Fixes #460

Type of change

Back port request

Approach

What is the motivation for this PR?

Regression due to: [action] [PR:15501] change swapsyncd to module level. #15938
This PR changes the scope of swapSyncd_on_selected_duts and dut_disable_ipv6 from class to module. This was to reduce execution time in T2. The fixtures used to fire multiple times for chassis.
There is a fixture in global conftest, core_dump_and_config_check which in its teardown restores DUT config to the state as it was prior to the test. https://github.com/sonic-net/sonic-mgmt/blob/202411/tests/conftest.py#L2542
All the three fixtures are currently autouse and module scoped and core_dump_and_config_check and dut_disable_ipv6 share no explicit dependencies hence they will fire in any order.
If the teardown of core_dump_and_config_check happens before dut_disable_ipv6 has restored ipv6 on the DUT, this failure will be seen.

How did you do it?

The proposed fix will be to make swapSyncd_on_selected_duts request core_dump_and_config_check. This way setup of core_dump_and_config_check will happen before qos_sai_base fixtures and its teardown will happen after the teardown of qos_sai_base fixtures.

How did you verify/test it?

Ran qos/test_qos_sai.py with dualtor topology on Arista-7050CX3 and no error was seen.

SKIPPED [3] qos/test_qos_sai.py:1559: multi-dut is not supported on T0 topologies
SKIPPED [6] qos/test_qos_sai.py:1635: multi-dut is not supported on T0 topologies
SKIPPED [3] qos/test_qos_sai.py:1742: multi-dut is not supported on T0 topologies
SKIPPED [6] qos/test_qos_sai.py:1851: multi-dut is not supported on T0 topologies
SKIPPED [3] qos/test_qos_sai.py:1935: multi-dut is not supported on T0 topologies
SKIPPED [6] qos/test_qos_sai.py:2059: multi-dut is not supported on T0 topologies
SKIPPED [3] qos/test_qos_sai.py:2120: multi-dut is not supported on T0 topologies
============================= 15 passed, 220 skipped, 1 warning in 4656.78s (1:17:36) ==============================

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

…ailure.

mssonicbld · 2025-02-26T15:20:07Z

/azp run

azure-pipelines · 2025-02-26T15:20:21Z

Azure Pipelines successfully started running 1 pipeline(s).

StormLiangMS

LGTM

mssonicbld · 2025-02-27T01:27:14Z

@vivekverma-arista PR conflicts with 202311 branch

…ailure. (sonic-net#17180) Approach What is the motivation for this PR? Regression due to: [action] [PR:15501] change swapsyncd to module level. sonic-net#15938 This PR changes the scope of swapSyncd_on_selected_duts and dut_disable_ipv6 from class to module. This was to reduce execution time in T2. The fixtures used to fire multiple times for chassis. There is a fixture in global conftest, core_dump_and_config_check which in its teardown restores DUT config to the state as it was prior to the test. https://github.com/sonic-net/sonic-mgmt/blob/202411/tests/conftest.py#L2542 All the three fixtures are currently autouse and module scoped and core_dump_and_config_check and dut_disable_ipv6 share no explicit dependencies hence they will fire in any order. If the teardown of core_dump_and_config_check happens before dut_disable_ipv6 has restored ipv6 on the DUT, this failure will be seen. How did you do it? The proposed fix will be to make swapSyncd_on_selected_duts request core_dump_and_config_check. This way setup of core_dump_and_config_check will happen before qos_sai_base fixtures and its teardown will happen after the teardown of qos_sai_base fixtures. How did you verify/test it? Ran qos/test_qos_sai.py with dualtor topology on Arista-7050CX3 and no error was seen. SKIPPED [3] qos/test_qos_sai.py:1559: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1635: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1742: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1851: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1935: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:2059: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:2120: multi-dut is not supported on T0 topologies ============================= 15 passed, 220 skipped, 1 warning in 4656.78s (1:17:36) ============================== Any platform specific information? Supported testbed topology if it's a new test case?

mssonicbld · 2025-02-27T01:27:22Z

Cherry-pick PR to 202411: #17198

…ailure. (sonic-net#17180) Approach What is the motivation for this PR? Regression due to: [action] [PR:15501] change swapsyncd to module level. sonic-net#15938 This PR changes the scope of swapSyncd_on_selected_duts and dut_disable_ipv6 from class to module. This was to reduce execution time in T2. The fixtures used to fire multiple times for chassis. There is a fixture in global conftest, core_dump_and_config_check which in its teardown restores DUT config to the state as it was prior to the test. https://github.com/sonic-net/sonic-mgmt/blob/202411/tests/conftest.py#L2542 All the three fixtures are currently autouse and module scoped and core_dump_and_config_check and dut_disable_ipv6 share no explicit dependencies hence they will fire in any order. If the teardown of core_dump_and_config_check happens before dut_disable_ipv6 has restored ipv6 on the DUT, this failure will be seen. How did you do it? The proposed fix will be to make swapSyncd_on_selected_duts request core_dump_and_config_check. This way setup of core_dump_and_config_check will happen before qos_sai_base fixtures and its teardown will happen after the teardown of qos_sai_base fixtures. How did you verify/test it? Ran qos/test_qos_sai.py with dualtor topology on Arista-7050CX3 and no error was seen. SKIPPED [3] qos/test_qos_sai.py:1559: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1635: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1742: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1851: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1935: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:2059: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:2120: multi-dut is not supported on T0 topologies ============================= 15 passed, 220 skipped, 1 warning in 4656.78s (1:17:36) ============================== Any platform specific information? Supported testbed topology if it's a new test case?

mssonicbld · 2025-02-27T01:28:39Z

Cherry-pick PR to 202405: #17199

vivekverma-arista · 2025-02-28T05:27:41Z

@vivekverma-arista PR conflicts with 202311 branch

We don't need this in 202311 as the PR which caused regression was only backported to 202405 and 202411
#15501

…ailure. (sonic-net#17180) Approach What is the motivation for this PR? Regression due to: [action] [PR:15501] change swapsyncd to module level. sonic-net#15938 This PR changes the scope of swapSyncd_on_selected_duts and dut_disable_ipv6 from class to module. This was to reduce execution time in T2. The fixtures used to fire multiple times for chassis. There is a fixture in global conftest, core_dump_and_config_check which in its teardown restores DUT config to the state as it was prior to the test. https://github.com/sonic-net/sonic-mgmt/blob/202411/tests/conftest.py#L2542 All the three fixtures are currently autouse and module scoped and core_dump_and_config_check and dut_disable_ipv6 share no explicit dependencies hence they will fire in any order. If the teardown of core_dump_and_config_check happens before dut_disable_ipv6 has restored ipv6 on the DUT, this failure will be seen. How did you do it? The proposed fix will be to make swapSyncd_on_selected_duts request core_dump_and_config_check. This way setup of core_dump_and_config_check will happen before qos_sai_base fixtures and its teardown will happen after the teardown of qos_sai_base fixtures. How did you verify/test it? Ran qos/test_qos_sai.py with dualtor topology on Arista-7050CX3 and no error was seen. SKIPPED [3] qos/test_qos_sai.py:1559: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1635: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1742: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1851: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1935: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:2059: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:2120: multi-dut is not supported on T0 topologies ============================= 15 passed, 220 skipped, 1 warning in 4656.78s (1:17:36) ============================== Any platform specific information? Supported testbed topology if it's a new test case?

…ailure. (#17180) Approach What is the motivation for this PR? Regression due to: [action] [PR:15501] change swapsyncd to module level. #15938 This PR changes the scope of swapSyncd_on_selected_duts and dut_disable_ipv6 from class to module. This was to reduce execution time in T2. The fixtures used to fire multiple times for chassis. There is a fixture in global conftest, core_dump_and_config_check which in its teardown restores DUT config to the state as it was prior to the test. https://github.com/sonic-net/sonic-mgmt/blob/202411/tests/conftest.py#L2542 All the three fixtures are currently autouse and module scoped and core_dump_and_config_check and dut_disable_ipv6 share no explicit dependencies hence they will fire in any order. If the teardown of core_dump_and_config_check happens before dut_disable_ipv6 has restored ipv6 on the DUT, this failure will be seen. How did you do it? The proposed fix will be to make swapSyncd_on_selected_duts request core_dump_and_config_check. This way setup of core_dump_and_config_check will happen before qos_sai_base fixtures and its teardown will happen after the teardown of qos_sai_base fixtures. How did you verify/test it? Ran qos/test_qos_sai.py with dualtor topology on Arista-7050CX3 and no error was seen. SKIPPED [3] qos/test_qos_sai.py:1559: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1635: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1742: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1851: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1935: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:2059: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:2120: multi-dut is not supported on T0 topologies ============================= 15 passed, 220 skipped, 1 warning in 4656.78s (1:17:36) ============================== Any platform specific information? Supported testbed topology if it's a new test case?

…ailure. (sonic-net#17180) Approach What is the motivation for this PR? Regression due to: [action] [PR:15501] change swapsyncd to module level. sonic-net#15938 This PR changes the scope of swapSyncd_on_selected_duts and dut_disable_ipv6 from class to module. This was to reduce execution time in T2. The fixtures used to fire multiple times for chassis. There is a fixture in global conftest, core_dump_and_config_check which in its teardown restores DUT config to the state as it was prior to the test. https://github.com/sonic-net/sonic-mgmt/blob/202411/tests/conftest.py#L2542 All the three fixtures are currently autouse and module scoped and core_dump_and_config_check and dut_disable_ipv6 share no explicit dependencies hence they will fire in any order. If the teardown of core_dump_and_config_check happens before dut_disable_ipv6 has restored ipv6 on the DUT, this failure will be seen. How did you do it? The proposed fix will be to make swapSyncd_on_selected_duts request core_dump_and_config_check. This way setup of core_dump_and_config_check will happen before qos_sai_base fixtures and its teardown will happen after the teardown of qos_sai_base fixtures. How did you verify/test it? Ran qos/test_qos_sai.py with dualtor topology on Arista-7050CX3 and no error was seen. SKIPPED [3] qos/test_qos_sai.py:1559: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1635: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1742: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1851: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1935: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:2059: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:2120: multi-dut is not supported on T0 topologies ============================= 15 passed, 220 skipped, 1 warning in 4656.78s (1:17:36) ============================== Any platform specific information? Supported testbed topology if it's a new test case?

…ailure. (#17180) Approach What is the motivation for this PR? Regression due to: [action] [PR:15501] change swapsyncd to module level. #15938 This PR changes the scope of swapSyncd_on_selected_duts and dut_disable_ipv6 from class to module. This was to reduce execution time in T2. The fixtures used to fire multiple times for chassis. There is a fixture in global conftest, core_dump_and_config_check which in its teardown restores DUT config to the state as it was prior to the test. https://github.com/sonic-net/sonic-mgmt/blob/202411/tests/conftest.py#L2542 All the three fixtures are currently autouse and module scoped and core_dump_and_config_check and dut_disable_ipv6 share no explicit dependencies hence they will fire in any order. If the teardown of core_dump_and_config_check happens before dut_disable_ipv6 has restored ipv6 on the DUT, this failure will be seen. How did you do it? The proposed fix will be to make swapSyncd_on_selected_duts request core_dump_and_config_check. This way setup of core_dump_and_config_check will happen before qos_sai_base fixtures and its teardown will happen after the teardown of qos_sai_base fixtures. How did you verify/test it? Ran qos/test_qos_sai.py with dualtor topology on Arista-7050CX3 and no error was seen. SKIPPED [3] qos/test_qos_sai.py:1559: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1635: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1742: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:1851: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:1935: multi-dut is not supported on T0 topologies SKIPPED [6] qos/test_qos_sai.py:2059: multi-dut is not supported on T0 topologies SKIPPED [3] qos/test_qos_sai.py:2120: multi-dut is not supported on T0 topologies ============================= 15 passed, 220 skipped, 1 warning in 4656.78s (1:17:36) ============================== Any platform specific information? Supported testbed topology if it's a new test case?

Code sync sonic-net/sonic-mgmt:202411 => 202412 ``` * 1f86dab (HEAD -> code-sync-202412, origin/code-sync-202412) r12f 250610:2314 - Merge remote-tracking branch 'base/202411' into code-sync-202412 |\ | * 2ba104e (base/202411) xwjiang-ms 250610:1604 - [202411] Use ceos 4.32.5M as default ceos image version (sonic-net#18878) | * 5fa5cda Longxiang Lyu 250610:0818 - [202411][dualtor-aa] Add `dualtor_aa` support to `test_nvgre_hash` (sonic-net#18883) | * afecbbf zitingguo-ms 250609:1334 - [Cherry-pick][ACL] Collect all upstream ports and Include service port into upstream neighbors in ACL tests (sonic-net#18847) | * 2e4247b pragnya-arista 250609:0632 - [202411][sonic-mgmt]Fix decap/test_subnet_decap.py::test_vlan_subnet_decap (sonic-net#18778) | * 0bfc7a8 Longxiang Lyu 250606:0943 - [dualtor-io] Fix duplication merge condition (sonic-net#18828) | * 52d3771 Zhaohui Sun 250605:2046 - Restore configuration after vxlan module (sonic-net#18714) | * 3d0922f Yaqiang Zhu 250605:2259 - [202411][pktgen] Skip test_pktgen in m0/mx/m1 (sonic-net#18822) | * 3de20a6 Zhaohui Sun 250516:1325 - Add secondary subnet config for t0 topologies (sonic-net#18399) | * bb3e0f9 Zhaohui Sun 250605:1416 - Xfail test_dir_bcast.py due to known issue on Broadcom platform (sonic-net#18787) | * 158c562 Justin Wong 250604:2058 - Add snmp lldp state check after config_reload (sonic-net#18805) | * e39c891 eyakubch 250605:0415 - bug: added fast reboot into reboot_type check (sonic-net#18551) | * 8c7dd3b Cong Hou 250604:1433 - Remove the skip/xfail for the dualtor_io link failure test (sonic-net#18712) | * 5dbc53d mssonicbld 250604:0952 - [dualtor-io] fix dualtor sniffer start slow issue (sonic-net#18758) (sonic-net#18776) | * 105cdf6 StormLiangMS 250603:0844 - [CRM AVAILABLE] To enhance the crm tests for TD3 and Cisco devices (sonic-net#18733) | * 4251b38 andywongarista 250601:1828 - Add restore_image fixture to test_multi_hop_upgrade_path (sonic-net#18230) (sonic-net#18532) | * 85a55d8 Longxiang Lyu 250528:2313 - [dualtor] Fix `test_orchagent_slb` (sonic-net#18666) | * 95a8764 Vivek Verma 250227:0656 - Fix fixture invocation order in qos_sai_base.py to prevent teardown failure. (sonic-net#17180) | * 9a72265 Justin Wong 250514:1844 - Add PTF parameter for ceos neighbor lacp multiplier (sonic-net#18215) | * cd1375d Longxiang Lyu 250529:1029 - [dualtor-io] Validate and recover active-active setup (sonic-net#18675) | * 763c1b3 Longxiang Lyu 250528:2314 - [dualtor] Fix loganalyzer not exist issue (sonic-net#18674) ```

Fix fixture invocation order in qos_sai_base.py to prevent teardown f…

0a2478e

…ailure.

StormLiangMS requested review from XuChen-MSFT and sdszhang February 27, 2025 01:24

StormLiangMS approved these changes Feb 27, 2025

View reviewed changes

StormLiangMS merged commit 9820ca2 into sonic-net:master Feb 27, 2025

StormLiangMS added Request for 202411 branch Approved for 202311 branch Approved for 202411 branch and removed Approved for 202311 branch labels Feb 27, 2025

mssonicbld added the Cherry Pick Conflict_202311 label Feb 27, 2025

mssonicbld added the Created PR to 202411 branch label Feb 27, 2025

mssonicbld mentioned this pull request Feb 27, 2025

[action] [PR:17180] Fix fixture invocation order in qos_sai_base.py to prevent teardown failure. #17198

Merged

10 tasks

StormLiangMS added Request for 202405 branch Approved for 202405 branch and removed Cherry Pick Conflict_202311 labels Feb 27, 2025

mssonicbld added the Created PR to 202405 branch label Feb 27, 2025

mssonicbld mentioned this pull request Feb 27, 2025

[action] [PR:17180] Fix fixture invocation order in qos_sai_base.py to prevent teardown failure. #17199

Merged

10 tasks

vivekverma-arista deleted the fix-qos-tests branch February 28, 2025 05:44

mssonicbld added Included in 202405 branch and removed Created PR to 202405 branch labels Mar 9, 2025

mssonicbld added Included in 202411 branch and removed Created PR to 202411 branch labels May 29, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix fixture invocation order in qos_sai_base.py to prevent teardown failure.#17180

Fix fixture invocation order in qos_sai_base.py to prevent teardown failure.#17180
StormLiangMS merged 1 commit intosonic-net:masterfrom
vivekverma-arista:fix-qos-tests

vivekverma-arista commented Feb 26, 2025 •

edited

Loading

Uh oh!

mssonicbld commented Feb 26, 2025

Uh oh!

azure-pipelines bot commented Feb 26, 2025

Uh oh!

StormLiangMS left a comment

Uh oh!

mssonicbld commented Feb 27, 2025

Uh oh!

mssonicbld commented Feb 27, 2025

Uh oh!

mssonicbld commented Feb 27, 2025

Uh oh!

vivekverma-arista commented Feb 28, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

vivekverma-arista commented Feb 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

mssonicbld commented Feb 26, 2025

Uh oh!

azure-pipelines bot commented Feb 26, 2025

Uh oh!

StormLiangMS left a comment

Choose a reason for hiding this comment

Uh oh!

mssonicbld commented Feb 27, 2025

Uh oh!

mssonicbld commented Feb 27, 2025

Uh oh!

mssonicbld commented Feb 27, 2025

Uh oh!

vivekverma-arista commented Feb 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vivekverma-arista commented Feb 26, 2025 •

edited

Loading

vivekverma-arista commented Feb 28, 2025 •

edited

Loading