The SAI port id of the uplink interface is not correct in test test_tunnel_qos_remap.py. The affected test cases are test_tunnel_decap_dscp_to_pg_mapping and test_xoff_for_pcbb. These two cases fails due to the wrong ids.
Steps to reproduce the issue:
The failures of the two test cases test_tunnel_decap_dscp_to_pg_mapping/test_xoff_for_pcbb share the same root cause, I'll take test_tunnel_decap_dscp_to_pg_mapping as the example.
- Run test case test_tunnel_decap_dscp_to_pg_mapping on a dualtor testbed.
Describe the results you received:
RunAnsibleModuleFail: run module shell failed, Ansible Results =>
{"changed": true, "cmd": ["/root/env-python3/bin/ptf", "--test-dir", "saitests/py3", "sai_qos_tests.TunnelDscpToPgMapping", "--platform-dir", "ptftests", "--platform", "remote", "-t", "cell_size=256;standby_tor_ip=u'10.1.0.33';inner_dscp_to_pg_map={0: 0, 1: 0, 2: 0, 3: 2, 4: 6, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0, 12: 0, 13: 0, 14: 0, 15: 0, 16: 0, 17: 0, 18: 0, 19: 0, 20: 0, 21: 0, 22: 0, 23: 0, 24: 0, 25: 0, 26: 0, 27: 0, 28: 0, 29: 0, 30: 0, 31: 0, 32: 0, 33: 0, 34: 0, 35: 0, 36: 0, 37: 0, 38: 0, 39: 0, 40: 0, 41: 0, 42: 0, 43: 0, 44: 0, 45: 0, 46: 0, 47: 0, 48: 0, 49: 0, 50: 0, 51: 0, 52: 0, 53: 0, 54: 0, 55: 0, 56: 0, 57: 0, 58: 0, 59: 0, 60: 0, 61: 0, 62: 0, 63: 0};src_port_id=34;dst_port_ip=u'192.168.0.3';active_tor_ip=u'10.1.0.32';standby_tor_mac=u'1c:34:da:cf:56:00';server=u'10.210.24.183';sonic_asic_type=u'mellanox';active_tor_mac=u'1c:34:da:c9:60:00';dst_port_id=2;port_map_file='/root/default_interface_to_front_map.ini'", "--disable-ipv6", "--disable-vxlan", "--disable-geneve", "--disable-erspan", "--disable-mpls", "--disable-nvgre", "--log-file", "/tmp/sai_qos_tests.TunnelDscpToPgMapping.log", "--test-case-timeout", "600"], "delta": "0:00:29.862991", "end": "2023-01-16 03:32:04.999542", "failed": true, "msg": "non-zero return code", "rc": 1, "start": "2023-01-16 03:31:35.136551", "stderr": "/root/env-python3/bin/ptf:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n import imp\nsai_qos_tests.TunnelDscpToPgMapping ... FAIL\n\n
======================================================================\n
FAIL: sai_qos_tests.TunnelDscpToPgMapping\n
----------------------------------------------------------------------\n
Traceback (most recent call last):\n
File \"saitests/py3/sai_qos_tests.py\", line 799, in runTest\n
assert(pg_shared_wm_res[pg] - pg_shared_wm_res_base[pg] >= (PKT_NUM - ERROR_TOLERANCE[pg]) * cell_size)\nAssertionError\n\n
----------------------------------------------------------------------\n
Ran 1 test in 28.467s\n\n
FAILED (failures=1)", "stderr_lines": ["/root/env-python3/bin/ptf:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses", " import imp",
"sai_qos_tests.TunnelDscpToPgMapping ... FAIL", "",
"======================================================================",
"FAIL: sai_qos_tests.TunnelDscpToPgMapping",
"----------------------------------------------------------------------",
"Traceback (most recent call last):",
" File \"saitests/py3/sai_qos_tests.py\", line 799, in runTest",
" assert(pg_shared_wm_res[pg] - pg_shared_wm_res_base[pg] >= (PKT_NUM - ERROR_TOLERANCE[pg]) * cell_size)",
"AssertionError", "",
"----------------------------------------------------------------------",
"Ran 1 test in 28.467s", "",
"FAILED (failures=1)"],
"stdout":
"Using packet manipulation module: ptf.packet_scapy\n\n
******************************************\n
ATTENTION: SOME TESTS DID NOT PASS!!!\n\n
The following tests failed:\n
TunnelDscpToPgMapping\n\n
******************************************",
"stdout_lines": ["Using packet manipulation module: ptf.packet_scapy",
"", "******************************************",
"ATTENTION: SOME TESTS DID NOT PASS!!!", "",
"The following tests failed:", "TunnelDscpToPgMapping", "",
"******************************************"]}
The failure is because the check for pg watermark counter failed:
assert(pg_shared_wm_res[pg] - pg_shared_wm_res_base[pg] >= (PKT_NUM - ERROR_TOLERANCE[pg]) * cell_size)"
And the root cause is the src_port_id(the id of the uplink interface used in the test) used by SAI thrift to get the SAI logical port number is not correct.
This test case uses a port_map_file(/root/default_interface_to_front_map.ini in ptf docker) to map the interface names to ptf port ids in the ptftest sai_qos_tests.TunnelDscpToPgMapping.
test_params.update({
"src_port_id": dut_config["lag_port_ptf_id"],
"dst_port_id": dut_config["server_port_ptf_id"],
"dst_port_ip": dut_config["server_ip"],
"active_tor_mac": dut_config["selected_tor_mac"],
"active_tor_ip": dut_config["selected_tor_loopback"],
"standby_tor_mac": dut_config["unselected_tor_mac"],
"standby_tor_ip": dut_config["unselected_tor_loopback"],
"server": dut_config["selected_tor_mgmt"],
"inner_dscp_to_pg_map": tunnel_qos_map["inner_dscp_to_pg_map"],
"port_map_file": dut_config["port_map_file"], #This port_map_file is defined in fixture ptf_portmap_file_module in tests/common/fixtures/ptfhost_utils.py
"sonic_asic_type": dut_config["asic_type"],
"cell_size": cell_size
})
root@b6eedd8e39dd:~# cat /root/default_interface_to_front_map.ini
# ptf host interface @ switch front port name
2@Ethernet8
0@Ethernet0
1@Ethernet4
62@Ethernet248
61@Ethernet244
41@Ethernet164
60@Ethernet240
50@Ethernet200
34@Ethernet124
51@Ethernet204
32@Ethernet120
52@Ethernet208
40@Ethernet160
32@Ethernet128
19@Ethernet76
18@Ethernet72
8@Ethernet32
9@Ethernet36
6@Ethernet24
34@Ethernet136
33@Ethernet132
58@Ethernet232
43@Ethernet172
59@Ethernet236
44@Ethernet176
7@Ethernet28
15@Ethernet60
5@Ethernet20
17@Ethernet68
45@Ethernet180
54@Ethernet216
46@Ethernet184
47@Ethernet188
27@Ethernet108
25@Ethernet100
53@Ethernet212
26@Ethernet104
57@Ethernet228
24@Ethernet96
37@Ethernet148
23@Ethernet92
16@Ethernet64
55@Ethernet220
36@Ethernet144
13@Ethernet52
56@Ethernet224
35@Ethernet140
14@Ethernet56
4@Ethernet16
3@Ethernet12
49@Ethernet196
48@Ethernet192
42@Ethernet168
30@Ethernet116
20@Ethernet80
28@Ethernet112
21@Ethernet84
38@Ethernet152
39@Ethernet156
12@Ethernet48
11@Ethernet44
10@Ethernet40
22@Ethernet88
63@Ethernet252
And in a dualtor testbed, the ptf port ids of the uplink interfaces are [28, 30, 32, 34] for upper tor and [29, 31, 33, 35] for lower tor.
Let's say, for example, the uplink port used in the test case is ptf port 34 which is the Ethernet 124 of upper tor. The id 34 will be used in the ptf test sai_qos_tests.TunnelDscpToPgMapping as the ptf port id and port id for SAI thrift to query the PG watermark counter.
Which means this id 34 will be used by SAI thrift to get the SAI logical port id in ansible/roles/test/files/saitests/py3/switch.py:
for interface, front in interface_to_front_mapping.items():
sai_port_id = client.sai_thrift_get_port_id_by_front_port(front) # Here the front is the id 34
port_list[int(interface)] = sai_port_id
But because in the dualtor testbed the ptf port ids are modified to allow both tors uplinks have different ids, the sai port id got by id 34 is not the sai port id of Ethernet124 but the id of Ethernet136. So the pg counters is not correct and the case will fail.
These two cases never passed on our dualtor testbed due to this port id issue.
Describe the results you expected:
There two cases should pass on dualtor testbeds.
Additional information you deem important:
**Output of `show version`:**
```
(paste your output here)
```
**Attach debug file `sudo generate_dump`:**
```
(paste your output here)
```
The SAI port id of the uplink interface is not correct in test test_tunnel_qos_remap.py. The affected test cases are test_tunnel_decap_dscp_to_pg_mapping and test_xoff_for_pcbb. These two cases fails due to the wrong ids.
Steps to reproduce the issue:
The failures of the two test cases test_tunnel_decap_dscp_to_pg_mapping/test_xoff_for_pcbb share the same root cause, I'll take test_tunnel_decap_dscp_to_pg_mapping as the example.
Describe the results you received:
The failure is because the check for pg watermark counter failed:
And the root cause is the src_port_id(the id of the uplink interface used in the test) used by SAI thrift to get the SAI logical port number is not correct.
This test case uses a port_map_file(/root/default_interface_to_front_map.ini in ptf docker) to map the interface names to ptf port ids in the ptftest sai_qos_tests.TunnelDscpToPgMapping.
And in a dualtor testbed, the ptf port ids of the uplink interfaces are [28, 30, 32, 34] for upper tor and [29, 31, 33, 35] for lower tor.
Let's say, for example, the uplink port used in the test case is ptf port 34 which is the Ethernet 124 of upper tor. The id 34 will be used in the ptf test sai_qos_tests.TunnelDscpToPgMapping as the ptf port id and port id for SAI thrift to query the PG watermark counter.
Which means this id 34 will be used by SAI thrift to get the SAI logical port id in ansible/roles/test/files/saitests/py3/switch.py:
But because in the dualtor testbed the ptf port ids are modified to allow both tors uplinks have different ids, the sai port id got by id 34 is not the sai port id of Ethernet124 but the id of Ethernet136. So the pg counters is not correct and the case will fail.
These two cases never passed on our dualtor testbed due to this port id issue.
Describe the results you expected:
There two cases should pass on dualtor testbeds.
Additional information you deem important: