Skip to content

Bug: test_snmp_interfaces fails - KeyError: 'name' #21792

@mkim-upscaleai

Description

@mkim-upscaleai

Is it platform specific

generic

Importance or Severity

Critical

Description of the bug

Summary

test_snmp_interfaces intermittently fails with a missing 'name' field for the VLAN interface returned by SNMP. This appears to be a race condition where SNMP responds before all MIB data is fully populated.

Issue

The current wait=True logic in get_snmp_facts() only waits for the subagent to start but does not guarantee that all MIB fields are ready. Even though the subagent is running, the VLAN MIB fields may need slightly more time before querying.

sonic-mgmt/tests/common/helpers/snmp_helpers.py

def get_snmp_facts(duthost, localhost, host, version, community, is_dell=False, module_ignore_errors=False,
                   wait=False, include_swap=False, timeout=DEF_WAIT_TIMEOUT, interval=DEF_CHECK_INTERVAL,
                   snmp_timeout=SNMP_DEFAULT_TIMEOUT):
    if not wait:
        return _get_snmp_facts(localhost, host, version, community, is_dell, include_swap, module_ignore_errors,
                               timeout=snmp_timeout)

    global global_snmp_facts

    pytest_assert(wait_until(timeout, interval, 0, _update_snmp_facts, localhost, host, version,
                             community, is_dell, include_swap, duthost, snmp_timeout), "Timeout waiting for SNMP facts")
    return global_snmp_facts

A similar issue was reported for test_snmp_fdb.py and was fixed by introducing a longer wait_until.

Log snippet showing missing 3000 'name'

23/12/2025 13:19:08 base._run   L0073 DEBUG  | /var/AzDevOps/tests/common/helpers/snmp_helpers.py::_get_snmp_facts#30: [localhost] AnsibleModule::snmp_facts, args=[], kwargs={"host": "192.168.121.10", "version": "v2c", "community": "public", "is_dell": false, "module_ignore_errors": false, "include_swap": false, "timeout": 20}
23/12/2025 13:19:18 base._run   L0111 DEBUG  | /var/AzDevOps/tests/common/helpers/snmp_helpers.py::_get_snmp_facts#30: [localhost] AnsibleModule::snmp_facts Result => ...
"snmp_interfaces": {
  "1": {"ifindex": "0", "name": "etp1a", ...},
  ...
  "10000": {"ifindex": "9999", "name": "eth0", ...},
  "3000": {"ifHighSpeed": "40000", "description": ""}
}

Example CLI output for snmp get

Healthy

 ~% snmpget -v2c -c public 192.168.121.10 1.3.6.1.2.1.2.2.1.2.3000
IF-MIB::ifDescr.3000 = STRING: Vlan1000

During config reload

~% snmpget -v2c -c public 192.168.121.10 1.3.6.1.2.1.2.2.1.2.3000
IF-MIB::ifDescr.3000 = No Such Object available on this agent at this OID

Steps to Reproduce

test_snmp_interfaces failed with the KeyError: 'name' issue 2/13 time when running tests in this order.

snmp/test_snmp_cpu.py::test_snmp_cpu
snmp/test_snmp_default_route.py::test_snmp_default_route
snmp/test_snmp_fdb.py::test_snmp_fdb_send_tagged
snmp/test_snmp_interfaces.py::test_snmp_interfaces

Actual Behavior and Expected Behavior

Before tests run: full data returned

During/just after config reload: missing key 'name' for specific interfaces (e.g., ifIndex 3000)

  • Expected is all MIB data is returned (including 'name')

Relevant log output

localhost = <tests.common.devices.local.Localhost object at 0x7f765d034760>
creds_all_duts = {'upscaleai-t0': {'7nodes_cisco_P1': 'vars/configdb_jsons/7nodes_cisco/P1.json', '7nodes_cisco_P2': 'vars/config...o_P3': 'vars/configdb_jsons/7nodes_cisco/P3.json', '7nodes_cisco_P4': 'vars/configdb_jsons/7nodes_cisco/P4.json', ...}}
duthosts = [<MultiAsicSonicHost upscaleai-t0>]
enum_rand_one_per_hwsku_hostname = 'upscaleai-t0'

    @pytest.mark.bsl
    def test_snmp_interfaces(localhost, creds_all_duts, duthosts, enum_rand_one_per_hwsku_hostname):
        """compare the snmp facts between observed states and target state"""
        duthost = duthosts[enum_rand_one_per_hwsku_hostname]
        hostip = duthost.host.options['inventory_manager'].get_host(
            duthost.hostname).vars['ansible_host']
    
        snmp_facts = get_snmp_facts(
            duthost, localhost, host=hostip, version="v2c",
            community=creds_all_duts[duthost.hostname]["snmp_rocommunity"], wait=True)['ansible_facts']
    
>       snmp_ifnames = [v['name']
                        for k, v in list(snmp_facts['snmp_interfaces'].items())]

creds_all_duts = {'t0': {'7nodes_cisco_P1': 'vars/configdb_jsons/7nodes_cisco/P1.json', '7nodes_cisco_P2': 'vars/config...o_P3': 'vars/configdb_jsons/7nodes_cisco/P3.json', '7nodes_cisco_P4': 'vars/configdb_jsons/7nodes_cisco/P4.json', ...}}
duthost    = <MultiAsicSonicHost upscaleai-t0>
duthosts   = [<MultiAsicSonicHost upscaleai-t0>]
enum_rand_one_per_hwsku_hostname = 'upscaleai-t0'
hostip     = '192.168.121.10'
localhost  = <tests.common.devices.local.Localhost object at 0x7f765d034760>
snmp_facts = {'ansible_all_ipv4_addresses': [], 'ansible_sysCachedMemory': 12235164, 'ansible_sysTotalBuffMemory': 99132, 'ansible_sysTotalFreeMemory': 14617100, ...}

snmp/test_snmp_interfaces.py:171: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

.0 = <list_iterator object at 0x7f764fb56d60>

>   snmp_ifnames = [v['name']
                    for k, v in list(snmp_facts['snmp_interfaces'].items())]
E   KeyError: 'name'

.0         = <list_iterator object at 0x7f764fb56d60>
k          = '3000'
v          = {'description': '', 'ifHighSpeed': '40000'}

snmp/test_snmp_interfaces.py:171: KeyError

Output of show version

Attach files (if any)

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions