The following code is used to get the counts, but it filters out "error" entries with when='setup' (and outcome='failed'):

keys = ('passed', 'failed', 'error', 'deselected')
counts = {
    k: len(list(filter(lambda r: getattr(r, 'when', '') == 'call', tr.stats.get(k, []))))
    for k in keys
}

This can happen with a "fixture 'foo' not found" error, and pytest-notifier would report a "Success", although the real outcome is "2 passed, X deselected, 25 error".