Skip to content

fix(containers): aggregate network stats over all container interfaces (#3669) - #3670

Merged
nicolargo merged 2 commits into
nicolargo:developfrom
ntdatt812:fix/3669-container-network-all-interfaces
Aug 22, 2026
Merged

nicolargo merged 2 commits into
nicolargo:developfrom
ntdatt812:fix/3669-container-network-all-interfaces

Conversation

@ntdatt812

Copy link
Copy Markdown
Contributor

Description

Fixes #3669.

DockerStatsFetcher._get_network_stats() read the container network counters from a
single hardcoded interface:

eth0_stats = self._streamer.stats.get('networks', {}).get('eth0')

The Docker stats endpoint returns one entry per container interface, so a container
attached to several networks (docker network connect, or several networks in a Compose
file) gets eth0, eth1, … and everything past eth0 was silently dropped. Reported
RX/TX for those containers was too low.

The method now sums rx_bytes/tx_bytes across all interfaces. Its output contract is
unchanged: same keys, same types, same units.

Two deliberate choices, as discussed in the issue:

  • lo is excluded. The Docker daemon does not normally report loopback in networks,
    but Docker-API-compatible runtimes have been observed to, and loopback traffic would
    massively inflate the totals.
  • Interfaces with missing counters are skipped, not fatal. Previously a malformed
    eth0 aborted the whole method; now one malformed interface among several no longer
    discards the valid ones. None is still returned when nothing usable remains, so
    --network host containers (no networks key) behave exactly as before.

On the decision point raised in the issue

The issue asked not to decide alone between (a) leave as is and (b) clamp the delta
to zero
, given that attaching/detaching a network between two samples changes the
interface set and can produce one negative rx/tx delta.

This PR implements (a), for two reasons:

  1. Negative deltas are already reachable today on container restart, when the counters
    reset. (b) would silently swallow that pre-existing signal too, which changes behaviour
    for a case that is not in the scope of this issue.
  2. (b) hides a real counter discontinuity rather than reporting it. If the glitch turns
    out to matter in practice, clamping is better addressed once for all cumulative
    counters than locally here.

Happy to switch to (b) if you prefer.

Tests

New file tests/test_plugin_containers.py, 9 cases, pytest style to match the sibling
test_plugin_*.py files. DockerStatsFetcher.__init__ opens a live stats stream, so the
instance is built via __new__ with the two attributes the method reads injected.

Case Expected
Single interface identical values to the previous implementation (regression guard)
Two interfaces counters summed
Loopback present lo excluded
Host network (no networks key) None
Empty networks dict None
All interfaces malformed None
One malformed among two malformed skipped, valid one still counted
Rate computation rx/tx are the summed deltas, time_since_update present
First sample no rx/tx/time_since_update keys

Verified the tests actually pin the bug — against the unpatched docker.py, exactly the
two cases that target it fail and the seven regression guards pass:

FAILED tests/test_plugin_containers.py::TestDockerNetworkStatsAggregation::test_two_interfaces_are_summed
        assert 100 == 150
FAILED tests/test_plugin_containers.py::TestDockerNetworkStatsRates::test_rates_use_the_summed_counters
        assert 60 == 110
2 failed, 7 passed

With the patch: 9 passed.

Regression run

Full suite before and after, same machine (Windows 11, Python 3.14, 1 running container):

before: 57 failed, 571 passed, 106 skipped, 15 errors
after:  58 failed, 579 passed, 106 skipped, 15 errors

The 72 pre-existing failures/errors — test_restful.py, test_xmlrpc.py, test_mcp.py,
test_browser_restful.py, test_amp_secure_popen.py, test_webui.py, test_perf.py, all
server/WebUI tests that need a chromedriver and a POSIX socket — are identical line for
line
before and after. The 9 new tests account for the passed count.

The single delta is test_memoryleak.py::test_memoryleak_no_history
(assert memory_leak < 15000, got 15524). It is not caused by this change:

  • The file passes on its own with the patch applied.

  • Its measurement is unstable on this machine, and it exceeds its own threshold on
    unpatched develop
    . Three runs each, reading the value the test itself logs:

    run unpatched develop this branch
    1 12 391 B 16 734 B
    2 7 948 B 20 268 B
    3 32 480 B ❌ 6 718 B
    mean 17 606 B 14 573 B

    The unpatched spread is 7 948 – 32 480 B, a 4× range against a hard-coded 15 000 B
    threshold, and the unpatched maximum is the largest of all six runs.

  • Isolating the changed method removes the ambiguity. Driving both implementations
    200 000 times under tracemalloc with the same single-interface payload:

       old (eth0 only): retained after 200,000 calls =  688 bytes  (0.0034 B/call)
      new (aggregated): retained after 200,000 calls =  656 bytes  (0.0033 B/call)
    

    Neither retains anything — that residue is tracemalloc's own bookkeeping — and the new
    form retains marginally less. The memory-leak test performs ~13 calls per run, so the
    per-call difference is well under a byte.

Flagging the threshold separately since it is unrelated to this PR: on a machine with any
background load, test_memoryleak_no_history looks like it can fail on a clean checkout.
Happy to open a separate issue if that is useful.

ruff format and ruff check are clean on both touched files. rstcheck NEWS.rst reports
the same 8 pre-existing issues before and after, shifted by the one added line.

Branches

Applied to develop. I re-verified the branch parity the issue asked about rather than
assuming it: glances/plugins/containers/engines/docker.py has blob SHA
c818def749329c9b2d00171c43cb2f2739fac5f4 on both develop and develop-v5, so the
cherry-pick to develop-v5 is clean. Say the word and I'll open the companion PR.

Resume

_get_network_stats() read the container network counters from a single
hardcoded interface. The Docker stats endpoint returns one entry per
container interface, so a container attached to several networks gets
eth0, eth1... and everything past eth0 was silently dropped, making the
reported RX/TX rates too low.

Sum rx_bytes/tx_bytes across all the interfaces instead. Loopback is
excluded, since Docker-API-compatible runtimes have been observed to
report it and its traffic would inflate the totals. Interfaces with
missing counters are skipped rather than aborting the whole method, so
one malformed interface no longer discards the valid ones. None is still
returned when nothing usable remains, keeping --network host containers
unchanged.

The output contract is untouched: same keys, same types, same units.

Fixes nicolargo#3669
…arry

Matches the convention in tests/test_plugin_sensors.py and clears the
docstring findings Codacy raised on the PR.
@ntdatt812

Copy link
Copy Markdown
Contributor Author

Two red checks on this PR, both traced — neither is a defect in the change, but you should decide on the second one.

AppVeyor — red on every open PR right now, not just this one: #3667, #3665 and #3613 all show continuous-integration/appveyor/pr = failure on their current heads. Nothing in this branch touches it.

Codacy — "13 new issues" — I reproduced the exact number locally. All 13 are W0212 protected-access, and they are the test harness itself:

$ pylint --disable=all --enable=W0212 tests/test_plugin_containers.py
tests/test_plugin_containers.py:20:4   W0212: Access to a protected member _container
tests/test_plugin_containers.py:21:4   W0212: Access to a protected member _streamer
tests/test_plugin_containers.py:25:4   W0212: Access to a protected member _old_computed_stats
tests/test_plugin_containers.py:26:4   W0212: Access to a protected member _last_stats_computed_time
tests/test_plugin_containers.py:36:16  W0212: Access to a protected member _get_network_stats
... (9 × _get_network_stats, one per test)

4 attribute injections in build_fetcher + 9 calls to _get_network_stats = 13. That is the harness the issue asked for — DockerStatsFetcher.__init__ opens a live stats stream, so the instance has to be built via __new__ with the attributes the method reads injected, and the method under test is private.

tests/test_core.py already does the same kind of thing (~22 protected accesses); Codacy only counts what a diff adds, so those don't show.

I have not "fixed" this, because both available fixes are your call and the repo has no precedent either way:

  • (a) leave it. perf: reuse the program aggregation until the process list changes #3655 merged with Codacy = action_required too, so the gate is evidently not blocking. Zero code churn.
  • (b) collapse the harness. Fold the sample into the helper — network_stats(networks, old_stats=None) builds and calls, returning the dict — so the 9 call sites become one. That takes it from 13 findings to 5 and arguably reads better, but it departs from the build_fetcher shape you specified in the issue, so I did not do it unilaterally.
  • (c) a module-level # pylint: disable=protected-access. Cheapest, but it would be the first pylint pragma in the codebase, which cuts against "match existing style".

Say which you prefer and I'll push it. Meanwhile I pushed the class docstrings the sibling test_plugin_*.py files carry (b11fd39) — that was a separate style gap, unrelated to the 13.

Also still open from the PR description: the decision point in the issue (I implemented (a) leave the negative delta as is, reasoning in the description), and whether you want the companion develop-v5 PR — the file is byte-identical there, blob c818def749329c9b2d00171c43cb2f2739fac5f4 on both branches.

@nicolargo
nicolargo merged commit a81f737 into nicolargo:develop Aug 22, 2026
1 of 3 checks passed
@nicolargo

Copy link
Copy Markdown
Owner

Thanks for the PR @ntdatt812

Just merged into develop. I just remove the line in the NEWS.md file because the patch will be release in Glances 4.5.7, not 4.5.6.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix container network accounting: sum all interfaces, not only eth0

2 participants