Skip to content

[Bug]: go.d/docker: removing a container also removes the charts of every container whose name starts with the same string #24230

Open

Description

@Ian-Lo

Bug description

When a container is removed, the go.d docker collector also removes the per-container charts of any other container whose name begins with the removed container's name. For example, removing foo also removes foo-bar's charts. Those charts are never re-created while the Netdata plugin keeps running, so foo-bar silently drops out of docker.container_state, docker.container_health_status and docker.container_writable_layer_size. Its docker_container_down and docker_container_unhealthy alerts go to REMOVED.

Cause (v2.12.0, unchanged on master and in v2.12.1):

charts.go#L166-L174 matches chart IDs by prefix without a separator:

func (c *Collector) removeContainerCharts(name string) {
	px := fmt.Sprintf("container_%s", name)

	for _, chart := range *c.Charts() {
		if strings.HasPrefix(chart.ID, px) {
			chart.MarkRemove()
			chart.MarkNotCreated()
		}
	}
}

So container_foo matches container_foo-bar_state, container_foo-bar_health_status and container_foo-bar_writable_layer_size.

collect.go#L188-L193 deletes only foo from c.containers, and collect.go#L162-L165 adds charts only for names that are not already in c.containers. foo-bar stays in the map, so its charts are never added again.

Expected behavior

Removing container foo removes only foo's charts. foo-bar's charts and alerts stay in place while it keeps running.

Steps to reproduce

  1. With the go.d docker collector enabled, start two containers:
    docker run -d --name foo-bar alpine sleep infinity and docker run -d --name foo alpine sleep infinity
  2. Wait for both containers' charts to appear (for example docker.container_state for foo and for foo-bar).
  3. docker rm -f foo
  4. After the next collection, foo-bar's charts are gone (marked obsolete) even though foo-bar is still running. They do not come back.
  5. Restart Netdata (or the go.d plugin): foo-bar's charts come back.

Installation method

docker

System info

$ uname -srvmo
Linux 6.8.0-146-generic #146-Ubuntu SMP PREEMPT_DYNAMIC Thu Sep  3 16:12:30 UTC 2026 x86_64 GNU/Linux
$ grep -HvE "^#|URL" /etc/*release
/etc/lsb-release:DISTRIB_ID=Ubuntu
/etc/lsb-release:DISTRIB_RELEASE=24.04
/etc/lsb-release:DISTRIB_CODENAME=noble
/etc/lsb-release:DISTRIB_DESCRIPTION="Ubuntu 24.04.5 LTS"
/etc/os-release:PRETTY_NAME="Ubuntu 24.04.5 LTS"
/etc/os-release:VERSION_ID="24.04"
/etc/os-release:VERSION_CODENAME=noble
/etc/os-release:ID=ubuntu
/etc/os-release:ID_LIKE=debian
Docker Engine 29.8.2

Netdata build info

Packaging:
    Netdata Version ____________________________________________ : v2.12.0
    Installation Type __________________________________________ : binpkg-deb
    Package Architecture _______________________________________ : x86_64
    Package Distro _____________________________________________ : ubuntu 24.04
Operating System:
    Kernel _____________________________________________________ : Linux
    Kernel Version _____________________________________________ : 6.8.0-146-generic
    Operating System ___________________________________________ : Ubuntu
    Operating System ID ________________________________________ : ubuntu
    Operating System ID Like ___________________________________ : debian
    Operating System Version ___________________________________ : 24.04.5 LTS (Noble Numbat)
    Operating System Version ID ________________________________ : 24.04
    Detection __________________________________________________ : /etc/os-release
(netdata -W buildinfo, Packaging and Operating System sections)

Additional info

Suggested fix: add the separator:

px := fmt.Sprintf("container_%s_", name)

This fixes the foo / foo-bar case. Container names may also contain _, though, so container_foo_ would still match a container named foo_state (container_foo_state_state). Matching the exact IDs built from the per-container chart templates avoids that, for example:

func (c *Collector) removeContainerCharts(name string) {
	ids := make(map[string]bool)
	for _, tmpl := range containerChartsTmpl {
		ids[fmt.Sprintf(tmpl.ID, name)] = true
	}
	for _, chart := range *c.Charts() {
		if ids[chart.ID] {
			chart.MarkRemove()
			chart.MarkNotCreated()
		}
	}
}

(containerChartsTmpl is the template set that addContainerCharts uses, charts.go#L92-L96.)

Some other go.d collectors use the same unsuffixed HasPrefix(chart.ID, ...) removal on master, for example ntpd (peer_%s), nvme (device_%s), wireguard (device_%s, peer_%s), logstash ("pipeline_"+id), upsd, proxysql (mysql_user_, mysql_command_) and k8s_state (pod_%s, node_%s, deployment_%s, cronjob_%s). I have not checked whether each of those can actually collide.

Activity

  1. added theissue type on Oct 10, 2026
  2. self-assigned this
    on Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions