fix: create IPv6 EndpointSlice for kubelet Service on dual-stack clusters by ckrause · Pull Request #8682 · prometheus-operator/prometheus-operator

ckrause · 2026-07-10T08:45:21Z

Description

getNodeAddresses() was discarding the full address map returned by nodeAddress() and only emitting a single IP per node, which always resolved to IPv4 on dual-stack clusters. This prevented the creation of an IPv6 EndpointSlice for the kubelet Service.

With Prometheus 3.11 (prometheus/prometheus#18192), EndpointSlices whose addressType does not match svc.Spec.IPFamilies[0] are skipped to avoid duplicate scraping. On IPv6-primary dual-stack clusters this caused all kubelet targets to be dropped, as only the IPv4 EndpointSlice existed.

Fix by iterating all InternalIP (or ExternalIP when priority is "external") addresses per node so that syncEndpointSlice() — which already has the dual-stack infrastructure — creates separate IPv4 and IPv6 EndpointSlices.

Closes: #8681

Type of change

What type of changes does your code introduce to the Prometheus operator? Put an x in the box that apply.

CHANGE (fix or feature that would cause existing functionality to not work as expected)
FEATURE (non-breaking change which adds functionality)
BUGFIX (non-breaking change which fixes an issue)
ENHANCEMENT (non-breaking change which improves existing functionality)
NONE (if none of the other choices apply. Example, tooling, build system, CI, docs, etc.)

Verification

Tested manually in a development cluster. Unit tests extended for dual-stack.

Changelog entry

- Fix kubelet scraping on dual-stack clusters (IPv6,IPv4)

…ters getNodeAddresses() was discarding the full address map returned by nodeAddress() and only emitting a single IP per node, which always resolved to IPv4 on dual-stack clusters. This prevented the creation of an IPv6 EndpointSlice for the kubelet Service. With Prometheus 3.11 (prometheus/prometheus#18192), EndpointSlices whose addressType does not match svc.Spec.IPFamilies[0] are skipped to avoid duplicate scraping. On IPv6-primary dual-stack clusters this caused all kubelet targets to be dropped, as only the IPv4 EndpointSlice existed. Fix by iterating all InternalIP (or ExternalIP when priority is "external") addresses per node so that syncEndpointSlice() — which already has the dual-stack infrastructure — creates separate IPv4 and IPv6 EndpointSlices. Also set IPFamilyPolicy: PreferDualStack on the kubelet Service so Kubernetes assigns both ClusterIPs on dual-stack clusters; the policy degrades gracefully to single-stack on clusters that only support one IP family. Fixes prometheus-operator#8681

CreateOrUpdateService always copies IPFamilies and IPFamilyPolicy from the existing Service on update, so setting them in syncService() has no effect after initial creation. On first creation Kubernetes assigns the correct values based on the cluster's network configuration.

ckrause · 2026-07-14T07:51:06Z

Update: I removed the redundant IPFamilyPolicy from syncService.
CreateOrUpdateService always copies IPFamilies and IPFamilyPolicy from the existing Service on update, so setting them in syncService() has no effect after initial creation. On first creation, Kubernetes assigns the
correct values based on the cluster's network configuration.
I successfully tested the new version in a cluster; it works as expected.

simonpasquier · 2026-07-15T15:22:25Z


 	for _, n := range nodes {
-		address, _, err := c.nodeAddress(n)
+		_, addrMap, err := c.nodeAddress(n)


should we change nodeAddress() to return the map only? If yes, we can also rename it.

simonpasquier · 2026-07-15T15:24:46Z

-			continue
+		// Select addresses by priority: use ExternalIP if priority is "external"
+		// and external addresses exist, otherwise fall back to InternalIP.
+		addrType := corev1.NodeInternalIP


should we use the NodeExternalIP addresses if the NodeInternalIP list is empty?

…p only The single string return value is no longer used after the fix to emit all InternalIP addresses per node. Simplify the signature to return the address map only and rename the function to better reflect its purpose.

…sesByType() Move the priority/fallback logic into a dedicated primaryAddressType() helper so getNodeAddresses() can use it to select the correct address type for dual-stack nodes. nodeAddressesByType() is simplified to just build and return the address map without any error return. Also restores the ExternalIP fallback that was missing after the initial dual-stack fix.

ckrause · 2026-07-17T09:05:42Z

Hi @simonpasquier, thanks for your comments. I addressed both of them. Can you please check the new version?

simonpasquier

Can/should we combine nodeAddressesByType and primaryAddressType in a single function? It would be good to add unit tests too.

simonpasquier · 2026-07-17T10:59:19Z

 	}
-
-	return "", m, fmt.Errorf("host address unknown")
+	return "", fmt.Errorf("host address unknown")


unreachable since we have a "default:" now.

In theory, the trailing return is still reachable: if a node has neither NodeInternalIP nor NodeExternalIP addresses, neither if condition in the matching branch will be true, and execution falls through to the return after the switch. This covers both the "external" case and the default: case when no addresses are present. Moving the error into each branch would duplicate code.

simonpasquier · 2026-07-17T12:43:36Z

It would be good to add unit tests too.

sorry I missed that you already added them 👍

ckrause · 2026-07-17T13:05:48Z

Thanks for the suggestion. I kept nodeAddressesByType and primaryAddressType as separate functions because primaryAddressType takes a plain map, which makes it easier to test in isolation without constructing full Node objects — as you can see in the new TestPrimaryAddressType I just added. Happy to combine them if you prefer though.

simonpasquier · 2026-07-17T13:25:02Z

Happy to combine them if you prefer though.

yes please

…eAddresses

ckrause · 2026-07-17T13:43:57Z

Sure, done.

ckrause · 2026-07-22T09:49:09Z

Hi @simonpasquier , is it ready to merge from your side now?

ckrause requested a review from a team as a code owner July 10, 2026 08:45

pull-request-size Bot added the size/M label Jul 10, 2026

simonpasquier reviewed Jul 15, 2026

View reviewed changes

ckrause added 2 commits July 17, 2026 10:55

pull-request-size Bot added size/L and removed size/M labels Jul 17, 2026

simonpasquier reviewed Jul 17, 2026

View reviewed changes

test: add unit tests for primaryAddressType()

8e6ba5d

test: fix formatting

a0b0636

refactor: combine nodeAddressesByType and primaryAddressType into nod…

b949fb7

…eAddresses

Uh oh!

Conversation

ckrause commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Verification

Changelog entry

Uh oh!

ckrause commented Jul 14, 2026

Uh oh!

simonpasquier Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

ckrause Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

simonpasquier Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

ckrause Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

ckrause commented Jul 17, 2026

Uh oh!

simonpasquier left a comment

Choose a reason for hiding this comment

Uh oh!

simonpasquier Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

ckrause Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

simonpasquier commented Jul 17, 2026

Uh oh!

ckrause commented Jul 17, 2026

Uh oh!

simonpasquier commented Jul 17, 2026

Uh oh!

ckrause commented Jul 17, 2026

Uh oh!

ckrause commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ckrause commented Jul 10, 2026 •

edited

Loading