Repository navigation
virt-operator: stop GETting the disabled ContainerPath webhook on every reconcile - #19499
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Hi @fjvbn2003. Thanks for your PR. PRs from untrusted users cannot be marked as trusted with I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
| // reconcile for every cluster that does not use ContainerPathVolumes. | ||
| _, exists, _ := r.stores.MutatingWebhookCache.Get(webhook) | ||
| if !exists { | ||
| // Also check directly in case cache is not populated |
There was a problem hiding this comment.
I am not sure if this was ever a big deal since we would just queue it up when event handler notices it appears/disappears?
I think we can drop the comment
There was a problem hiding this comment.
Good point, the informer requeues on add/delete anyway, so the comment overstated it. Dropped it.
When the ContainerPathVolumes feature gate is disabled, the operator looks up the virt-launcher-pod-mutator MutatingWebhookConfiguration in its informer cache and, on a cache miss, sends a GET to the API server in case the cache is not populated yet. On clusters that do not use the feature gate the webhook never exists, so every reconcile sends a GET that returns 404. Make the controller wait for the MutatingWebhook informer to sync, as it already does for the ValidatingWebhook informer, and trust the cache. The webhook is always created with the managed-by label that the informer selects on, so a webhook the operator created is always visible there. Assisted-by: Claude <[email protected]> Signed-off-by: YoungjuKim <[email protected]>
356c37d to
47a37b0
Compare
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: mhenriks The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
Required labels detected, running phase 2 presubmits: |
What this PR does
Before this PR:
With the
ContainerPathVolumesfeature gate disabled (the default), every virt-operator reconcile callscreateOrDeleteContainerPathVolumesWebhook, misses thevirt-launcher-pod-mutatorwebhook in the informercache (it does not exist), and falls back to
GET mutatingwebhookconfigurations/virt-launcher-pod-mutator,which returns 404.
Measured on a homelab cluster (KubeVirt v1.9.0, Kubernetes v1.34.10, leader virt-operator up for 59 days),
from the operator's own
kubevirt_rest_client_requests_total:GET mutatingwebhookconfigurations→ 404LIST nodes→ 200 (once per reconcile, for comparison)So there is one 404 per reconcile. These 404s are 5.4% of all operator requests and all but 4 of its non-2xx
responses. They keep a constant ~5% floor under the error ratio that
VirtOperatorRESTErrorsBurstis builton (far below that alert's 80% threshold, so no alert fires, but it hides real errors on dashboards).
After this PR:
the only webhook informer missing from
hasSynced;ValidationWebhookis already there).References
of Lease updates. virt-operator: ignore unrelated lease updates and renewals #19417 lowers how often the reconcile runs; this PR removes a request that is not needed
at any reconcile frequency.
Why we need it and why it was done in this way
The following tradeoffs were made:
app.kubernetes.io/managed-bylabel from the webhook by hand, the operator no longersees it and will not delete it when the gate is off. That matches how the operator treats its other
label-selected objects.
The following alternatives were considered:
hasSyncedgives the same guarantee with one line.Links to places where the discussion took place: none
Special notes for your reviewer
spec: it fails if any
getaction for this webhook name is recorded on the fake clientset.GOFLAGS=-mod=vendor, onmain@ 6e80bca): the focused spec-ginkgo.focus "ContainerPathVolumes feature gate"fails with only the test change applied (3 ran, 1 failed) and passes with the fix;go test ./pkg/virt-operator/...passes. One earlier run of./pkg/virt-operator/resource/apply/reported aTestInstallStrategyfailure that did not reproduce on re-runs with or without this change.Checklist
AI disclosure: I used Claude (Anthropic) to help trace the cause from the metrics and to draft the change and
the test. I reviewed every line and measured the numbers above on my own cluster. The commit carries an
Assisted-bytrailer.Release note
🤖 Generated with Claude Code