Skip to content

helm: configure kube client-go exponential backoff by default#36648

Merged
pchaigno merged 2 commits intocilium:mainfrom
wedaly:default-client-go-exponential-backoff
Feb 11, 2025
Merged

helm: configure kube client-go exponential backoff by default#36648
pchaigno merged 2 commits intocilium:mainfrom
wedaly:default-client-go-exponential-backoff

Conversation

@wedaly
Copy link
Copy Markdown

@wedaly wedaly commented Dec 16, 2024

Configure env vars KUBE_CLIENT_BACKOFF_BASE and
KUBE_CLIENT_BACKOFF_DURATION in cilium-agent.

This is used by client-go to enable exponential backoff to avoid a thundering herd of LIST requests when apiserver is overloaded.

Users can configure the backoff using these helmvalues:

k8sClientExponentialBackoff:
    enabled: true
    backoffBaseSeconds: 1
    backoffMaxDurationSeconds: 120

Fixes: #36525

Enable client-go exponential backoff in cilium agent by default.

@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Dec 16, 2024
@wedaly wedaly force-pushed the default-client-go-exponential-backoff branch 3 times, most recently from c368f3d to 327aa67 Compare December 18, 2024 17:16
@github-actions

This comment was marked as resolved.

Sign in to view
@github-actions github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Jan 18, 2025
@wedaly wedaly marked this pull request as ready for review January 21, 2025 22:10
@wedaly wedaly requested review from a team as code owners January 21, 2025 22:10
@wedaly wedaly requested a review from squeed January 21, 2025 22:10
@wedaly

This comment was marked as outdated.

Sign in to view
@wedaly wedaly force-pushed the default-client-go-exponential-backoff branch from 327aa67 to 383ba85 Compare January 21, 2025 23:05
@wedaly

This comment was marked as outdated.

Sign in to view
@github-actions github-actions bot removed the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Jan 22, 2025
@wedaly
Copy link
Copy Markdown
Author

wedaly commented Jan 23, 2025

discussed in sig-scalability meeting this morning: TODO on me to add an upgrade note for this in the docs

@wedaly wedaly force-pushed the default-client-go-exponential-backoff branch from 383ba85 to 0f078b8 Compare January 23, 2025 22:17
@wedaly wedaly requested a review from a team as a code owner January 23, 2025 22:17
@wedaly wedaly requested a review from qmonnet January 23, 2025 22:17
@maintainer-s-little-helper

This comment was marked as resolved.

Sign in to view
@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-sign-off The author needs to add signoff to their commits before merge. label Jan 23, 2025
@wedaly
Copy link
Copy Markdown
Author

wedaly commented Jan 23, 2025 •
edited
Loading

rebased and added upgrade note to docs. Please let me know if you'd prefer this to be in the same commit as the helm chart change.

@wedaly wedaly force-pushed the default-client-go-exponential-backoff branch from 0f078b8 to 7a44e46 Compare January 23, 2025 22:32
@maintainer-s-little-helper maintainer-s-little-helper bot removed the dont-merge/needs-sign-off The author needs to add signoff to their commits before merge. label Jan 23, 2025
@wedaly

This comment was marked as outdated.

Sign in to view
@wedaly wedaly force-pushed the default-client-go-exponential-backoff branch from 7a44e46 to bc4a399 Compare January 23, 2025 22:40
@wedaly
Copy link
Copy Markdown
Author

wedaly commented Jan 23, 2025

/test

qmonnet
qmonnet approved these changes Jan 24, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@qmonnet qmonnet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good to me, thank you!

@qmonnet qmonnet added release-note/minor This PR changes functionality that users may find relevant to operating Cilium. integration/cloud Related to integration with cloud environments such as AKS, EKS, GKE, etc. area/helm Impacts helm charts and user deployment experience labels Jan 24, 2025
This was referenced Oct 2, 2025
Open
Open
Open
Open
Closed
Open
Merged
Open
Merged
Open
This was referenced Oct 20, 2025
Merged
Open
@marcolongol-bot marcolongol-bot bot mentioned this pull request Oct 23, 2025
Merged
1 task
@flkr-23 flkr-23 bot mentioned this pull request Oct 23, 2025
Open
1 task
@mfin-bot mfin-bot bot mentioned this pull request Oct 23, 2025
Merged
1 task
@obocaman obocaman bot mentioned this pull request Oct 23, 2025
Closed
1 task
@parsec-renovate parsec-renovate bot mentioned this pull request Oct 23, 2025
Open
1 task
@klabautermann-bot klabautermann-bot bot mentioned this pull request Oct 23, 2025
Open
1 task
@robo-ross robo-ross bot mentioned this pull request Oct 23, 2025
Open
1 task
This was referenced Oct 23, 2025
Open
Open
This was referenced Oct 23, 2025
Open
Open
@glad-os-bot glad-os-bot bot mentioned this pull request Oct 23, 2025
Closed
1 task
This was referenced Oct 23, 2025
Closed
Open
Open
Open
Open
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@squeed squeed squeed approved these changes

@marseel marseel marseel approved these changes

@qmonnet qmonnet qmonnet approved these changes

Assignees

No one assigned

Labels

area/helm Impacts helm charts and user deployment experience integration/cloud Related to integration with cloud environments such as AKS, EKS, GKE, etc. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/minor This PR changes functionality that users may find relevant to operating Cilium. sig/scalability Impacts how well Cilium handles a high rate of events or churn.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

CFP: configure kube client-go exponential backoff by default

5 participants

@wedaly @marseel @squeed @qmonnet @pchaigno