Skip to content

[CASCL-1386] (7/11) Evict EKS managed node groups#3174

Merged
L3n41c merged 3 commits into
mainfrom
lenaic/CASCL-1386-evict-07-eks-mng
Jul 3, 2026
Merged

[CASCL-1386] (7/11) Evict EKS managed node groups#3174
L3n41c merged 3 commits into
mainfrom
lenaic/CASCL-1386-evict-07-eks-mng

Conversation

@L3n41c

@L3n41c L3n41c commented Jun 18, 2026

Copy link
Copy Markdown
Member

What does this PR do?

Implements EKS managed node group eviction (scale-to-zero then wait; EKS drains pods server-side, so it needs no client-side drain primitives).

Motivation

PR #3026 is too large to review as a single change. This is part 7 of 11 of a stack that splits it into small, individually-reviewable pieces that each build and pass tests on their own. See #3026 for the full feature context and end-to-end QA.

Additional Notes

Stacked PR — the base branch is lenaic/CASCL-1386-evict-06-ec2-instance-id-helper, not main, so the diff shows only this step's change. Implements an evictor the skeleton (part 1) left stubbed. The code is taken verbatim from #3026 (rebased onto main); there are no logic changes versus #3026.

Minimum Agent Versions

N/A — kubectl-datadog plugin only.

Describe your test plan

go test ./cmd/kubectl-datadog/autoscaling/cluster/... passes at this commit. Full end-to-end QA is tracked on #3026.

Checklist

@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Jun 18, 2026

Copy link
Copy Markdown

Code Coverage

🎯 Code Coverage (details)
Patch Coverage: 93.18%
Overall Coverage: 45.42% (+0.07%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 7f9f9f5 | Docs | Datadog PR Page | Give us feedback!

@codecov-commenter

codecov-commenter commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 44.12%. Comparing base (0f3bbd1) to head (6c79f69).

Files with missing lines Patch % Lines
...bectl-datadog/autoscaling/cluster/evict/eks_mng.go 95.00% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@                                  Coverage Diff                                  @@
##           lenaic/CASCL-1386-evict-06-ec2-instance-id-helper    #3174      +/-   ##
=====================================================================================
+ Coverage                                              44.05%   44.12%   +0.07%     
=====================================================================================
  Files                                                    389      389              
  Lines                                                  31093    31132      +39     
=====================================================================================
+ Hits                                                   13698    13737      +39     
  Misses                                                 16501    16501              
  Partials                                                 894      894              
Flag Coverage Δ
unittests 44.12% <95.00%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...bectl-datadog/autoscaling/cluster/evict/eks_mng.go 95.12% <95.00%> (+95.12%) ⬆️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f3bbd1...6c79f69. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-07-eks-mng branch from c0c8b6a to 9250b71 Compare June 19, 2026 14:21
@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-06-ec2-instance-id-helper branch from dd81350 to 0f545f7 Compare June 19, 2026 14:21
@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-07-eks-mng branch from 9250b71 to f9c10d1 Compare June 22, 2026 13:24
@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-06-ec2-instance-id-helper branch from 0f545f7 to 3b7cda6 Compare June 22, 2026 13:25
@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-07-eks-mng branch 2 times, most recently from 8e43ad1 to 3718447 Compare June 25, 2026 08:18
@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-06-ec2-instance-id-helper branch 2 times, most recently from 91c9a7b to 0f3bbd1 Compare June 26, 2026 12:16
@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-07-eks-mng branch from 3718447 to 6c79f69 Compare June 26, 2026 12:16
@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-06-ec2-instance-id-helper branch from 0f3bbd1 to e69973f Compare June 26, 2026 14:45
@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-07-eks-mng branch 2 times, most recently from f91f53d to 7322549 Compare June 29, 2026 15:41
Base automatically changed from lenaic/CASCL-1386-evict-06-ec2-instance-id-helper to main June 29, 2026 15:44
@L3n41c
L3n41c marked this pull request as ready for review June 29, 2026 15:58
@L3n41c
L3n41c requested review from a team as code owners June 29, 2026 15:58

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3d6b7023b3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread cmd/kubectl-datadog/autoscaling/cluster/evict/eks_mng.go
@L3n41c
L3n41c changed the base branch from main to lenaic/CASCL-1386-evict-06.5-cordon-drain July 1, 2026 13:46
@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-07-eks-mng branch 2 times, most recently from 409d632 to 5884694 Compare July 1, 2026 22:58
L3n41c added 2 commits July 2, 2026 11:38
Introduce the reusable node-cordon and pod-eviction/drain primitives that the
per-manager evictors (EKS managed node groups, ASG, Karpenter, standalone)
build on:

- cordonNodes/cordonNode: idempotent, conflict-retrying cordon that treats an
  already-gone node as a silent skip and returns the cordoned Node objects.
- drainNode: evicts every evictable pod (skipping mirror, DaemonSet, completed
  and terminating pods), then waits for the node to empty.
- evictPodWithRetry: PDB-aware eviction that retries on 429 and treats 404 as
  success.
- listPodsOnNode, waitForNodeEmpty and the pod-classification predicates
  (podOccupiesNode/shouldSkipEviction/isMirrorPod/isDaemonSetPod/isCompleted).

These primitives have no production caller yet; the first (evictEKSManagedNode-
Group) lands in the next PR in the stack. A temporary blank-identifier
reference keeps golangci-lint's `unused` quiet until then (the linter runs with
tests = false, so the unit tests do not count as use) and is removed by that PR.
Part of a stack splitting #3026 (too large to review in one piece) into
small pieces that each build and pass tests on their own. The command is
fully functional only once the whole stack lands.
@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-07-eks-mng branch from 5884694 to eec5e58 Compare July 2, 2026 09:43
@L3n41c
L3n41c force-pushed the lenaic/CASCL-1386-evict-06.5-cordon-drain branch from 47f7ed9 to 39bbf26 Compare July 2, 2026 09:43
Comment thread cmd/kubectl-datadog/autoscaling/cluster/evict/eks_mng.go
Comment thread cmd/kubectl-datadog/autoscaling/cluster/evict/eks_mng.go
Base automatically changed from lenaic/CASCL-1386-evict-06.5-cordon-drain to main July 3, 2026 09:05
@L3n41c
L3n41c merged commit 9f6c015 into main Jul 3, 2026
30 of 38 checks passed
@L3n41c
L3n41c deleted the lenaic/CASCL-1386-evict-07-eks-mng branch July 3, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants