Repository navigation
Commit fac497c
committed
### What changes were proposed in this pull request?
This PR aims to make `ExecutorResizePlugin` skip executor pods whose `spark-exec-id` label is the placeholder `EXECID`, like `ExecutorPodsSnapshot` does.
### Why are the changes needed?
`deployment` and `statefulset` pods allocators create executor pods with the `EXECID` label and patch the real executor ID later on a best-effort basis. Currently, `ExecutorResizePlugin` runs only with the `direct` pods allocator, so this is a defensive improvement. It keeps the placeholder out of the `cappedExecutors` keys and the logs.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs with the newly added test case.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #59175 from dongjoon-hyun/SPARK-59914.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 6ed637f)
Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent a20c062 commit fac497c
2 files changed
Lines changed: 30 additions & 5 deletions
File tree
- resource-managers/kubernetes/core/src
- main/scala/org/apache/spark/scheduler/cluster/k8s
- test/scala/org/apache/spark/scheduler/cluster/k8s
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
114 | 124 | | |
115 | | - | |
116 | | - | |
117 | | - | |
| 125 | + | |
118 | 126 | | |
119 | | - | |
120 | | - | |
| 127 | + | |
121 | 128 | | |
122 | 129 | | |
123 | 130 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
161 | 161 | | |
162 | 162 | | |
163 | 163 | | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
164 | 182 | | |
165 | 183 | | |
166 | 184 | | |
| |||
0 commit comments