Repository navigation
Commit 4f76227
authored
Forward termination signals from supervisor to task subprocess (#61627)
When a Kubernetes worker pod receives SIGTERM (spot interruption, scaling
down, rolling update), the signal is delivered to the supervisor process
(PID 1 in the container). The supervisor has no signal handler installed
and exits with default behavior, leaving the task subprocess orphaned
without ever calling the operator's `on_kill()` hook. Spawned resources
(pods, subprocesses, etc.) are never cleaned up.
The task subprocess already has a SIGTERM handler registered in
`task_runner.py` that calls `on_kill()`, but the signal never reaches it
because the supervisor process terminates first.
Install SIGTERM/SIGINT signal handlers in `ActivitySubprocess.wait()`
that forward the received signal to the task subprocess via `os.kill()`.
The child's existing handler then calls `on_kill()` as expected,
restoring the Airflow 2 behavior. Handlers are saved before
`_monitor_subprocess()` and restored in a `finally` block.
Placing the handler in `ActivitySubprocess.wait()` (rather than
`supervise_task`) makes the forwarding work for any coordinator that
uses `ActivitySubprocess` and keeps the handler colocated with the
process it forwards to.
Fixes: #58936
Signed-off-by: André Ahlert <[email protected]>1 parent 120b49a commit 4f76227
3 files changed
Lines changed: 115 additions & 1 deletion
File tree
- task-sdk
- src/airflow/sdk/execution_time
- tests/task_sdk
- dags
- execution_time
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1391 | 1391 | | |
1392 | 1392 | | |
1393 | 1393 | | |
| 1394 | + | |
| 1395 | + | |
| 1396 | + | |
| 1397 | + | |
| 1398 | + | |
| 1399 | + | |
| 1400 | + | |
| 1401 | + | |
| 1402 | + | |
| 1403 | + | |
| 1404 | + | |
| 1405 | + | |
| 1406 | + | |
| 1407 | + | |
| 1408 | + | |
| 1409 | + | |
| 1410 | + | |
| 1411 | + | |
| 1412 | + | |
1394 | 1413 | | |
1395 | 1414 | | |
1396 | 1415 | | |
1397 | 1416 | | |
| 1417 | + | |
| 1418 | + | |
1398 | 1419 | | |
1399 | 1420 | | |
1400 | 1421 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
30 | 31 | | |
31 | | - | |
| 32 | + | |
32 | 33 | | |
33 | 34 | | |
34 | 35 | | |
| |||
262 | 263 | | |
263 | 264 | | |
264 | 265 | | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
265 | 314 | | |
266 | 315 | | |
267 | 316 | | |
| |||
0 commit comments