Officially support HA for scheduler component (AIP-15) #9630
Description
- Detect orphaned task instances by SchedulerJob id and heartbeat #10729 Detect orphaned task instances by SchedulerJob id and heartbeat
- Mark task as failed when it fails "sending" in Celery #10881 Mark task as failed when it fails sending in Celery
- Allow CeleryExecutor to "adopt" an orphaned queued or running task #10949 Allow CeleryExecutor to "adopt" an orphaned queued or running task
- Kubernetes executor can adopt tasks from other schedulers #10996 Allow Kubernetes executor to adopt tasks
- Add task adoption to CeleryKubernetesExecutor #11244 Add adoption support in to the new CeleryKubernetesExecutor (added in Add CeleryKubernetesExecutor #10901)
- The big one - scheduler HA pr Fully support running more than one scheduler concurrently #10956
- Perform "mini scheduling run" after task has finished #11498 Add "fast follow" to end of task run (this is a mini "scheduler" run at the end of task execution.)
- Performance optimisation PRs
- Decide if we need Ensure idle database connections don't hang around for ever #10933 or not
- Ensure smart sensor DAG is loaded by scheduler even when DAG serialization is enabled.