You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 495f5b3
Browse filesBrowse the repository at this point in the historyBrowse files
Stop failing scheduled tasks when a serialized Dag is briefly missing
A serialized Dag that cannot be read is almost always transient -- the Dag processor
is mid-write, or the row was briefly unreadable -- but the scheduler treated it as
final and failed every SCHEDULED task instance of the Dag, across all of its Dag
runs, bypassing retries and failure callbacks. The Dag run was not finished either,
so those tasks could only be recovered by clearing them by hand.
The bulk update also doubled as a starvation filter: flipping the rows to FAILED is
what kept them out of the next iteration of the critical-section query. Starving the
run preserves that effect, so an unresolvable Dag cannot hold up the Dags queued
behind it. The error is reported once per Dag rather than once per task instance, so
a Dag with many stuck runs cannot flood the scheduler log now that its task
instances survive the round.
closes: apache#62050
Stop failing every ``SCHEDULED`` task instance of a Dag when its serialized Dag cannot be read while evaluating task-level concurrency limits; such a lookup is usually transient, so the task instances are now left ``SCHEDULED`` and reconsidered on the next scheduling loop.
0 commit comments