Keep Dag parse results tied to their source snapshot - #74789

Holding a bundle's files back until its in-flight imports finish made
reparse requests lose their place in the queue, let a steady stream of
callbacks keep the refresh waiting indefinitely, and gave operators no
sign that files were held back. Checking every bundle on each loop while
one waited also added a state query per bundle per loop.

A parse that crashed after its bundle re-initialized says nothing about
the new source, and copying version metadata on every dispatch grows
with manifest size. Files parsed more often than max_runs were queued
again, so `airflow dag-processor --num-runs` could never finish.

Fresh CI workers can reach this test before the first 300-second interval has elapsed. The test needs a due refresh followed by a retry within the same interval to cover import draining reliably.

Safe refresh failures should not stop ordinary Dag parsing or consume a
reparse request before fresh source is available. Callbacks and finite
processing runs must preserve that distinction, and metrics must remain
meaningful while files wait for a usable source.

Known limitation: the Git provider's Airflow 3.4 version gate still assumes
DagBundleRefreshError exists in the installed core. Compatibility with
unpatched 3.4 core requires a follow-up before publication.

Some 3.4 core builds predate the refresh-error contract even though they
satisfy the provider's version gate. Git bundle configuration must remain
usable with those supported cores and retain the original fetch error
when the contract is unavailable.

The refresh-error contract is not available in every Airflow 3.4 core build, so provider guidance must describe capability detection. Logger values in the callback source regression only need identity, not mock behavior.

Temporary bundle outages should not delay unrelated parsing or hide its metrics, and credential failures must not discard an intact checkout. Provider compatibility guidance must reflect the features available in the installed core.

When a bundle's refresh succeeded but listing its files or recording the
refresh then failed, the bundle was due again at the next check, and a
bundle that changes files on refresh drains its active parses before
each refresh. A persistent failure, such as a recursive symlink in a Git
bundle, therefore cut that bundle's parse throughput at every refresh
check for as long as it lasted. Those retries now wait for the bundle's
refresh interval, but only when a retry would drain parses. A bundle
with nothing listed yet, or one whose refresh never changes files, still
retries at the next check, so a failed first listing does not hide its
Dags for the whole interval.

Passing the failed discovery sources as a new argument to the
import-error cleanup made manager subclasses that override it with its
original signature crash on their first successful listing, the same
kind of break #64245 fixed for Dag deactivation. The cleanup keeps its
original signature and reads the failed sources the manager records for
the bundle, as Dag deactivation already does.

In multi-team deployments, team-filtered dashboards rely on the
team_name tag that every other per-bundle Dag processing metric carries,
so the per-bundle count of files awaiting a source now carries it too.
Starting a processor now requires the parse source, because the dispatch
loop always chooses it first and that choice also decides whether the
file must wait; the fallback bypassed that decision and its result was
discarded as stale in a running processor.

The Dag bundle guide now says that object-store bundles pause on every
refresh failure, that the retry delay applies only when the provider
reports an intact source or when listing or recording a successful
refresh fails, and that a processor started with --num-runs waits for a
paused bundle, which can otherwise look like a hang. Tests now cover
the pause warning, the failed sources reaching Dag deactivation, a
skip-refresh override on a bundle's first refresh, and a regular
duplicate of a running file.

The provider supports every Airflow 3 core, and cores from 3.0 onwards
can lack the refresh-error class, not only some 3.4 builds. The import
test patched an Airflow version the provider no longer reads, so its two
cases were identical, and its version-based skip kept it from running on
the older cores where the probe matters most.

The retry after a failed listing or state write is held for the
bundle's refresh interval rather than capped by it, and only a failed
listing keeps the earlier file list, so the caveat misdescribed both.

Pausing a bundle's parsing while its refresh fails changes what S3, GCS
and Azure Blob Storage bundle users saw before, and a processor started
with --num-runs can now wait for that refresh to succeed. Upgraders need
to read that before they upgrade.