Why

Python 3.14 changed the Unix (non-macOS) default multiprocessing start method from fork to forkserver (python/cpython#84559).

• forkserver/spawn children re-import Airflow instead of sharing it copy-on-write, so the scheduler's LocalExecutor workers and the triggerer each pay for a private copy.
• Benchmarking the upgrade from Airflow 3.2.2 / Python 3.13.14 (fork default) to 3.3.0b1 / Python 3.14.6 (forkserver default) measured steady-state PSS (Proportional Set Size, the copy-on-write-aware resident memory metric) up +96% for the scheduler and +90% for the triggerer; setting mp_start_method = fork for the scheduler and triggerer on the new stack brings them back to +17% / +20% of that baseline.

This restores control over the start method so deployments can recover the pre-3.14 footprint. It is opt-in: unset keeps the platform default, so there is no behaviour change unless configured.

What

• Add [core] mp_start_method (overridable per component via [scheduler], [triggerer] or [dag_processor]) and [core] mp_forkserver_preload.
• Add resolve_mp_start_method / set_component_mp_start_method to process_utils; the latter applies the configured method and forkserver preload modules, and is a safe no-op when unset or when the method is unavailable on the platform.
• Call set_component_mp_start_method from the scheduler, triggerer and dag-processor CLI entrypoints, before any multiprocessing work.
• Resolve LocalExecutor.is_mp_using_fork per instance instead of once at import, so it honours a start method set by the CLI before the executor is created.

Note: [core] mp_start_method existed historically but was removed (functional part in #48817, config remnants in #61150) and only ever governed the pre-3.0 Dag-processing manager, never LocalExecutor. The version restored here is wired into the executor and component start paths so it actually controls the workers.

Scheduler / triggerer memory benchmark — patch-mode summary

Modes: baseline (no patch) | gc (periodic gc.collect) | fspreload (forkserver + set_forkserver_preload) | fork (set_start_method fork) | gc_fspreload / gc_fork (combinations) | forkserver (forkserver, no preload).

Permutation setup & effective start method

Reference baseline (Δ denominator): 3.2.2 / py3.13.14 (baseline).

• forkserver preload list: airflow,airflow.models,airflow.executors.local_executor,airflow.sdk,sqlalchemy,pydantic

Steady-state PSS (MB), Δ vs the reference baseline

Per-stack mitigation (Δ vs each stack's own baseline)

The headline table above is vs the pre-regression reference. These pivots instead show each patch mode vs that same stack's baseline column, i.e. the mitigation effect on the PR stack itself (fork/fspreload/forkserver vs the unset run on 3.3.0b1/3.14.6).

How to read this

• sched/trig live must equal the configured method (fork/forkserver) and dfp live must stay at the platform default — that proves the per-component config took effect only where set.
• In the headline table, fork should pull the scheduler & triggerer back near the 3.2.2/3.13.14 reference (Δ ~0), while the unset/forkserver runs stay well above it (the forkserver regression); fspreload lands in between (helps the scheduler, less the triggerer).
• The dag-processor PSS should be flat across every run (raw os.fork, untouched).

Was generative AI tooling used to co-author this PR?

• Yes, with help of Claude Code (Opus 4.8) following the guidelines