Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: apache/airflow
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: dag-processor-activation
Choose a base ref
...
head repository: apache/airflow
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: dag-processor-deployment
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 9 commits
  • 18 files changed
  • 1 contributor

Commits on Nov 11, 2026

  1. Add API-mode Dag processor deployment support

    API-mode processors need health probes that work without database
    credentials. Each processor session must also remain exclusive while
    pods are replaced, and isolated deployments need explicit credential
    and network boundaries around the trusted API server.
    ephraimbuddy committed Nov 11, 2026
    Configuration menu
    Copy the full SHA
    c337246 View commit details
    Browse the repository at this point in the history
  2. Rotate Dag processor session tokens in Kubernetes Secrets

    Dag processors using the Execution API need renewable credentials without access to signing keys. Kubernetes deployments also need to retain their session when the token publisher restarts or an API response is lost.
    ephraimbuddy committed Nov 11, 2026
    Configuration menu
    Copy the full SHA
    e8de12c View commit details
    Browse the repository at this point in the history
  3. Clarify isolated Dag processor credential and migration guidance

    Catalog synchronization and token provisioning have different credential
    requirements. Bundle migrations must also avoid having trusted and isolated
    processors reconcile different inventories for the same bundle.
    ephraimbuddy committed Nov 11, 2026
    Configuration menu
    Copy the full SHA
    1fc3ea9 View commit details
    Browse the repository at this point in the history
  4. Clarify the Dag processor rollout status check

    Deployment checks should name the command being described so operators
    can match the guidance to the example and distinguish process startup
    from continued token validity.
    ephraimbuddy committed Nov 11, 2026
    Configuration menu
    Copy the full SHA
    b5c4966 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    0a14701 View commit details
    Browse the repository at this point in the history
  6. Handle invalid input in the Dag processor token provisioner

    A token whose subject is not a string, written to the Secret by
    anything other than the provisioner, made the session lookup raise
    AttributeError. Kubernetes rotation then re-read that token on every
    iteration and never replaced it, so the processor's token eventually
    expired, and file mode crashed at startup the same way. The
    provisioner now treats such a token as having no session and starts a
    new one.
    
    A bad Secret name, namespace or API port, or a missing cluster CA,
    ended the command with a traceback, and the CA error did not say which
    file it tried to load. These now exit with a message that names the
    problem, including the CA path.
    
    The name-validation test could not fail: without the in-cluster
    environment variables, the store raised a different ValueError that
    the test's match also accepted. It now sets those variables and
    matches the DNS-label and DNS-subdomain messages.
    ephraimbuddy committed Nov 11, 2026
    Configuration menu
    Copy the full SHA
    92d1d33 View commit details
    Browse the repository at this point in the history
  7. Fix the isolated Dag processor deployment examples and chart checks

    The token provisioner example signed with the key in the chart's
    generated <release>-jwt-secret. Every helm upgrade of the trusted
    release writes a new key there and rolls the API server onto it, but
    the provisioner keeps the old key in its environment until it
    restarts, so the API server rejected every token it issued after an
    upgrade. The example now reads a pre-created Secret that the trusted
    release also uses as jwtSecretName.
    
    The processor-only Helm values set LocalExecutor just to pass the
    chart's validation, which also made the processor parse Dags with a
    different [core] executor from the trusted deployment. Tasks naming
    another executor then failed to import. The example now sets the
    processor's executor to match, and the guide says it must.
    
    The chart chooses probes, update strategy and the replica limit only
    from config.dag_processor.execution_api_token_file. Setting the token
    file through AIRFLOW__DAG_PROCESSOR__EXECUTION_API_TOKEN_FILE instead
    kept a liveness probe that fails without database credentials, so the
    chart now rejects that variable. The chart tests now also cover
    rendering without a token file, including a null config.dag_processor.
    
    The guide overstated how close to token expiry a Kubernetes API
    outage can end without stopping the processor, because it left out
    the retry backoff and the Secret projection delay. Its install command
    also lacked a chart version, and a few smaller statements were wrong.
    
    No CI job ran the Compose deployment example tests, so the quick-start
    Compose job now runs them.
    ephraimbuddy committed Nov 11, 2026
    Configuration menu
    Copy the full SHA
    0e10cab View commit details
    Browse the repository at this point in the history
  8. Select the chart's Dag processor API mode from use_execution_api

    The Dag processor now enters API mode only when [dag_processor]
    use_execution_api is enabled; execution_api_token_file is just the
    credential source. The chart still chose the local probes, the Recreate
    strategy and the one-replica limit from the token file, so a release
    that set only the token file ran a database-mode processor with
    API-mode probes. "airflow dag-processor --check-health" then exits
    because local health probes need API mode, and Kubernetes restarts the
    pod in a loop.
    
    A processor with API mode enabled and no token file fails at startup,
    so the chart refuses to render that combination instead of deploying a
    pod that cannot start. Setting AIRFLOW__DAG_PROCESSOR__USE_EXECUTION_API
    in env or dagProcessor.env would bypass the chart's API-mode settings
    the same way the token-file variable did, so the chart rejects it too.
    
    The quick-start Compose healthcheck tested the token file in the same
    way, so it now follows use_execution_api as well.
    ephraimbuddy committed Nov 11, 2026
    Configuration menu
    Copy the full SHA
    1a65e04 View commit details
    Browse the repository at this point in the history
  9. Keep Dag processor deployment probes consistent with API mode

    Airflow accepts whitespace and inline comments in boolean configuration.
    Deployment selectors must recognize the same values so processors receive
    matching probes, rollout behavior and session-token safeguards.
    ephraimbuddy committed Nov 11, 2026
    Configuration menu
    Copy the full SHA
    91bbfd1 View commit details
    Browse the repository at this point in the history
Loading