Skip to content

[Airflow 3] Keycloak Auth Manager: 401 Redirect Loop due to Missing 'kid' in Internal JWT Header #62876

Description

@slloyd09

Apache Airflow version

3.1.7

If "Other Airflow 3 version" selected, which one?

No response

What happened?

When using the KeycloakAuthManager in Airflow 3, the authentication flow enters an infinite redirect loop. After a successful Keycloak login, the login_callback correctly receives the code and user info, but the internal session JWT generated by Airflow to provide to the browser is rejected by the FastAPI security middleware.

The logs consistently show:
[error] JWT token is not valid: Missing 'kid' in token header [airflow.api_fastapi.auth.managers.base_auth_manager]

Attempted Workarounds:

  • Symmetric (HS256): Providing AIRFLOW__API_AUTH__JWT_SECRET (singular) or AIRFLOW__API_AUTH__JWT_SECRETS (plural JSON). The resulting token lacks the kid header, which the FastAPI validator strictly requires.
  • Asymmetric (RS256) with Public Key: Resulted in jwt.exceptions.InvalidKeyError because the login_callback attempted to sign the internal token using the public key instead of the private key.
  • Asymmetric (RS256) with Private Key: Even with a valid RSA Private Key provided, the generated token header still omitted the kid, leading back to the validation failure.

What you think should happen instead?

The KeycloakAuthManager should correctly initialize its internal Signer using the configured jwt_key_id. When generating the session JWT during the login_callback phase, it must inject the kid into the JWT header to satisfy the JWTValidator used by the FastAPI framework in Airflow 3.

How to reproduce

  1. Configure Airflow 3 with AIRFLOW__CORE__AUTH_MANAGER set to KeycloakAuthManager.
  2. Provide AIRFLOW__API_AUTH__TRUSTED_JWKS_URL for the Keycloak realm.
  3. Set AIRFLOW__API_AUTH__JWT_ALGORITHM to HS256 and provide a JWT_SECRET.
  4. Attempt to log in via the UI.
  5. Observe the 303 redirect from login_callback followed by immediate 401 Unauthorized errors on /ui/config and /ui/auth/menus due to the missing kid header in the airflow_jwt_token cookie.

Operating System

Linux (Running inside official Airflow Docker images/Kubernetes)

Versions of Apache Airflow Providers

  • Apache Airflow Core: 3.0.0 (Development/Alpha)
  • apache-airflow-providers-keycloak: 3.1.7 (Tag: providers-keycloak/3.1.7)
  • Auth Manager Class: airflow.providers.keycloak.auth_manager.keycloak_auth_manager.KeycloakAuthManager
  • Keycloak Server: 26.5.0

Deployment

Official Apache Airflow Helm Chart

Deployment details

  • Deployment Method: Helm Chart (Official)
  • Orchestration: Kubernetes
  • Proxy: Nginx Ingress Controller (TLS Terminated)
  • Database: Postgres

Anything else?

We verified via airflow config get-value that AIRFLOW__API_FASTAPI__AUTH_JWT_KEY_ID and AIRFLOW__KEYCLOAK_AUTH_MANAGER__JWT_KEY_ID were correctly mapped within the pod. However, the generate_jwt method in base_auth_manager.py appears to ignore these settings during Signer initialization, or the Signer itself is failing to include the kid when using symmetric encryption.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions