Skip to content

Commit b4b5417

Browse files
authored
Celery: explicitly specify psycopg2 in driverless postgres URLs (#62159)
To use the v3 driver, users should specify it explicitly: `postgresql+psycopg://...`
1 parent 2851c74 commit b4b5417

3 files changed

Lines changed: 7 additions & 3 deletions

File tree

‎airflow-core/docs/howto/docker-compose/docker-compose.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ x-airflow-common:
5858
AIRFLOW__CORE__EXECUTOR: CeleryExecutor
5959
AIRFLOW__CORE__AUTH_MANAGER: airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager
6060
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres/airflow
61-
AIRFLOW__CELERY__RESULT_BACKEND: db+postgresql://airflow:airflow@postgres/airflow
61+
AIRFLOW__CELERY__RESULT_BACKEND: db+postgresql+psycopg2://airflow:airflow@postgres/airflow
6262
AIRFLOW__CELERY__BROKER_URL: redis://:@redis:6379/0
6363
AIRFLOW__CORE__FERNET_KEY: ${FERNET_KEY}
6464
AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION: 'true'

‎providers/celery/src/airflow/providers/celery/executors/default_celery.py‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,11 @@ def get_default_celery_config(team_conf) -> dict[str, Any]:
7272
result_backend = team_conf.get_mandatory_value("celery", "RESULT_BACKEND")
7373
else:
7474
log.debug("Value for celery result_backend not found. Using sql_alchemy_conn with db+ prefix.")
75-
result_backend = f"db+{team_conf.get('database', 'SQL_ALCHEMY_CONN')}"
75+
sql_alchemy_conn = team_conf.get("database", "SQL_ALCHEMY_CONN")
76+
# In SQLAlchemy 2.1 the default PostgreSQL driver changed from psycopg2 to psycopg (v3).
77+
# To maintain existing behavior, we explicitly specify psycopg2 for driverless PostgreSQL URLs.
78+
sql_alchemy_conn = sql_alchemy_conn.replace("postgresql://", "postgresql+psycopg2://", 1)
79+
result_backend = f"db+{sql_alchemy_conn}"
7680

7781
# Handle result backend transport options (for Redis Sentinel support)
7882
result_backend_transport_options: dict = (

‎scripts/ci/docker-compose/backend-postgres.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ services:
2020
environment:
2121
- BACKEND=postgres
2222
- AIRFLOW__DATABASE__SQL_ALCHEMY_CONN=postgresql+psycopg2://postgres:airflow@postgres/airflow
23-
- AIRFLOW__CELERY__RESULT_BACKEND=db+postgresql://postgres:airflow@postgres/airflow
23+
- AIRFLOW__CELERY__RESULT_BACKEND=db+postgresql+psycopg2://postgres:airflow@postgres/airflow
2424
depends_on:
2525
postgres:
2626
condition: service_healthy

0 commit comments

Comments
 (0)