Skip to content

Commit 0a4b5d8

Browse files
github-actions[bot]Lee-Wpotiuk
authored
[main] Upgrade important CI environment (#73822)
* [main] CI: Upgrade important CI environment * Fix "prek run update-providers-build-files --all-files" * Fix CI breakages from the dependency upgrade - snowflake: snowflake-sqlalchemy (up to 1.11.1) cannot be imported on SQLAlchemy 2.1, which made ORMSelectCompileState private. Cap sqlalchemy<2.1 in the provider; the workspace lock resolves back to SQLAlchemy 2.0.54. Removing the cap is tracked in #75118. - airflow-core: type the dag_display_name and duration hybrid expressions against the class (type[...] -> ColumnElement) and fix _get_dialect's return type, so mypy holds on SQLAlchemy 2.0 and 2.1. - common.ai: declare the OpenAI Decisions and System One vendors added in pydantic-ai 2.53/2.55; reject sqlglot >= 30.22's exp.ListStage (Snowflake LIST/LS @stage) on the allow_writes path; adapt tests to pydantic-ai's streamed Anthropic requests and new request counting, and to sqlglot's MySQL "--+" tokenization. Generated-by: Claude Opus 5 * Type Backfill.duration expression against the class #75078 dropped the type: ignores in this expression, which only held while mypy did not check the body; typing cls as type[Backfill] keeps it clean on SQLAlchemy 2.0 and 2.1, as dag_display_name and DagRun.duration already are. Also annotate the Anthropic stream fixture so mypy can index its events. Generated-by: Claude Opus 5 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Wei Lee <[email protected]> Co-authored-by: Jarek Potiuk <[email protected]>
1 parent bd29fed commit 0a4b5d8

52 files changed

Lines changed: 4969 additions & 4082 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.pre-commit-config.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -721,7 +721,7 @@ repos:
721721
require_serial: true
722722
entry: zizmor
723723
- repo: https://github.com/lycheeverse/lychee
724-
rev: 2c1fb3ad1373b1eb5148176bb6dcd4773f2a564d # frozen: nightly
724+
rev: 54f8bf47ffbd35973b8eb016ccbc67221c75ba5f # frozen: nightly
725725
hooks:
726726
# Use the upstream `lychee-docker` variant rather than the script-based
727727
# `lychee` hook because the prebuilt lychee binaries that the script
@@ -821,7 +821,7 @@ repos:
821821
types_or: [python, pyi]
822822
args: [--fix]
823823
require_serial: true
824-
additional_dependencies: ['ruff==0.16.8']
824+
additional_dependencies: ['ruff==0.17.0']
825825
exclude: ^airflow-core/tests/unit/dags/test_imports\.py$|^performance/tests/test_.*\.py$
826826
- id: ruff-format
827827
name: Run 'ruff format'

‎Dockerfile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ ARG AIRFLOW_USER_HOME_DIR=/home/airflow
5151
ARG AIRFLOW_VERSION="3.3.2"
5252

5353
ARG BASE_IMAGE="debian:bookworm-slim"
54-
ARG AIRFLOW_PYTHON_VERSION="3.13.15"
54+
ARG AIRFLOW_PYTHON_VERSION="3.13.16"
5555

5656
# PYTHON_LTO: Controls whether Python is built with Link-Time Optimization (LTO).
5757
#
@@ -73,7 +73,7 @@ ARG PYTHON_LTO="true"
7373
# Also use `force pip` label on your PR to swap all places we use `uv` to `pip`
7474
ARG AIRFLOW_PIP_VERSION=26.2.1
7575
# ARG AIRFLOW_PIP_VERSION="git+https://github.com/pypa/pip.git@main"
76-
ARG AIRFLOW_UV_VERSION=0.12.18
76+
ARG AIRFLOW_UV_VERSION=0.13.1
7777
ARG AIRFLOW_USE_UV="false"
7878
ARG AIRFLOW_IMAGE_REPOSITORY="https://github.com/apache/airflow"
7979
ARG AIRFLOW_IMAGE_README_URL="https://raw.githubusercontent.com/apache/airflow/main/docs/docker-stack/README.md"

‎Dockerfile.ci‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1756,9 +1756,9 @@ ENV DEV_APT_COMMAND=${DEV_APT_COMMAND} \
17561756
ADDITIONAL_DEV_APT_COMMAND=${ADDITIONAL_DEV_APT_COMMAND}
17571757

17581758

1759-
ARG AIRFLOW_PYTHON_VERSION="3.13.15"
1759+
ARG AIRFLOW_PYTHON_VERSION="3.13.16"
17601760
ENV AIRFLOW_PYTHON_VERSION=${AIRFLOW_PYTHON_VERSION}
1761-
ENV GOLANG_MAJOR_MINOR_VERSION="1.27.1"
1761+
ENV GOLANG_MAJOR_MINOR_VERSION="1.27.2"
17621762
ENV TEMURIN_VERSION="11"
17631763
ENV NODEJS_VERSION="22.23.1"
17641764
# keep this in sync with package.json files
@@ -1920,8 +1920,8 @@ COPY --from=scripts common.sh install_packaging_tools.sh install_additional_depe
19201920
# Also use `force pip` label on your PR to swap all places we use `uv` to `pip`
19211921
ARG AIRFLOW_PIP_VERSION=26.2.1
19221922
# ARG AIRFLOW_PIP_VERSION="git+https://github.com/pypa/pip.git@main"
1923-
ARG AIRFLOW_UV_VERSION=0.12.18
1924-
ARG AIRFLOW_PREK_VERSION="0.5.3"
1923+
ARG AIRFLOW_UV_VERSION=0.13.1
1924+
ARG AIRFLOW_PREK_VERSION="0.5.5"
19251925

19261926
# UV_LINK_MODE=copy is needed since we are using cache mounted from the host
19271927
ENV AIRFLOW_PIP_VERSION=${AIRFLOW_PIP_VERSION} \

‎airflow-core/docs/best-practices.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -319,7 +319,7 @@ Installing and Using ruff
319319

320320
.. code-block:: bash
321321
322-
pip install "ruff>=0.16.8"
322+
pip install "ruff>=0.17.0"
323323
324324
2. **Running ruff**: Execute ``ruff`` to check your Dags for potential issues:
325325

‎airflow-core/pyproject.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
[build-system]
1919
requires = [
2020
"gitdb==4.0.12",
21-
"GitPython==3.1.62",
21+
"GitPython==3.2.1",
2222
"hatchling==1.32.4",
2323
"packaging==26.3",
2424
"pathspec==1.1.1",

‎airflow-core/src/airflow/models/backfill.py‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@
6262

6363
if TYPE_CHECKING:
6464
from sqlalchemy.orm import Session
65+
from sqlalchemy.sql.elements import ColumnElement
6566

6667
from airflow.models.dagrun import DagRun
6768
from airflow.serialization.definitions.dag import SerializedDAG
@@ -218,9 +219,9 @@ def duration(self) -> float | None:
218219
return None
219220

220221
@duration.expression # type: ignore[no-redef]
221-
def duration(cls) -> Any:
222+
def duration(cls: type[Backfill]) -> ColumnElement[Any]:
222223
@provide_session
223-
def _get_dialect(*, session: Session = NEW_SESSION) -> str:
224+
def _get_dialect(*, session: Session = NEW_SESSION) -> str | None:
224225
return get_dialect_name(session=session)
225226

226227
dialect_name = _get_dialect()

‎airflow-core/src/airflow/models/dag.py‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,7 @@
8383

8484
from dateutil.relativedelta import relativedelta
8585
from sqlalchemy.orm.state import InstanceState
86+
from sqlalchemy.sql.elements import ColumnElement
8687

8788
from airflow.sdk import Context
8889
from airflow.serialization.definitions.assets import (
@@ -650,15 +651,15 @@ def dag_display_name(self) -> str:
650651
return self._dag_display_property_value or self.dag_id
651652

652653
@dag_display_name.expression # type: ignore[no-redef]
653-
def dag_display_name(self) -> str:
654+
def dag_display_name(cls: type[DagModel]) -> ColumnElement[Any]:
654655
"""
655656
Expression part of the ``dag_display`` name hybrid property.
656657
657658
:meta private:
658659
"""
659660
return case(
660-
(self._dag_display_property_value.is_not(None), self._dag_display_property_value),
661-
else_=self.dag_id,
661+
(cls._dag_display_property_value.is_not(None), cls._dag_display_property_value),
662+
else_=cls.dag_id,
662663
)
663664

664665
@classmethod

‎airflow-core/src/airflow/models/dagrun.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@
122122
from pydantic import NonNegativeInt
123123
from sqlalchemy.engine import ScalarResult
124124
from sqlalchemy.orm import Session
125-
from sqlalchemy.sql.elements import Case, ColumnElement
125+
from sqlalchemy.sql.elements import ColumnElement
126126
from sqlalchemy.sql.selectable import Select
127127

128128
from airflow.api_fastapi.execution_api.datamodels.taskinstance import (
@@ -585,7 +585,7 @@ def duration(self) -> float | None:
585585

586586
@duration.expression # type: ignore[no-redef]
587587
@provide_session
588-
def duration(cls, *, session: Session = NEW_SESSION) -> Case:
588+
def duration(cls: type[DagRun], *, session: Session = NEW_SESSION) -> ColumnElement[Any]:
589589
dialect_name = get_dialect_name(session)
590590
if dialect_name == "mysql":
591591
return func.timestampdiff(text("SECOND"), cls.start_date, cls.end_date)

‎airflow-ctl/src/airflowctl/api/datamodels/auth_generated.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# generated by datamodel-codegen:
22
# filename: v2-simple-auth-manager-generated.yaml
3-
# version: 0.82.0
3+
# version: 0.83.0
44

55
from __future__ import annotations
66

‎airflow-ctl/src/airflowctl/api/datamodels/generated.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# generated by datamodel-codegen:
22
# filename: v2-rest-api-generated.yaml
3-
# version: 0.82.0
3+
# version: 0.83.0
44

55
from __future__ import annotations
66

0 commit comments

Comments
 (0)