[v3-1-test] fix(api): allow null dag_run_conf in BackfillResponse serialization (#63259)#63799
[v3-1-test] fix(api): allow null dag_run_conf in BackfillResponse serialization (#63259)#63799github-actions[bot] wants to merge 85 commits into
Conversation
…#59801) (#62979) Cherry-picked from 3428dc9 with conflict resolution: - context.py: Added `import inspect` (skip `import functools` as `from functools import cache` already exists) - supervisor.py: Adopted early-return pattern and explicit `del` for GC, kept simpler env var handling (no `_AIRFLOW_PROCESS_CONTEXT` which doesn't exist in v3-1-test) - test_supervisor.py: Replaced `@pytest.mark.xfail` workaround with proper `use_real_secrets_backends` fixture Co-authored-by: Claude Opus 4.6 <[email protected]>
…ctory with 3 updates (#62968) Bumps the core-ui-package-updates group with 1 update in the /airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui directory: [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin). Updates `@typescript-eslint/eslint-plugin` from 8.50.0 to 8.56.1 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.56.1/packages/eslint-plugin) Updates `@typescript-eslint/parser` from 8.50.0 to 8.56.1 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.56.1/packages/parser) Updates `ts-morph` from 23.0.0 to 27.0.2 - [Release notes](https://github.com/dsherret/ts-morph/releases) - [Commits](dsherret/ts-morph@23.0.0...27.0.2) --- updated-dependencies: - dependency-name: "@typescript-eslint/eslint-plugin" dependency-version: 8.56.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: "@typescript-eslint/parser" dependency-version: 8.56.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: ts-morph dependency-version: 27.0.2 dependency-type: direct:development update-type: version-update:semver-major dependency-group: core-ui-package-updates ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…torical retry attempts (#62475) (#63000) * fix: gracefully handle 404 from worker log server for historical retry attempts (cherry picked from commit 25e9284) Co-authored-by: Pradeep Kalluri <[email protected]>
…PR template (#63006) (#63012) The CI workflow added in #62975 validates that newsfragment filenames use the PR number, so allowing issue numbers would cause false CI failures. Align the PR template with the contributing docs and the new validation. (cherry picked from commit 41969f4) Co-authored-by: Xiaodong DENG <[email protected]>
* [v3-1-test] CI: Upgrade important CI environment (#62989) (cherry picked from commit 60b52b7) Co-authored-by: Jarek Potiuk <[email protected]> * Fix rebase --------- Co-authored-by: Jarek Potiuk <[email protected]>
…lStack config (#62980) (#62993) (cherry picked from commit 6722c4b) Co-authored-by: Jason(Zhe-You) Liu <[email protected]>
… installation (#63051) (#63057) When building PROD from docker-context-files - i.e. when we run main build with providers built from sources, we should fall back to no constraints build when there is a conflict with constraints. This is a follow up after #62378 (cherry picked from commit fef2e62) Co-authored-by: Jarek Potiuk <[email protected]>
* Remove global from FastAPI app.py * Remove global from FastAPI app.py Co-authored-by: Jens Scheffler <[email protected]>
Bumps [actions/setup-java](https://github.com/actions/setup-java) from 4.7.1 to 5.2.0. - [Release notes](https://github.com/actions/setup-java/releases) - [Commits](actions/setup-java@c5195ef...be666c2) --- updated-dependencies: - dependency-name: actions/setup-java dependency-version: 5.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/stale](https://github.com/actions/stale) from 9.1.0 to 10.2.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@5bef64f...b5d41d4) --- updated-dependencies: - dependency-name: actions/stale dependency-version: 10.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.2 to 6.0.2. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@11bd719...de0fac2) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 6.0.2 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.6.0 to 6.2.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@a26af69...a309ff8) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: 6.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/github-script](https://github.com/actions/github-script) from 7.0.1 to 8.0.0. - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@60a0d83...ed59741) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 8.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…62889) (#63007) * Ignore major npm dependabot upgrades * Move major versions to different group (cherry picked from commit a2e3613) Co-authored-by: Brent Bovenzi <[email protected]>
…y with 2 updates (#63069) Bumps the core-ui-package-updates group with 2 updates in the /airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui directory: [@chakra-ui/react](https://github.com/chakra-ui/chakra-ui/tree/HEAD/packages/react) and [happy-dom](https://github.com/capricorn86/happy-dom). Updates `@chakra-ui/react` from 3.33.0 to 3.34.0 - [Release notes](https://github.com/chakra-ui/chakra-ui/releases) - [Changelog](https://github.com/chakra-ui/chakra-ui/blob/main/packages/react/CHANGELOG.md) - [Commits](https://github.com/chakra-ui/chakra-ui/commits/@chakra-ui/[email protected]/packages/react) Updates `happy-dom` from 20.7.0 to 20.8.3 - [Release notes](https://github.com/capricorn86/happy-dom/releases) - [Commits](capricorn86/happy-dom@v20.7.0...v20.8.3) --- updated-dependencies: - dependency-name: "@chakra-ui/react" dependency-version: 3.34.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: happy-dom dependency-version: 20.8.3 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.6.2 to 7.0.0. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@ea165f8...bbbca2d) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-version: 7.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.5.0 to 6.3.0. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@d35c59a...4b73464) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials) from 4.0.1 to 6.0.0. - [Release notes](https://github.com/aws-actions/configure-aws-credentials/releases) - [Changelog](https://github.com/aws-actions/configure-aws-credentials/blob/main/CHANGELOG.md) - [Commits](aws-actions/configure-aws-credentials@010d0da...8df5847) --- updated-dependencies: - dependency-name: aws-actions/configure-aws-credentials dependency-version: 6.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4.4.0 to 6.3.0. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@49933ea...53b8394) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…63133) (#63138) Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.5.0 to 6.3.0. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@d35c59a...4b73464) (cherry picked from commit 274c2b0) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… 8.0.0 (#63065) (#63145) Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4.3.0 to 8.0.0. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@d3f86a1...70fc10c) (cherry picked from commit 2f4646c) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: 8.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…3157) Bumps the github-actions-updates group with 2 updates: [pnpm/action-setup](https://github.com/pnpm/action-setup) and [slackapi/slack-github-action](https://github.com/slackapi/slack-github-action). Updates `pnpm/action-setup` from 4.0.0 to 4.2.0 - [Release notes](https://github.com/pnpm/action-setup/releases) - [Commits](pnpm/action-setup@fe02b34...41ff726) Updates `slackapi/slack-github-action` from 2.0.0 to 2.1.1 - [Release notes](https://github.com/slackapi/slack-github-action/releases) - [Commits](slackapi/slack-github-action@485a9d4...91efab1) --- updated-dependencies: - dependency-name: pnpm/action-setup dependency-version: 4.2.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions-updates - dependency-name: slackapi/slack-github-action dependency-version: 2.1.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…duce data fetched per task instance (#62482) (#62996) * perf: use load_only() in eager_load_dag_run_for_validation to reduce data fetched The get_dag_runs API endpoint was slow on large deployments because eager_load_dag_run_for_validation() used selectinload on task_instances and task_instances_histories without restricting which columns were fetched. This caused SQLAlchemy to load all heavyweight columns (executor_config with pickled data, hostname, rendered fields, etc.) for every task instance across every DAG run in the result page — even though only dag_version_id is needed to traverse the association proxy to DagVersion. Add load_only(TaskInstance.dag_version_id) and load_only(TaskInstanceHistory.dag_version_id) to the selectinload chains so the SELECT for task instances fetches only the identity columns and the FK needed to resolve the dag_version relationship, significantly reducing the volume of data transferred from the database on busy deployments. Fixes #62025 * Fix static checks --------- (cherry picked from commit 13af96b) Co-authored-by: Lakshmi Sravya <[email protected]> Co-authored-by: pierrejeambrun <[email protected]>
This PR addresses a significant performance issue in the /ui/dags endpoint
where page load times scaled poorly with the number of DAGs (12-16 seconds
for just 25 DAGs in our testing).
Two optimizations are implemented:
1. Cache URLSafeSerializer for file_token generation
- Previously, a new URLSafeSerializer was instantiated and
conf.get_mandatory_value() was called for every DAG
- Now uses @lru_cache to create the serializer once and reuse it
2. Eliminate redundant Pydantic validation in response construction
- The original pattern used model_validate -> model_dump -> model_validate
which caused triple serialization overhead per DAG
- Now validates once with DAGResponse.model_validate(), then uses
model_construct() to build DAGWithLatestDagRunsResponse
Together, these changes reduced page load time from 12-16 seconds to
~130ms in our dev environment.
(cherry picked from commit a915216)
Co-authored-by: john-rodriguez-mgni <[email protected]>
Co-authored-by: Cursor <[email protected]>
(cherry picked from commit 0a985ea) Co-authored-by: Elad Kalif <[email protected]>
…y with 2 updates (#63153) Bumps the core-ui-package-updates group with 1 update in the /airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui directory: [@chakra-ui/react](https://github.com/chakra-ui/chakra-ui/tree/HEAD/packages/react). Updates `@chakra-ui/react` from 3.33.0 to 3.34.0 - [Release notes](https://github.com/chakra-ui/chakra-ui/releases) - [Changelog](https://github.com/chakra-ui/chakra-ui/blob/main/packages/react/CHANGELOG.md) - [Commits](https://github.com/chakra-ui/chakra-ui/commits/@chakra-ui/[email protected]/packages/react) Updates `happy-dom` from 20.7.0 to 20.8.3 - [Release notes](https://github.com/capricorn86/happy-dom/releases) - [Commits](capricorn86/happy-dom@v20.7.0...v20.8.3) --- updated-dependencies: - dependency-name: "@chakra-ui/react" dependency-version: 3.34.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: happy-dom dependency-version: 20.8.3 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…redoc fixes it) (#63202) (#63203) (cherry picked from commit b528e50) Co-authored-by: Jarek Potiuk <[email protected]>
FAB FastAPI routes call get_application_builder() on every request, which creates a new Flask app and invokes init_app(). Concurrent calls race on the singleton auth_manager's appbuilder and security_manager, causing KeyError: 'AUTH_USER_REGISTRATION' and AttributeError. Add _init_app_lock around the critical section in init_app() that mutates the singleton auth_manager state and registers views, so concurrent get_application_builder() calls are serialized. Co-authored-by: Young-Ki Kim <[email protected]>
…e objects (#62656) (#63721) * Fix structlog JSON serialization crash on non-serializable objects Wrap the enc_hook in json_dumps() with a safe_default() fallback that catches TypeError and falls back to str(). This prevents the logging pipeline from crashing when log event dicts contain objects that msgspec cannot serialize. Closes: #62472 Closes: #62201 * Address review: hoist safe_default into factory, add tests - Move the inline safe_default closure out of json_dumps into a module-level _make_safe_enc_hook factory function to avoid recreating it on every JSON log line. - Add test for the actual crash scenario: an object whose __structlog__() raises TypeError now falls back to str() instead of crashing the logger. - Add regression test confirming normal custom objects still serialize via repr() through the standard enc_hook path. - Add unit test for the default=None edge case in _make_safe_enc_hook. * Handle UnicodeEncodeError in structlog JSON serialization Broaden exception handling to catch ValueError (including UnicodeEncodeError) in the enc_hook, and add a fallback around msgspec.json.encode() to replace surrogate characters that can't be encoded to UTF-8 JSON. --------- (cherry picked from commit 0b43077) Co-authored-by: deepinsight coder <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]>
…ialization (#63259) (cherry picked from commit bc1bf43) Co-authored-by: Yoann <[email protected]> Co-authored-by: Wei Lee <[email protected]>
|
@github-actions rebase |
|
@github-actions This PR has been converted to draft because it does not yet meet our Pull Request quality criteria. Issues found:
What to do next:
Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the Airflow Slack. Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you. |
61e8f26 to
6fb8f94
Compare
|
This pull request has been converted to draft due to quality issues more than a week ago and there has been no response from the author since then. We are closing it for now to keep the review queue manageable. @github-actions, you are welcome to reopen this PR after addressing the review comments. Thank you for your contribution! |
(cherry picked from commit bc1bf43)
Co-authored-by: Yoann [email protected]
Co-authored-by: Wei Lee [email protected]