Skip to content

Comments

[airflow]: extend names moved from core to provider (AIR303)#15159

Merged
MichaReiser merged 1 commit intoastral-sh:mainfrom
astronomer:extend-AIR303
Dec 27, 2024
Merged

[airflow]: extend names moved from core to provider (AIR303)#15159
MichaReiser merged 1 commit intoastral-sh:mainfrom
astronomer:extend-AIR303

Conversation

@Lee-W
Copy link
Contributor

@Lee-W Lee-W commented Dec 27, 2024

Summary

Many core Airflow features have been deprecated and moved to Airflow Providers since users might need to install an additional package (e.g., apache-airflow-provider-fab==1.0.0); a separate rule (AIR303) is created for this.

  • apache-airflow-providers-common-sql >= 1.0.0
    • airflow.hooks.dbapi.ConnectorProtocolairflow.providers.common.sql.hooks.sql.ConnectorProtocol
    • airflow.hooks.dbapi.DbApiHookairflow.providers.common.sql.hooks.sql.DbApiHook
  • apache-airflow-providers-cncf-kubernetes >= 7.4.0
    • airflow.executors.kubernetes_executor_types.ALL_NAMESPACESairflow.providers.cncf.kubernetes.executors.kubernetes_executor_types.ALL_NAMESPACES
    • airflow.executors.kubernetes_executor_types.POD_EXECUTOR_DONE_KEYairflow.providers.cncf.kubernetes.executors.kubernetes_executor_types.POD_EXECUTOR_DONE_KEY
  • apache-airflow-providers-celery >= 3.3.0
    • airflow.executors.celery_executor.appairflow.providers.celery.executors.celery_executor_utils.app
    • airflow.config_templates.default_celery.DEFAULT_CELERY_CONFIGairflow.providers.celery.executors.default_celery.DEFAULT_CELERY_CONFIG
    • airflow.executors.celery_kubernetes_executor.CeleryKubernetesExecutorairflow.providers.celery.executors.celery_kubernetes_executor
    • airflow.executors.celery_executor.CeleryExecutorairflow.providers.celery.executors.celery_executor.CeleryExecutor
  • apache-airflow-providers-apache-hive >= 1.0.0
    • airflow.hooks.hive_hooks.HIVE_QUEUE_PRIORITIESairflow.providers.apache.hive.hooks.hive.HIVE_QUEUE_PRIORITIES
  • apache-airflow-providers-apache-hive >= 5.1.0
    • airflow.macros.hive.closest_ds_partitionairflow.providers.apache.hive.macros.hive.closest_ds_partition
      airflow.executors.kubernetes_executor.KubernetesExecutor → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor.KubernetesE
  • apache-airflow-providers-cncf-kubernetes >= 7.4.0
    • airflow.executors.kubernetes_executor_types.ALL_NAMESPACES → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_types.ALL
    • airflow.executors.kubernetes_executor_types.POD_EXECUTOR_DONE_KEY → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_ty
    • airflow.executors.kubernetes_executor_utils.AirflowKubernetesScheduler → `airflow.providers.cncf.kubernetes.executors.kubernetes_execut
    • airflow.executors.kubernetes_executor_utils.KubernetesJobWatcher → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_uti
    • airflow.executors.kubernetes_executor_utils.ResourceVersion → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_utils.Re
    • airflow.executors.local_kubernetes_executor.LocalKubernetesExecutorairflow.providers.cncf.kubernetes.executors.LocalKubernetesExecu airflow.macros.hive.max_partitionairflow.providers.apache.hive.macros.hive.max_partition`
  • apache-airflow-providers-daskexecutor >= 1.0.0
    • airflow.executors.dask_executor.DaskExecutorairflow.providers.daskexecutor.executors.dask_executor.DaskExecutor

Test Plan

A test fixture has been included for the rule.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 27, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

* apache-airflow-providers-common-sql >= 1.0.0
    * `airflow.hooks.dbapi.ConnectorProtocol` → `airflow.providers.common.sql.hooks.sql.ConnectorProtocol`
    * `airflow.hooks.dbapi.DbApiHook` → `airflow.providers.common.sql.hooks.sql.DbApiHook`
* apache-airflow-providers-cncf-kubernetes >= 7.4.0
    * `airflow.executors.kubernetes_executor_types.ALL_NAMESPACES` → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_types.ALL_NAMESPACES`
    * `airflow.executors.kubernetes_executor_types.POD_EXECUTOR_DONE_KEY` → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_types.POD_EXECUTOR_DONE_KEY`
* apache-airflow-providers-celery == 3.3.0
    * `airflow.executors.celery_executor.app` → `airflow.providers.celery.executors.celery_executor_utils.app`
    * `airflow.config_templates.default_celery.DEFAULT_CELERY_CONFIG` → `airflow.providers.celery.executors.default_celery.DEFAULT_CELERY_CONFIG`
* apache-airflow-providers-apache-hive == 1.0.0
    * `airflow.hooks.hive_hooks.HIVE_QUEUE_PRIORITIES` → `airflow.providers.apache.hive.hooks.hive.HIVE_QUEUE_PRIORITIES`
* apache-airflow-providers-apache-hive == 5.1.0
    * `airflow.macros.hive.closest_ds_partition` → `airflow.providers.apache.hive.macros.hive.closest_ds_partition`
    * `airflow.macros.hive.max_partition` → `airflow.providers.apache.hive.macros.hive.max_partition`
@MichaReiser MichaReiser added rule Implementing or modifying a lint rule preview Related to preview mode features labels Dec 27, 2024
Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@MichaReiser MichaReiser merged commit 2288cc7 into astral-sh:main Dec 27, 2024
21 checks passed
dcreager added a commit that referenced this pull request Dec 30, 2024
* main:
  Add all PEP-585 names to UP006 rule (#5454)
  [`flake8-simplify`] More precise inference for dictionaries (`SIM300`) (#15164)
  `@no_type_check` support (#15122)
  Visit PEP 764 inline `TypedDict`s' keys as non-type-expressions (#15073)
  [red-knot] Add diagnostic for invalid unpacking (#15086)
  [`flake8-use-pathlib`] Catch redundant joins in `PTH201` and avoid syntax errors (#15177)
  Update Rust crate glob to v0.3.2 (#15185)
  Update astral-sh/setup-uv action to v5 (#15193)
  Update dependency mdformat-mkdocs to v4.1.1 (#15192)
  Update Rust crate serde_with to v3.12.0 (#15191)
  Update NPM Development dependencies (#15190)
  Update pre-commit hook rhysd/actionlint to v1.7.5 (#15189)
  Update Rust crate syn to v2.0.93 (#15188)
  Update Rust crate serde to v1.0.217 (#15187)
  Update Rust crate quote to v1.0.38 (#15186)
  Update Rust crate compact_str to v0.8.1 (#15184)
  [`flake8-type-checking`] Disable TC006 & TC007 in stub files (#15179)
  Test explicit shadowing involving `def`s (#15174)
  Fix typo in `NameImport.qualified_name` docstring (#15170)
  [airflow]: extend names moved from core to provider (AIR303) (#15159)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

preview Related to preview mode features rule Implementing or modifying a lint rule

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants