Skip to content

Conversation

@dimberman
Copy link
Contributor

@dimberman dimberman commented Dec 1, 2020

Adds a check that ensures that any hook that uses the
run, get_pandas_df or get_records functions does not import from the
base_hook

addresses #11039


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@dimberman dimberman linked an issue Dec 1, 2020 that may be closed by this pull request
@github-actions
Copy link

github-actions bot commented Dec 1, 2020

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

@github-actions github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Dec 1, 2020
Adds a check that ensures that any hook that uses the
run, get_pandas_df or get_records functions does not import from the
base_hook
get_records = check_get_records(child)
if get_records:
incorrect_implementations.append(get_records)
return incorrect_implementations
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rule only verifies custom hooks but not operators that may use BaseHook methods. Btw. will it work with hooks defined on PYTHONPATH or in plugins?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@turbaszek are there operators that inherit from BaseHook? What would an example operator be?

I think it should work with anything i the globals

@dimberman dimberman merged commit 2e1f813 into v1-10-stable Dec 11, 2020
@dimberman dimberman deleted the check-sql-hook branch December 11, 2020 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full tests needed We need to run full set of tests for this PR to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create UsingSQLFromBaseHookRule to ease upgrade to Airflow 2.0

3 participants