[AIRFLOW-6721] Organize hive tests #7468
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Feb 19, 2020
Move the WebHdfsSensor tests out of the operator tests and into a separate module.
Movee the TestHiveEnvironment class and setUp method out to the hive test module so that it can be used in other hive tests.
Move skipped test for HiveStatsCollectionOperator to the test_hive_stats module to make it easier to locate.
Move the NamedHivePartitionSensor tests from the operators module into the test module for NamedHivePartitionSensor.
Move the HivePartitionSensor tests from the operators module into the a new test module.
Move the HiveToMySqlTransfer test in the operators module into the existing test module and use the shared test environment for these tests.
Move the HdfsSensor test in the operators module into a new test module and use the shared test environment.
Move the Hive2SambaOperator test in the operators module into the existing test module and use the shared test environment.
Move the MetastorePartitionSensor test from the operators module into a new module and use the shared test environment.
Move the PrestoToMySqlTransfer test from the operators module into an existing test module and use shared Hive test environment.
Move the PrestoCheckOperator test from the Hive operators module into a new Presto operators test module while still using the shared Hive test environment.
Move the SqlSensor in the Hive operators module into the existing SqlSensor test module and label it as such. Use the shared test environment for all tests, ensuring that original setUp properties are preserved for the existing tests.
coopergillan
force-pushed
the
AIRFLOW-6721-reorganize-hive-tests
branch
from
1495461 to
edc415d
Compare
Codecov Report
@@ Coverage Diff @@
## master #7468 +/- ##
==========================================
- Coverage 86.68% 86.22% -0.46%
==========================================
Files 882 882
Lines 41526 41526
==========================================
- Hits 35997 35806 -191
- Misses 5529 5720 +191
Continue to review full report at Codecov.
|
Member
|
Can you create a ticket about |
Contributor
Author
galuszkak
pushed a commit
to FlyrInc/apache-airflow
that referenced
this pull request
Mar 5, 2020
* [AIRFLOW-6721] Move WebHdfsSensor tests to own module Move the WebHdfsSensor tests out of the operator tests and into a separate module. * [AIRFLOW-6721] Move TestHiveEnvironment to test module Movee the TestHiveEnvironment class and setUp method out to the hive test module so that it can be used in other hive tests. * [AIRFLOW-6721] Move Hive Stats tests to separate module Move skipped test for HiveStatsCollectionOperator to the test_hive_stats module to make it easier to locate. * [AIRFLOW-6721] Move NamedHivePartitionSensor tests Move the NamedHivePartitionSensor tests from the operators module into the test module for NamedHivePartitionSensor. * [AIRFLOW-6721] Move HivePartitionSensor tests Move the HivePartitionSensor tests from the operators module into the a new test module. * [AIRFLOW-6721] Move HiveToMySqlTransfer test Move the HiveToMySqlTransfer test in the operators module into the existing test module and use the shared test environment for these tests. * [AIRFLOW-6721] Move HdfsSensor test to separate module Move the HdfsSensor test in the operators module into a new test module and use the shared test environment. * [AIRFLOW-6721] Move Hive2SambaOperator test Move the Hive2SambaOperator test in the operators module into the existing test module and use the shared test environment. * [AIRFLOW-6721] Move Hive MetastorePartitionSensor test Move the MetastorePartitionSensor test from the operators module into a new module and use the shared test environment. * [AIRFLOW-6721] Move Hive PrestoToMySqlTransfer test Move the PrestoToMySqlTransfer test from the operators module into an existing test module and use shared Hive test environment. * [AIRFLOW-6721] Move PrestoCheckOperator test Move the PrestoCheckOperator test from the Hive operators module into a new Presto operators test module while still using the shared Hive test environment. * [AIRFLOW-6721] Move Hive SqlSensor test Move the SqlSensor in the Hive operators module into the existing SqlSensor test module and label it as such. Use the shared test environment for all tests, ensuring that original setUp properties are preserved for the existing tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
BEST REVIEWED COMMIT-BY-COMMIT
Problem
Currently many different sensors and operators are tested in the Hive operators module. Some of these operators already have test modules and others do not. This was originally called out by @mik-lag in #7316.
Solution
TestHiveEnvironmentclass fromunittest.TestCasethat each of the test modules can usetest_sql_sensor.pyIssue link: AIRFLOW-6721
Make sure to mark the boxes below before creating PR: [x]
[AIRFLOW-NNNN]. AIRFLOW-NNNN = JIRA ID** For document-only changes commit message can start with
[AIRFLOW-XXXX].In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.