Selective tests - depends on files changed in the commit. #11417
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
potiuk
requested review from
ashb,
dimberman,
feluelle,
houqp,
kaxil,
mik-laj,
turbaszek and
zhongjiajie
Member
Author
|
I am still testing that one in my private fork (https://github.com/potiuk/airflow/pulls). There are 12 pull requests that (I hope) should run faster or longer, and have more or less checks depending on the files that are changed in the upcoming commit. This is final step of implementing #10507 and I think this will be ultimate saver on the job queues that we have in Github Actions - and for most contributors it will significantly speed up the feedback time - if they only change docs + some of the providers for example. I will use the linked 12 PRs from my fork to make sure that it works as expected before merging but I am happy to get reviews before. |
potiuk
commented
Oct 11, 2020
scripts/ci/selective_tests.sh
Outdated
Member
Author
There was a problem hiding this comment.
If we have as many savings as I think we will, we can likely even bring back all python variants for all tests.
potiuk
force-pushed
the
selective-tests-2nd-attempt
branch
from
2790c5e to
69bb493
Compare
This was referenced Oct 11, 2020
Closed
potiuk
force-pushed
the
selective-tests-2nd-attempt
branch
2 times, most recently
from
1e7a72b to
993409e
Compare
This is final step of implementing apache#10507 - selective tests. Depending on files changed by the incoming commit, only subset of the tests are exucted. The conditions below are evaluated in the sequence specified below as well: * In case of "push" and "schedule" type of events, all tests are executed. * If no important files and folders changed - no tests are executed. This is a typical case for doc-only changes. * If any of the environment files (Dockerfile/setup.py etc.) all tests are executed. * If no "core/other" files are changed, only the relevant types of tests are executed: * API - if any of the API files/tests changed * CLI - if any of the CLI files/tests changed * WWW - if any of the WWW files/tests changed * Providers - if any of the Providers files/tests changed * Integration Heisentests, Quarantined, Postgres and MySQL runs are always run unless all tests are skipped like in case of doc-only changes. * If "Kubernetes" related files/tests are changed, the "Kubernetes" tests with Kind are run. Note that those tests are run separately using Host environment and those tests are stored in "kubernetes_tests" folder. * If some of the core/other files change, all tests are run. This is calculated by substracting all the files count calculated above from the total count of important files. Fixes: apache#10507
potiuk
force-pushed
the
selective-tests-2nd-attempt
branch
from
993409e to
5cf673f
Compare
potiuk
changed the title
Member
Author
|
Hello everyone - this one is ready for review and it looks like it works as expected: You can see all the 13 different cases where I tested small changes in some of the combinations: Examples:
|
Member
Author
|
Would be great to merge this one for the coming weel - it will significantly decrease the strain on the GA. |
kaxil
approved these changes
Oct 11, 2020
potiuk
added a commit
that referenced
this pull request
Nov 14, 2020
This is final step of implementing #10507 - selective tests. Depending on files changed by the incoming commit, only subset of the tests are exucted. The conditions below are evaluated in the sequence specified below as well: * In case of "push" and "schedule" type of events, all tests are executed. * If no important files and folders changed - no tests are executed. This is a typical case for doc-only changes. * If any of the environment files (Dockerfile/setup.py etc.) all tests are executed. * If no "core/other" files are changed, only the relevant types of tests are executed: * API - if any of the API files/tests changed * CLI - if any of the CLI files/tests changed * WWW - if any of the WWW files/tests changed * Providers - if any of the Providers files/tests changed * Integration Heisentests, Quarantined, Postgres and MySQL runs are always run unless all tests are skipped like in case of doc-only changes. * If "Kubernetes" related files/tests are changed, the "Kubernetes" tests with Kind are run. Note that those tests are run separately using Host environment and those tests are stored in "kubernetes_tests" folder. * If some of the core/other files change, all tests are run. This is calculated by substracting all the files count calculated above from the total count of important files. Fixes: #10507 (cherry picked from commit 369bbf0)
potiuk
added
the
type:misc/internal
potiuk
added a commit
that referenced
this pull request
Nov 16, 2020
This is final step of implementing #10507 - selective tests. Depending on files changed by the incoming commit, only subset of the tests are exucted. The conditions below are evaluated in the sequence specified below as well: * In case of "push" and "schedule" type of events, all tests are executed. * If no important files and folders changed - no tests are executed. This is a typical case for doc-only changes. * If any of the environment files (Dockerfile/setup.py etc.) all tests are executed. * If no "core/other" files are changed, only the relevant types of tests are executed: * API - if any of the API files/tests changed * CLI - if any of the CLI files/tests changed * WWW - if any of the WWW files/tests changed * Providers - if any of the Providers files/tests changed * Integration Heisentests, Quarantined, Postgres and MySQL runs are always run unless all tests are skipped like in case of doc-only changes. * If "Kubernetes" related files/tests are changed, the "Kubernetes" tests with Kind are run. Note that those tests are run separately using Host environment and those tests are stored in "kubernetes_tests" folder. * If some of the core/other files change, all tests are run. This is calculated by substracting all the files count calculated above from the total count of important files. Fixes: #10507 (cherry picked from commit 369bbf0)
potiuk
added a commit
that referenced
this pull request
Nov 16, 2020
This is final step of implementing #10507 - selective tests. Depending on files changed by the incoming commit, only subset of the tests are exucted. The conditions below are evaluated in the sequence specified below as well: * In case of "push" and "schedule" type of events, all tests are executed. * If no important files and folders changed - no tests are executed. This is a typical case for doc-only changes. * If any of the environment files (Dockerfile/setup.py etc.) all tests are executed. * If no "core/other" files are changed, only the relevant types of tests are executed: * API - if any of the API files/tests changed * CLI - if any of the CLI files/tests changed * WWW - if any of the WWW files/tests changed * Providers - if any of the Providers files/tests changed * Integration Heisentests, Quarantined, Postgres and MySQL runs are always run unless all tests are skipped like in case of doc-only changes. * If "Kubernetes" related files/tests are changed, the "Kubernetes" tests with Kind are run. Note that those tests are run separately using Host environment and those tests are stored in "kubernetes_tests" folder. * If some of the core/other files change, all tests are run. This is calculated by substracting all the files count calculated above from the total count of important files. Fixes: #10507 (cherry picked from commit 369bbf0)
kaxil
pushed a commit
that referenced
this pull request
Nov 18, 2020
This is final step of implementing #10507 - selective tests. Depending on files changed by the incoming commit, only subset of the tests are exucted. The conditions below are evaluated in the sequence specified below as well: * In case of "push" and "schedule" type of events, all tests are executed. * If no important files and folders changed - no tests are executed. This is a typical case for doc-only changes. * If any of the environment files (Dockerfile/setup.py etc.) all tests are executed. * If no "core/other" files are changed, only the relevant types of tests are executed: * API - if any of the API files/tests changed * CLI - if any of the CLI files/tests changed * WWW - if any of the WWW files/tests changed * Providers - if any of the Providers files/tests changed * Integration Heisentests, Quarantined, Postgres and MySQL runs are always run unless all tests are skipped like in case of doc-only changes. * If "Kubernetes" related files/tests are changed, the "Kubernetes" tests with Kind are run. Note that those tests are run separately using Host environment and those tests are stored in "kubernetes_tests" folder. * If some of the core/other files change, all tests are run. This is calculated by substracting all the files count calculated above from the total count of important files. Fixes: #10507 (cherry picked from commit 369bbf0)
cfei18
pushed a commit
to cfei18/incubator-airflow
that referenced
this pull request
Mar 5, 2021
This is final step of implementing apache#10507 - selective tests. Depending on files changed by the incoming commit, only subset of the tests are exucted. The conditions below are evaluated in the sequence specified below as well: * In case of "push" and "schedule" type of events, all tests are executed. * If no important files and folders changed - no tests are executed. This is a typical case for doc-only changes. * If any of the environment files (Dockerfile/setup.py etc.) all tests are executed. * If no "core/other" files are changed, only the relevant types of tests are executed: * API - if any of the API files/tests changed * CLI - if any of the CLI files/tests changed * WWW - if any of the WWW files/tests changed * Providers - if any of the Providers files/tests changed * Integration Heisentests, Quarantined, Postgres and MySQL runs are always run unless all tests are skipped like in case of doc-only changes. * If "Kubernetes" related files/tests are changed, the "Kubernetes" tests with Kind are run. Note that those tests are run separately using Host environment and those tests are stored in "kubernetes_tests" folder. * If some of the core/other files change, all tests are run. This is calculated by substracting all the files count calculated above from the total count of important files. Fixes: apache#10507 (cherry picked from commit 369bbf0)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is final step of implementing #10507 - selective tests.
Depending on files changed by the incoming commit, only subset of
the tests are exucted. The conditions below are evaluated in the
sequence specified below as well:
In case of "push" and "schedule" type of events, all tests
are executed.
If no important files and folders changed - no tests are executed.
This is a typical case for doc-only changes.
If any of the environment files (Dockerfile/setup.py etc.) all
tests are executed.
If no "core/other" files are changed, only the relevant types
of tests are executed:
Integration Heisentests, Quarantined, Postgres and MySQL
runs are always run unless all tests are skipped like in
case of doc-only changes.
If "Kubernetes" related files/tests are changed, the
"Kubernetes" tests with Kind are run. Note that those tests
are run separately using Host environment and those tests
are stored in "kubernetes_tests" folder.
If some of the core/other files change, all tests are run. This
is calculated by substracting all the files count calculated
above from the total count of important files.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.