Skip to content

Commit b37452e

Browse files
potiukpotiuk
authored
Integration tests are separated into separate command and CI job (apache#28207)
Integration tests so far were a separate test type among the unit tests, however we have to start them differently. This PR introduces new command in Breeze: breeze testing integration-tests The `--integration` option has been removed from the regular unit tests, and now it is used to the integration-tests command. The integration-tests command has no parallel option.
1 parent 1ed01b5 commit b37452e

32 files changed

+764
-665
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 107 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -810,7 +810,7 @@ jobs:
810810
uses: ./.github/actions/prepare_breeze_and_image
811811
- name: "Migration Tests: ${{matrix.python-version}}:${{needs.build-info.outputs.test-types}}"
812812
uses: ./.github/actions/migration_tests
813-
- name: "Tests: ${{matrix.python-version}}:${{needs.build-info.outputs.test-types}}"
813+
- name: "Tests: ${{matrix.python-version}}:${{needs.build-info.outputs.test-types}} (w/Kerberos)"
814814
run: breeze testing tests --run-in-parallel
815815
- name: "Post Tests: ${{matrix.python-version}}:${{needs.build-info.outputs.test-types}}"
816816
uses: ./.github/actions/post_tests
@@ -938,6 +938,104 @@ jobs:
938938
- name: "Post Tests: ${{matrix.python-version}}:${{needs.build-info.outputs.test-types}}"
939939
uses: ./.github/actions/post_tests
940940

941+
tests-integration-postgres:
942+
timeout-minutes: 130
943+
name: Integration Tests Postgres
944+
runs-on: "${{needs.build-info.outputs.runs-on}}"
945+
needs: [build-info, wait-for-ci-images]
946+
env:
947+
RUNS_ON: "${{needs.build-info.outputs.runs-on}}"
948+
TEST_TYPES: "${{needs.build-info.outputs.test-types}}"
949+
PR_LABELS: "${{needs.build-info.outputs.pull-request-labels}}"
950+
FULL_TESTS_NEEDED: "${{needs.build-info.outputs.full-tests-needed}}"
951+
DEBUG_RESOURCES: "${{needs.build-info.outputs.debug-resources}}"
952+
BACKEND: "postgres"
953+
PYTHON_MAJOR_MINOR_VERSION: "${{needs.build-info.outputs.default-python-version}}"
954+
POSTGRES_VERSION: "${{needs.build-info.outputs.default-postgres-version}}"
955+
BACKEND_VERSION: "${{needs.build-info.outputs.default-python-version}}"
956+
JOB_ID: "integration"
957+
COVERAGE: "${{needs.build-info.outputs.run-coverage}}"
958+
if: needs.build-info.outputs.run-tests == 'true' && needs.build-info.outputs.default-branch == 'main'
959+
steps:
960+
- name: Cleanup repo
961+
shell: bash
962+
run: docker run -v "${GITHUB_WORKSPACE}:/workspace" -u 0:0 bash -c "rm -rf /workspace/*"
963+
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
964+
uses: actions/checkout@v3
965+
with:
966+
persist-credentials: false
967+
- name: "Prepare breeze & CI image: ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{env.IMAGE_TAG}}"
968+
uses: ./.github/actions/prepare_breeze_and_image
969+
- name: "Integration Tests Postgres: cassandra"
970+
run: |
971+
breeze testing integration-tests --integration cassandra
972+
breeze stop
973+
if: needs.build-info.outputs.runs-on != 'self-hosted'
974+
- name: "Integration Tests Postgres: mongo"
975+
run: |
976+
breeze testing integration-tests --integration mongo
977+
breeze stop
978+
if: needs.build-info.outputs.runs-on != 'self-hosted'
979+
- name: "Integration Tests Postgres: pinot"
980+
run: |
981+
breeze testing integration-tests --integration pinot
982+
breeze stop
983+
if: needs.build-info.outputs.runs-on != 'self-hosted'
984+
- name: "Integration Tests Postgres: celery"
985+
run: |
986+
breeze testing integration-tests --integration celery
987+
breeze stop
988+
if: needs.build-info.outputs.runs-on != 'self-hosted'
989+
- name: "Integration Tests Postgres: trino, kerberos"
990+
run: |
991+
breeze testing integration-tests --integration trino --integration kerberos
992+
breeze stop
993+
if: needs.build-info.outputs.runs-on != 'self-hosted'
994+
- name: "Integration Tests Postgres: all"
995+
run: breeze testing integration-tests --integration all
996+
if: needs.build-info.outputs.runs-on == 'self-hosted'
997+
- name: "Post Tests: ${{matrix.python-version}}:${{needs.build-info.outputs.test-types}}"
998+
uses: ./.github/actions/post_tests
999+
1000+
tests-integration-mysql:
1001+
timeout-minutes: 130
1002+
name: Integration Tests MySQL
1003+
runs-on: "${{needs.build-info.outputs.runs-on}}"
1004+
needs: [build-info, wait-for-ci-images]
1005+
env:
1006+
RUNS_ON: "${{needs.build-info.outputs.runs-on}}"
1007+
TEST_TYPES: "${{needs.build-info.outputs.test-types}}"
1008+
PR_LABELS: "${{needs.build-info.outputs.pull-request-labels}}"
1009+
FULL_TESTS_NEEDED: "${{needs.build-info.outputs.full-tests-needed}}"
1010+
DEBUG_RESOURCES: "${{needs.build-info.outputs.debug-resources}}"
1011+
BACKEND: "postgres"
1012+
PYTHON_MAJOR_MINOR_VERSION: "${{needs.build-info.outputs.default-python-version}}"
1013+
POSTGRES_VERSION: "${{needs.build-info.outputs.default-postgres-version}}"
1014+
BACKEND_VERSION: "${{needs.build-info.outputs.default-python-version}}"
1015+
JOB_ID: "integration"
1016+
COVERAGE: "${{needs.build-info.outputs.run-coverage}}"
1017+
if: needs.build-info.outputs.run-tests == 'true' && needs.build-info.outputs.default-branch == 'main'
1018+
steps:
1019+
- name: Cleanup repo
1020+
shell: bash
1021+
run: docker run -v "${GITHUB_WORKSPACE}:/workspace" -u 0:0 bash -c "rm -rf /workspace/*"
1022+
if: needs.build-info.outputs.runs-on == 'self-hosted'
1023+
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
1024+
uses: actions/checkout@v3
1025+
with:
1026+
persist-credentials: false
1027+
if: needs.build-info.outputs.runs-on == 'self-hosted'
1028+
- name: "Prepare breeze & CI image: ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{env.IMAGE_TAG}}"
1029+
uses: ./.github/actions/prepare_breeze_and_image
1030+
if: needs.build-info.outputs.runs-on == 'self-hosted'
1031+
- name: "Integration Tests MySQL: all"
1032+
run: breeze testing integration-tests --integration all
1033+
if: needs.build-info.outputs.runs-on == 'self-hosted'
1034+
- name: "Post Tests: ${{matrix.python-version}}:${{needs.build-info.outputs.test-types}}"
1035+
uses: ./.github/actions/post_tests
1036+
if: needs.build-info.outputs.runs-on == 'self-hosted'
1037+
1038+
9411039
tests-quarantined:
9421040
timeout-minutes: 60
9431041
name: "Quarantined tests"
@@ -985,6 +1083,8 @@ jobs:
9851083
- tests-mysql
9861084
- tests-mssql
9871085
- tests-quarantined
1086+
- tests-integration-postgres
1087+
- tests-integration-mysql
9881088
env:
9891089
RUNS_ON: "${{needs.build-info.outputs.runs-on}}"
9901090
# Only upload coverage on merges to main
@@ -1019,6 +1119,8 @@ jobs:
10191119
- tests-mysql
10201120
- tests-mssql
10211121
- tests-quarantined
1122+
- tests-integration-postgres
1123+
- tests-integration-mysql
10221124
env:
10231125
RUNS_ON: "${{needs.build-info.outputs.runs-on}}"
10241126
steps:
@@ -1180,6 +1282,8 @@ jobs:
11801282
- tests-mysql
11811283
- tests-mssql
11821284
- tests-postgres
1285+
- tests-integration-postgres
1286+
- tests-integration-mysql
11831287
- push-early-buildx-cache-to-github-registry
11841288
env:
11851289
RUNS_ON: "${{needs.build-info.outputs.runs-on}}"
@@ -1340,6 +1444,8 @@ jobs:
13401444
- tests-mysql
13411445
- tests-mssql
13421446
- tests-postgres
1447+
- tests-integration-postgres
1448+
- tests-integration-mysql
13431449
env:
13441450
DEFAULT_BRANCH: ${{ needs.build-info.outputs.default-branch }}
13451451
DEFAULT_CONSTRAINTS_BRANCH: ${{ needs.build-info.outputs.default-constraints-branch }}

‎BREEZE.rst‎

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -687,12 +687,12 @@ on how to run them.
687687

688688
This applies to all kind of tests - all our tests can be run using pytest.
689689

690-
Running unit/integration tests in groups
691-
........................................
690+
Running unit tests
691+
..................
692692

693-
Another option you have is that you can also run tests via built-in ``breeze testing`` command.
693+
Another option you have is that you can also run tests via built-in ``breeze testing tests`` command.
694694
The iterative ``pytest`` command allows to run test individually, or by class or in any other way
695-
pytest allows to test them and run them interactively, but ``breeze testing`` command allows to
695+
pytest allows to test them and run them interactively, but ``breeze testing tests`` command allows to
696696
run the tests in the same test "types" that are used to run the tests in CI: for example Core, Always
697697
API, Providers. This how our CI runs them - running each group in parallel to other groups and you can
698698
replicate this behaviour.
@@ -724,6 +724,28 @@ Here is the detailed set of options for the ``breeze testing tests`` command.
724724
:width: 100%
725725
:alt: Breeze testing tests
726726

727+
Running integration tests
728+
.........................
729+
730+
You can also run integration tests via built-in ``breeze testing integration-tests`` command. Some of our
731+
tests require additional integrations to be started in docker-compose. The integration tests command will
732+
run the expected integration and tests that need that integration.
733+
734+
For example this will only run kerberos tests:
735+
736+
.. code-block:: bash
737+
738+
breeze testing integration-tests --integration Kerberos
739+
740+
741+
Here is the detailed set of options for the ``breeze testing integration-tests`` command.
742+
743+
.. image:: ./images/breeze/output_testing_integration-tests.svg
744+
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_testing_integration_tests.svg
745+
:width: 100%
746+
:alt: Breeze testing integration-tests
747+
748+
727749
Running Helm tests
728750
..................
729751

‎Dockerfile.ci‎

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -948,13 +948,7 @@ fi
948948
readonly SELECTED_TESTS CLI_TESTS API_TESTS PROVIDERS_TESTS CORE_TESTS WWW_TESTS \
949949
ALL_TESTS ALL_PRESELECTED_TESTS
950950

951-
if [[ -n ${LIST_OF_INTEGRATION_TESTS_TO_RUN=} ]]; then
952-
# Integration tests
953-
for INT in ${LIST_OF_INTEGRATION_TESTS_TO_RUN}
954-
do
955-
EXTRA_PYTEST_ARGS+=("--integration" "${INT}")
956-
done
957-
elif [[ ${TEST_TYPE:=""} == "Long" ]]; then
951+
if [[ ${TEST_TYPE:=""} == "Long" ]]; then
958952
EXTRA_PYTEST_ARGS+=(
959953
"-m" "long_running"
960954
"--include-long-running"

0 commit comments

Comments
 (0)