Skip to content

Commit 07cb54f

Browse files
arthurpassosarthurpassos
committed
Merge branch 'master' into merge_parquet_minmax_bloom_filter_evaluation
2 parents f0c4cee + d99795b commit 07cb54f

File tree

1,033 files changed

+25274
-16528
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,033 files changed

+25274
-16528
lines changed

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,14 +47,13 @@ At a minimum, the following information should be added (but add more as needed)
4747

4848
All builds in Builds_1 and Builds_2 stages are always mandatory and will run independently of the checks below:
4949
- [ ] <!---ci_include_stateless--> Only: Stateless tests
50-
- [ ] <!---ci_include_stateful--> Only: Stateful tests
5150
- [ ] <!---ci_include_integration--> Only: Integration tests
5251
- [ ] <!---ci_include_performance--> Only: Performance tests
5352
---
5453
- [ ] <!---ci_exclude_style--> Skip: Style check
5554
- [ ] <!---ci_exclude_fast--> Skip: Fast test
5655
---
57-
- [ ] <!---woolen_wolfdog--> Non-blocking CI mode (Resource-intensive. All test jobs execute in parallel).
56+
- [ ] <!---woolen_wolfdog--> Run all checks ignoring all possible failures (Resource-intensive. All test jobs execute in parallel).
5857
- [ ] <!---no_ci_cache--> Disable CI cache
5958

6059
<!--

‎.github/workflows/backport_branches.yml‎

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -210,17 +210,6 @@ jobs:
210210
runner_type: func-tester
211211
data: ${{ needs.RunConfig.outputs.data }}
212212
##############################################################################################
213-
############################ FUNCTIONAl STATEFUL TESTS #######################################
214-
##############################################################################################
215-
FunctionalStatefulTestDebug:
216-
needs: [RunConfig, BuilderDebDebug]
217-
if: ${{ !failure() && !cancelled() }}
218-
uses: ./.github/workflows/reusable_test.yml
219-
with:
220-
test_name: Stateful tests (debug)
221-
runner_type: func-tester
222-
data: ${{ needs.RunConfig.outputs.data }}
223-
##############################################################################################
224213
######################################### STRESS TESTS #######################################
225214
##############################################################################################
226215
StressTestTsan:
@@ -256,7 +245,6 @@ jobs:
256245
- RunConfig
257246
- Builds_Report
258247
- FunctionalStatelessTestAsan
259-
- FunctionalStatefulTestDebug
260248
- StressTestTsan
261249
- IntegrationTestsTsan
262250
- IntegrationTestsAsanOldAnalyzer

‎.github/workflows/master.yml‎

Lines changed: 21 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -27,12 +27,6 @@ jobs:
2727
run: |
2828
cd "$GITHUB_WORKSPACE/tests/ci"
2929
python3 sync_pr.py --merge || :
30-
# Runs in MQ:
31-
# - name: Python unit tests
32-
# run: |
33-
# cd "$GITHUB_WORKSPACE/tests/ci"
34-
# echo "Testing the main ci directory"
35-
# python3 -m unittest discover -s . -p 'test_*.py'
3630
- name: PrepareRunConfig
3731
id: runconfig
3832
run: |
@@ -50,27 +44,13 @@ jobs:
5044
- name: Re-create GH statuses for skipped jobs if any
5145
run: |
5246
python3 "$GITHUB_WORKSPACE/tests/ci/ci.py" --infile ${{ runner.temp }}/ci_run_data.json --update-gh-statuses
53-
# Runs in MQ:
54-
# BuildDockers:
55-
# needs: [RunConfig]
56-
# if: ${{ !failure() && !cancelled() }}
57-
# uses: ./.github/workflows/docker_test_images.yml
58-
# with:
59-
# data: ${{ needs.RunConfig.outputs.data }}
60-
# StyleCheck:
61-
# needs: [RunConfig, BuildDockers]
62-
# if: ${{ !failure() && !cancelled() }}
63-
# uses: ./.github/workflows/reusable_test.yml
64-
# with:
65-
# test_name: Style check
66-
# runner_type: style-checker
67-
# data: ${{ needs.RunConfig.outputs.data }}
68-
# run_command: |
69-
# python3 style_check.py --no-push
70-
71-
################################# Main stages #################################
72-
# for main CI chain
73-
#
47+
Builds_0:
48+
needs: [RunConfig]
49+
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Builds_0') }}
50+
uses: ./.github/workflows/reusable_build_stage.yml
51+
with:
52+
stage: Builds_0
53+
data: ${{ needs.RunConfig.outputs.data }}
7454
Builds_1:
7555
needs: [RunConfig]
7656
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Builds_1') }}
@@ -79,30 +59,29 @@ jobs:
7959
with:
8060
stage: Builds_1
8161
data: ${{ needs.RunConfig.outputs.data }}
82-
Tests_1:
83-
needs: [RunConfig, Builds_1]
84-
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Tests_1') }}
85-
uses: ./.github/workflows/reusable_test_stage.yml
86-
with:
87-
stage: Tests_1
88-
data: ${{ needs.RunConfig.outputs.data }}
8962
Builds_2:
9063
needs: [RunConfig, Builds_1]
9164
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Builds_2') }}
9265
uses: ./.github/workflows/reusable_build_stage.yml
9366
with:
9467
stage: Builds_2
9568
data: ${{ needs.RunConfig.outputs.data }}
96-
Tests_2_ww:
97-
needs: [RunConfig, Builds_2]
98-
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Tests_2_ww') }}
69+
Tests_0:
70+
needs: [RunConfig, Builds_0]
71+
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Tests_0') }}
9972
uses: ./.github/workflows/reusable_test_stage.yml
10073
with:
101-
stage: Tests_2_ww
74+
stage: Tests_0
10275
data: ${{ needs.RunConfig.outputs.data }}
103-
Tests_2:
104-
# Test_3 should not wait for Test_1/Test_2 and should not be blocked by them on master branch since all jobs need to run there.
76+
Tests_1:
10577
needs: [RunConfig, Builds_1]
78+
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Tests_1') }}
79+
uses: ./.github/workflows/reusable_test_stage.yml
80+
with:
81+
stage: Tests_1
82+
data: ${{ needs.RunConfig.outputs.data }}
83+
Tests_2:
84+
needs: [RunConfig, Builds_1, Builds_0]
10685
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Tests_2') }}
10786
uses: ./.github/workflows/reusable_test_stage.yml
10887
with:
@@ -114,7 +93,7 @@ jobs:
11493
Builds_Report:
11594
# run report check for failed builds to indicate the CI error
11695
if: ${{ !cancelled() && needs.RunConfig.result == 'success' && contains(fromJson(needs.RunConfig.outputs.data).jobs_data.jobs_to_do, 'Builds') }}
117-
needs: [RunConfig, Builds_1, Builds_2]
96+
needs: [RunConfig, Builds_1, Builds_2, Builds_0]
11897
uses: ./.github/workflows/reusable_test.yml
11998
with:
12099
test_name: Builds
@@ -123,7 +102,7 @@ jobs:
123102

124103
FinishCheck:
125104
if: ${{ !cancelled() }}
126-
needs: [RunConfig, Builds_1, Builds_2, Builds_Report, Tests_1, Tests_2_ww, Tests_2]
105+
needs: [RunConfig, Builds_0, Builds_1, Builds_2, Builds_Report, Tests_1, Tests_2]
127106
runs-on: [self-hosted, style-checker-aarch64]
128107
steps:
129108
- name: Check out repository code

‎.github/workflows/pull_request.yml‎

Lines changed: 27 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ jobs:
8181
secrets:
8282
robot_git_token: ${{secrets.ROBOT_CLICKHOUSE_SSH_KEY}}
8383
FastTest:
84-
needs: [RunConfig, BuildDockers, StyleCheck]
84+
needs: [RunConfig, BuildDockers]
8585
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).jobs_data.jobs_to_do, 'Fast test') }}
8686
uses: ./.github/workflows/reusable_test.yml
8787
with:
@@ -90,10 +90,16 @@ jobs:
9090
data: ${{ needs.RunConfig.outputs.data }}
9191
run_command: |
9292
python3 fast_test_check.py
93-
9493
################################# Main stages #################################
9594
# for main CI chain
9695
#
96+
Builds_0:
97+
needs: [RunConfig, BuildDockers, StyleCheck, FastTest]
98+
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Builds_0') }}
99+
uses: ./.github/workflows/reusable_build_stage.yml
100+
with:
101+
stage: Builds_0
102+
data: ${{ needs.RunConfig.outputs.data }}
97103
Builds_1:
98104
needs: [RunConfig, StyleCheck, FastTest]
99105
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Builds_1') }}
@@ -102,20 +108,27 @@ jobs:
102108
with:
103109
stage: Builds_1
104110
data: ${{ needs.RunConfig.outputs.data }}
105-
Tests_1:
106-
needs: [RunConfig, Builds_1]
107-
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Tests_1') }}
108-
uses: ./.github/workflows/reusable_test_stage.yml
109-
with:
110-
stage: Tests_1
111-
data: ${{ needs.RunConfig.outputs.data }}
112111
Builds_2:
113-
needs: [RunConfig, Builds_1]
112+
needs: [RunConfig, StyleCheck, FastTest]
114113
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Builds_2') }}
115114
uses: ./.github/workflows/reusable_build_stage.yml
116115
with:
117116
stage: Builds_2
118117
data: ${{ needs.RunConfig.outputs.data }}
118+
Tests_0:
119+
needs: [RunConfig, Builds_0]
120+
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Tests_0') }}
121+
uses: ./.github/workflows/reusable_test_stage.yml
122+
with:
123+
stage: Tests_0
124+
data: ${{ needs.RunConfig.outputs.data }}
125+
Tests_1:
126+
needs: [RunConfig, Builds_1]
127+
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Tests_0') }}
128+
uses: ./.github/workflows/reusable_test_stage.yml
129+
with:
130+
stage: Tests_1
131+
data: ${{ needs.RunConfig.outputs.data }}
119132
# stage for running non-required checks without being blocked by required checks (Test_1) if corresponding settings is selected
120133
Tests_2_ww:
121134
needs: [RunConfig, Builds_1]
@@ -125,7 +138,7 @@ jobs:
125138
stage: Tests_2_ww
126139
data: ${{ needs.RunConfig.outputs.data }}
127140
Tests_2:
128-
needs: [RunConfig, Builds_1, Tests_1]
141+
needs: [RunConfig, Builds_1, Tests_1, Builds_0, Tests_0]
129142
if: ${{ !failure() && !cancelled() && contains(fromJson(needs.RunConfig.outputs.data).stages_data.stages_to_do, 'Tests_2') }}
130143
uses: ./.github/workflows/reusable_test_stage.yml
131144
with:
@@ -142,7 +155,7 @@ jobs:
142155
&& needs.FastTest.result != 'failure'
143156
&& needs.BuildDockers.result != 'failure'
144157
&& contains(fromJson(needs.RunConfig.outputs.data).jobs_data.jobs_to_do, 'Builds') }}
145-
needs: [RunConfig, BuildDockers, StyleCheck, FastTest, Builds_1, Builds_2]
158+
needs: [RunConfig, BuildDockers, StyleCheck, FastTest, Builds_0, Builds_1, Builds_2]
146159
uses: ./.github/workflows/reusable_test.yml
147160
with:
148161
test_name: Builds
@@ -153,7 +166,7 @@ jobs:
153166
if: ${{ !cancelled() }}
154167
# Test_2 or Test_3 do not have the jobs required for Mergeable check,
155168
# however, set them as "needs" to get all checks results before the automatic merge occurs.
156-
needs: [RunConfig, BuildDockers, StyleCheck, FastTest, Builds_1, Builds_2, Builds_Report, Tests_1, Tests_2_ww, Tests_2]
169+
needs: [RunConfig, BuildDockers, StyleCheck, FastTest, Builds_1, Builds_2, Builds_Report, Tests_1, Tests_2_ww, Tests_2, Builds_0, Tests_0]
157170
runs-on: [self-hosted, style-checker-aarch64]
158171
steps:
159172
- name: Check out repository code
@@ -178,7 +191,7 @@ jobs:
178191
#
179192
FinishCheck:
180193
if: ${{ !failure() && !cancelled() }}
181-
needs: [RunConfig, BuildDockers, StyleCheck, FastTest, Builds_1, Builds_2, Builds_Report, Tests_1, Tests_2_ww, Tests_2]
194+
needs: [RunConfig, BuildDockers, StyleCheck, FastTest, Builds_1, Builds_2, Builds_Report, Tests_1, Tests_2_ww, Tests_2, Builds_0]
182195
runs-on: [self-hosted, style-checker-aarch64]
183196
steps:
184197
- name: Check out repository code

‎.github/workflows/release_branches.yml‎

Lines changed: 0 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -307,65 +307,6 @@ jobs:
307307
runner_type: func-tester
308308
data: ${{ needs.RunConfig.outputs.data }}
309309
##############################################################################################
310-
############################ FUNCTIONAl STATEFUL TESTS #######################################
311-
##############################################################################################
312-
FunctionalStatefulTestRelease:
313-
needs: [RunConfig, BuilderDebRelease]
314-
if: ${{ !failure() && !cancelled() }}
315-
uses: ./.github/workflows/reusable_test.yml
316-
with:
317-
test_name: Stateful tests (release)
318-
runner_type: func-tester
319-
data: ${{ needs.RunConfig.outputs.data }}
320-
FunctionalStatefulTestAarch64:
321-
needs: [RunConfig, BuilderDebAarch64]
322-
if: ${{ !failure() && !cancelled() }}
323-
uses: ./.github/workflows/reusable_test.yml
324-
with:
325-
test_name: Stateful tests (aarch64)
326-
runner_type: func-tester-aarch64
327-
data: ${{ needs.RunConfig.outputs.data }}
328-
FunctionalStatefulTestAsan:
329-
needs: [RunConfig, BuilderDebAsan]
330-
if: ${{ !failure() && !cancelled() }}
331-
uses: ./.github/workflows/reusable_test.yml
332-
with:
333-
test_name: Stateful tests (asan)
334-
runner_type: func-tester
335-
data: ${{ needs.RunConfig.outputs.data }}
336-
FunctionalStatefulTestTsan:
337-
needs: [RunConfig, BuilderDebTsan]
338-
if: ${{ !failure() && !cancelled() }}
339-
uses: ./.github/workflows/reusable_test.yml
340-
with:
341-
test_name: Stateful tests (tsan)
342-
runner_type: func-tester
343-
data: ${{ needs.RunConfig.outputs.data }}
344-
FunctionalStatefulTestMsan:
345-
needs: [RunConfig, BuilderDebMsan]
346-
if: ${{ !failure() && !cancelled() }}
347-
uses: ./.github/workflows/reusable_test.yml
348-
with:
349-
test_name: Stateful tests (msan)
350-
runner_type: func-tester
351-
data: ${{ needs.RunConfig.outputs.data }}
352-
FunctionalStatefulTestUBsan:
353-
needs: [RunConfig, BuilderDebUBsan]
354-
if: ${{ !failure() && !cancelled() }}
355-
uses: ./.github/workflows/reusable_test.yml
356-
with:
357-
test_name: Stateful tests (ubsan)
358-
runner_type: func-tester
359-
data: ${{ needs.RunConfig.outputs.data }}
360-
FunctionalStatefulTestDebug:
361-
needs: [RunConfig, BuilderDebDebug]
362-
if: ${{ !failure() && !cancelled() }}
363-
uses: ./.github/workflows/reusable_test.yml
364-
with:
365-
test_name: Stateful tests (debug)
366-
runner_type: func-tester
367-
data: ${{ needs.RunConfig.outputs.data }}
368-
##############################################################################################
369310
######################################### STRESS TESTS #######################################
370311
##############################################################################################
371312
StressTestAsan:
@@ -458,13 +399,6 @@ jobs:
458399
- FunctionalStatelessTestTsan
459400
- FunctionalStatelessTestMsan
460401
- FunctionalStatelessTestUBsan
461-
- FunctionalStatefulTestDebug
462-
- FunctionalStatefulTestRelease
463-
- FunctionalStatefulTestAarch64
464-
- FunctionalStatefulTestAsan
465-
- FunctionalStatefulTestTsan
466-
- FunctionalStatefulTestMsan
467-
- FunctionalStatefulTestUBsan
468402
- StressTestDebug
469403
- StressTestAsan
470404
- StressTestTsan

0 commit comments

Comments
 (0)