Skip to content

Commit b15394a

Browse files
authored
Centralize SQL benchmark matrices in bench-orchestrator (#9017)
## Rationale for this change Closes: #4935 Revived from: #8801 SQL benchmark coverage was spread across large workflow JSON matrices and difficult to audit. ## What changes are included in this PR? Moves benchmark coverage into one typed Python catalog. `vx-bench matrix <preset>` validates and renders it for GitHub Actions; the surrounding modules only model, validate, and render the catalog. The `pr`, `pr-full`, `develop`, and `nightly` outputs remain unchanged. Signed-off-by: Connor Tsui <[email protected]>
1 parent ea88bf4 commit b15394a

15 files changed

Lines changed: 717 additions & 591 deletions

File tree

‎.github/workflows/bench-dispatch.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,4 +64,4 @@ jobs:
6464
uses: ./.github/workflows/sql-pr.yml
6565
secrets: inherit
6666
with:
67-
benchmark_profile: "full"
67+
matrix_preset: "pr-full"

‎.github/workflows/bench-pr.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,3 +139,4 @@ jobs:
139139
secrets: inherit
140140
with:
141141
mode: "pr"
142+
matrix_preset: "pr-full"

‎.github/workflows/bench.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -171,3 +171,4 @@ jobs:
171171
secrets: inherit
172172
with:
173173
mode: "develop"
174+
matrix_preset: "develop"

‎.github/workflows/nightly-bench.yml‎

Lines changed: 1 addition & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -23,49 +23,7 @@ jobs:
2323
with:
2424
mode: "develop"
2525
machine_type: ${{ matrix.machine_type.instance_name }}
26-
benchmark_matrix: |
27-
[
28-
{
29-
"id": "tpch-nvme",
30-
"subcommand": "tpch",
31-
"name": "TPC-H on NVME",
32-
"data_formats": ["parquet", "vortex"],
33-
"pr_targets": [
34-
{"engine": "datafusion", "format": "parquet"},
35-
{"engine": "datafusion", "format": "vortex"},
36-
{"engine": "duckdb", "format": "parquet"},
37-
{"engine": "duckdb", "format": "vortex"}
38-
],
39-
"develop_targets": [
40-
{"engine": "datafusion", "format": "parquet"},
41-
{"engine": "datafusion", "format": "vortex"},
42-
{"engine": "duckdb", "format": "parquet"},
43-
{"engine": "duckdb", "format": "vortex"}
44-
],
45-
"scale_factor": "100"
46-
},
47-
{
48-
"id": "tpch-s3",
49-
"subcommand": "tpch",
50-
"name": "TPC-H on S3",
51-
"local_dir": "vortex-bench/data/tpch/100.0",
52-
"remote_storage": "s3://vortex-ci-benchmark-datasets/${{github.ref_name}}/${{github.run_id}}/tpch/100.0/",
53-
"data_formats": ["parquet", "vortex"],
54-
"pr_targets": [
55-
{"engine": "datafusion", "format": "parquet"},
56-
{"engine": "datafusion", "format": "vortex"},
57-
{"engine": "duckdb", "format": "parquet"},
58-
{"engine": "duckdb", "format": "vortex"}
59-
],
60-
"develop_targets": [
61-
{"engine": "datafusion", "format": "parquet"},
62-
{"engine": "datafusion", "format": "vortex"},
63-
{"engine": "duckdb", "format": "parquet"},
64-
{"engine": "duckdb", "format": "vortex"}
65-
],
66-
"scale_factor": "100.0"
67-
}
68-
]
26+
matrix_preset: "nightly"
6927
strategy:
7028
# A single run not should kill the others
7129
fail-fast: false

0 commit comments

Comments
 (0)