Skip to content

Commit 85b70cf

Browse files
Adds clickbench logical & physical plans (#9794)
Signed-off-by: Joe Isaacs <[email protected]>
1 parent 3451462 commit 85b70cf

189 files changed

Lines changed: 15223 additions & 41 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -719,7 +719,7 @@ jobs:
719719
enable-sccache: "true"
720720
- name: Run sqllogictest tests
721721
run: |
722-
./vortex-sqllogictest/slt/tpch/generate_data.sh
722+
VORTEX_SLT_PROFILE=ci ./vortex-sqllogictest/slt/generate_data.sh
723723
cargo nextest run --cargo-profile ci -p vortex-sqllogictest
724724
- uses: ./.github/actions/check-rebuild
725725
with:

‎REUSE.toml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,13 @@ path = "vortex-btrblocks/decision-tree.*"
1212
SPDX-FileCopyrightText = "Copyright the Vortex contributors"
1313
SPDX-License-Identifier = "CC-BY-4.0"
1414

15+
# ClickBench result fixtures contain raw control characters from the dataset, so REUSE treats
16+
# them as binary and ignores their SPDX headers.
17+
[[annotations]]
18+
path = "vortex-sqllogictest/slt/clickbench/**/results/*.slt.no"
19+
SPDX-FileCopyrightText = "Copyright the Vortex contributors"
20+
SPDX-License-Identifier = "Apache-2.0"
21+
1522
# Golden files are licensed under CC-BY-4.0.
1623
[[annotations]]
1724
path = "**/goldenfiles/**"

‎vortex-sqllogictest/README.md‎

Lines changed: 34 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -10,28 +10,48 @@ sequentially for one engine. Each test is named `slt::<engine>::<relative-path>`
1010

1111
## Running tests
1212

13-
Some tests use TPC-H data at scale factor 0.1. Generate it first, then run the suite with
13+
Some tests use TPC-H data at scale factor 0.1 and one shard of the partitioned ClickBench
14+
dataset (about one million rows of `hits`). Generate both first, then run the suite with
1415
`cargo nextest`:
1516

1617
```shell
17-
./vortex-sqllogictest/slt/tpch/generate_data.sh
18+
./vortex-sqllogictest/slt/generate_data.sh
1819
cargo nextest run -p vortex-sqllogictest
1920
# the built-in cargo test harness also works:
2021
cargo test -p vortex-sqllogictest --test sqllogictests
2122
```
2223

23-
The generated Vortex and Parquet data lives under `slt/tpch/data/` (git-ignored). Both formats
24-
are required; regenerate older fixtures if they only contain Vortex files. If either format is
25-
missing, the TPC-H tests are reported as **ignored**, so the rest of the suite still runs. The TPC-H
26-
`.slt` files load their tables through paths relative to the crate root, so run the tests via
27-
`cargo nextest`/`cargo test`, which set the working directory accordingly.
24+
`generate_data.sh` accepts dataset names to generate only some of the fixtures, for example
25+
`./vortex-sqllogictest/slt/generate_data.sh tpch` or `... clickbench`. Run it with `--help` to list
26+
the datasets. It runs each dataset's own script, `slt/tpch/generate_data.sh` or
27+
`slt/clickbench/generate_data.sh`, which can also be run directly.
28+
29+
The generated Vortex and Parquet data lives under `slt/tpch/data/` and `slt/clickbench/data/`
30+
(git-ignored). Both formats are required; regenerate older fixtures if they only contain Vortex
31+
files. If either format is missing, that suite's tests are reported as **ignored**, so the rest of
32+
the suite still runs. These `.slt` files load their tables through paths relative to the crate
33+
root, so run the tests via `cargo nextest`/`cargo test`, which set the working directory
34+
accordingly.
35+
36+
TPC-H scripts live under `slt/tpch/datafusion/` and `slt/tpch/duckdb/`, ClickBench scripts under
37+
`slt/clickbench/datafusion/` and `slt/clickbench/duckdb/`. Each engine has its own
38+
`create.slt.no`, `results/q*.slt.no` (`q1` to `q22` for TPC-H, `q0` to `q42` for ClickBench,
39+
matching the upstream numbering), and `drop.slt.no`. Its `tpch.slt`/`clickbench.slt` runs these
40+
against Vortex and asserts EXPLAIN output from the matching `plans/q*.slt.no`. Its `parquet.slt`
41+
runs the same queries against the original Parquet fixtures and checks the same expected results.
42+
DataFusion uses external tables; DuckDB uses views over files. The `FILE_FORMAT` substitution
43+
variable selects the format in each engine's table setup.
44+
45+
ClickBench plans explain the upstream queries unchanged. Where an upstream query leaves the order
46+
of tied rows unspecified, its result record adds tie-breaking `ORDER BY` columns so both formats
47+
and repeated runs produce the same rows. The ClickBench generator also runs
48+
`slt/clickbench/duckdb/parity.slt` right after converting the shard; it reads both files through
49+
DuckDB and fails if the Parquet and Vortex data differ. After completing ClickBench DataFusion
50+
plans, replace the byte ranges in `file_groups` with `<slt:ignore>`, as the TPC-H plans do:
2851

29-
TPC-H scripts live under `slt/tpch/datafusion/` and `slt/tpch/duckdb/`. Each engine has its own
30-
`create.slt.no`, `results/q1.slt.no` through `results/q22.slt.no`, and `drop.slt.no`. Its `tpch.slt`
31-
runs these against Vortex and asserts EXPLAIN output from `plans/q1.slt.no` through `plans/q22.slt.no`.
32-
Its `parquet.slt` runs the same queries against the original Parquet fixtures and checks the same
33-
expected results. DataFusion uses external tables; DuckDB uses views over files. The `FILE_FORMAT`
34-
substitution variable selects the format in each engine's table setup.
52+
```shell
53+
sed -i -E 's/hits\.vortex:[0-9]+\.\.[0-9]+/hits.vortex:<slt:ignore>/g' vortex-sqllogictest/slt/clickbench/datafusion/plans/*.slt.no
54+
```
3555

3656
Because the harness is `libtest-mimic`-based, the standard test flags work, including
3757
`cargo nextest`, filtering, and listing:
@@ -103,7 +123,7 @@ engine currently produces, instead of comparing against it. This is useful after
103123
change to query results or plan formatting.
104124

105125
```shell
106-
# Complete every file (generate TPC-H data first if you want its result files updated):
126+
# Complete every file (generate TPC-H and ClickBench data first if you want their result files updated):
107127
cargo test -p vortex-sqllogictest --test sqllogictests -- --complete
108128
# Complete only the files whose name matches a substring:
109129
cargo test -p vortex-sqllogictest --test sqllogictests -- --complete strings

‎vortex-sqllogictest/bin/sqllogictests-runner.rs‎

Lines changed: 23 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -165,20 +165,29 @@ fn engines_for(path: &Path) -> (bool, bool) {
165165
(datafusion, duckdb)
166166
}
167167

168-
fn is_tpch(path: &Path) -> bool {
169-
path.components().any(|c| c.as_os_str() == "tpch")
168+
/// Suites whose tables come from a `generate_data.sh` script rather than the
169+
/// test itself: the `slt/` subdirectory holding the suite, and a file whose
170+
/// Vortex and Parquet versions both have to exist for the suite to run.
171+
const GENERATED_DATASETS: &[(&str, &str)] = &[
172+
("tpch", "tpch/data/lineitem"),
173+
("clickbench", "clickbench/data/hits"),
174+
];
175+
176+
/// Whether `path` belongs to a generated-data suite whose data is absent.
177+
fn missing_generated_data(path: &Path) -> bool {
178+
GENERATED_DATASETS.iter().any(|(dir, fixture)| {
179+
path.components().any(|c| c.as_os_str() == *dir)
180+
&& !["vortex", "parquet"]
181+
.into_iter()
182+
.all(|format| SLT_ROOT.join(format!("{fixture}.{format}")).exists())
183+
})
170184
}
171185

172186
/// Rewrites the expected output of each file in place, completing from a single
173187
/// reference engine per file (DuckDB for `duckdb/` files, DataFusion otherwise).
174-
fn complete_files(
175-
args: &Arguments,
176-
files: &[PathBuf],
177-
slt_root: &Path,
178-
has_tpch_data: bool,
179-
) -> anyhow::Result<()> {
188+
fn complete_files(args: &Arguments, files: &[PathBuf], slt_root: &Path) -> anyhow::Result<()> {
180189
for path in files {
181-
if is_tpch(path) && !has_tpch_data {
190+
if missing_generated_data(path) {
182191
continue;
183192
}
184193
let name = path
@@ -222,26 +231,21 @@ fn main() -> anyhow::Result<ExitCode> {
222231
};
223232
let args = Arguments::from_iter(raw_args);
224233

225-
let has_tpch_data = ["vortex", "parquet"].into_iter().all(|format| {
226-
SLT_ROOT
227-
.join(format!("tpch/data/lineitem.{format}"))
228-
.exists()
229-
});
230-
231234
let mut files = list_files(SLT_ROOT.as_path())?;
232235
files.sort();
233236

234237
if complete {
235-
complete_files(&args, &files, SLT_ROOT.as_path(), has_tpch_data)?;
238+
complete_files(&args, &files, SLT_ROOT.as_path())?;
236239
return Ok(ExitCode::SUCCESS);
237240
}
238241

239242
let mut trials = Vec::new();
240243
for path in files {
241244
let (run_datafusion, run_duckdb) = engines_for(&path);
242-
// TPC-H trials are ignored (rather than removed) when the generated data
243-
// is absent, so `--list` and the run summary still account for them.
244-
let ignored = is_tpch(&path) && !has_tpch_data;
245+
// TPC-H and ClickBench trials are ignored (rather than removed) when the
246+
// generated data is absent, so `--list` and the run summary still
247+
// account for them.
248+
let ignored = missing_generated_data(&path);
245249
let name = path
246250
.strip_prefix(SLT_ROOT.as_path())
247251
.unwrap_or(&path)
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
/data
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# SPDX-License-Identifier: Apache-2.0
2+
# SPDX-FileCopyrightText: Copyright the Vortex contributors
3+
4+
include ../../setup.slt.no
5+
6+
let FILE_FORMAT
7+
SELECT 'vortex';
8+
9+
include ./create.slt.no
10+
include ./results/*.slt.no
11+
include ./plans/*.slt.no
12+
include ./drop.slt.no
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# SPDX-License-Identifier: Apache-2.0
2+
# SPDX-FileCopyrightText: Copyright the Vortex contributors
3+
4+
# Keep the fixture under WORK_DIR so EXPLAIN output can normalize its path.
5+
# Plans ignore byte ranges because they depend on the generated fixture size.
6+
system ok
7+
cp slt/clickbench/data/hits.$FILE_FORMAT "$WORK_DIR/"
8+
9+
statement ok
10+
CREATE EXTERNAL TABLE IF NOT EXISTS hits
11+
STORED AS ${FILE_FORMAT}
12+
LOCATION '${WORK_DIR}/hits.${FILE_FORMAT}';
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# SPDX-License-Identifier: Apache-2.0
2+
# SPDX-FileCopyrightText: Copyright the Vortex contributors
3+
4+
statement ok
5+
DROP TABLE hits;
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# SPDX-License-Identifier: Apache-2.0
2+
# SPDX-FileCopyrightText: Copyright the Vortex contributors
3+
4+
include ../../setup.slt.no
5+
6+
let FILE_FORMAT
7+
SELECT 'parquet';
8+
9+
include ./create.slt.no
10+
include ./results/*.slt.no
11+
include ./drop.slt.no
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# SPDX-License-Identifier: Apache-2.0
2+
# SPDX-FileCopyrightText: Copyright the Vortex contributors
3+
4+
query TT
5+
EXPLAIN
6+
SELECT COUNT(*) FROM hits;
7+
----
8+
logical_plan
9+
01)Projection: count(Int64(1)) AS count(*)
10+
02)--Aggregate: groupBy=[[]], aggr=[[count(Int64(1))]]
11+
03)----TableScan: hits projection=[]
12+
physical_plan
13+
01)ProjectionExec: expr=[1000000 as count(*)]
14+
02)--PlaceholderRowExec

0 commit comments

Comments
 (0)