@@ -10,28 +10,48 @@ sequentially for one engine. Each test is named `slt::<engine>::<relative-path>`
1010
1111## Running tests
1212
13- Some tests use TPC-H data at scale factor 0.1. Generate it first, then run the suite with
13+ Some tests use TPC-H data at scale factor 0.1 and one shard of the partitioned ClickBench
14+ dataset (about one million rows of ` hits ` ). Generate both first, then run the suite with
1415` cargo nextest ` :
1516
1617``` shell
17- ./vortex-sqllogictest/slt/tpch/ generate_data.sh
18+ ./vortex-sqllogictest/slt/generate_data.sh
1819cargo nextest run -p vortex-sqllogictest
1920# the built-in cargo test harness also works:
2021cargo test -p vortex-sqllogictest --test sqllogictests
2122```
2223
23- The generated Vortex and Parquet data lives under ` slt/tpch/data/ ` (git-ignored). Both formats
24- are required; regenerate older fixtures if they only contain Vortex files. If either format is
25- missing, the TPC-H tests are reported as ** ignored** , so the rest of the suite still runs. The TPC-H
26- ` .slt ` files load their tables through paths relative to the crate root, so run the tests via
27- ` cargo nextest ` /` cargo test ` , which set the working directory accordingly.
24+ ` generate_data.sh ` accepts dataset names to generate only some of the fixtures, for example
25+ ` ./vortex-sqllogictest/slt/generate_data.sh tpch ` or ` ... clickbench ` . Run it with ` --help ` to list
26+ the datasets. It runs each dataset's own script, ` slt/tpch/generate_data.sh ` or
27+ ` slt/clickbench/generate_data.sh ` , which can also be run directly.
28+
29+ The generated Vortex and Parquet data lives under ` slt/tpch/data/ ` and ` slt/clickbench/data/ `
30+ (git-ignored). Both formats are required; regenerate older fixtures if they only contain Vortex
31+ files. If either format is missing, that suite's tests are reported as ** ignored** , so the rest of
32+ the suite still runs. These ` .slt ` files load their tables through paths relative to the crate
33+ root, so run the tests via ` cargo nextest ` /` cargo test ` , which set the working directory
34+ accordingly.
35+
36+ TPC-H scripts live under ` slt/tpch/datafusion/ ` and ` slt/tpch/duckdb/ ` , ClickBench scripts under
37+ ` slt/clickbench/datafusion/ ` and ` slt/clickbench/duckdb/ ` . Each engine has its own
38+ ` create.slt.no ` , ` results/q*.slt.no ` (` q1 ` to ` q22 ` for TPC-H, ` q0 ` to ` q42 ` for ClickBench,
39+ matching the upstream numbering), and ` drop.slt.no ` . Its ` tpch.slt ` /` clickbench.slt ` runs these
40+ against Vortex and asserts EXPLAIN output from the matching ` plans/q*.slt.no ` . Its ` parquet.slt `
41+ runs the same queries against the original Parquet fixtures and checks the same expected results.
42+ DataFusion uses external tables; DuckDB uses views over files. The ` FILE_FORMAT ` substitution
43+ variable selects the format in each engine's table setup.
44+
45+ ClickBench plans explain the upstream queries unchanged. Where an upstream query leaves the order
46+ of tied rows unspecified, its result record adds tie-breaking ` ORDER BY ` columns so both formats
47+ and repeated runs produce the same rows. The ClickBench generator also runs
48+ ` slt/clickbench/duckdb/parity.slt ` right after converting the shard; it reads both files through
49+ DuckDB and fails if the Parquet and Vortex data differ. After completing ClickBench DataFusion
50+ plans, replace the byte ranges in ` file_groups ` with ` <slt:ignore> ` , as the TPC-H plans do:
2851
29- TPC-H scripts live under ` slt/tpch/datafusion/ ` and ` slt/tpch/duckdb/ ` . Each engine has its own
30- ` create.slt.no ` , ` results/q1.slt.no ` through ` results/q22.slt.no ` , and ` drop.slt.no ` . Its ` tpch.slt `
31- runs these against Vortex and asserts EXPLAIN output from ` plans/q1.slt.no ` through ` plans/q22.slt.no ` .
32- Its ` parquet.slt ` runs the same queries against the original Parquet fixtures and checks the same
33- expected results. DataFusion uses external tables; DuckDB uses views over files. The ` FILE_FORMAT `
34- substitution variable selects the format in each engine's table setup.
52+ ``` shell
53+ sed -i -E ' s/hits\.vortex:[0-9]+\.\.[0-9]+/hits.vortex:<slt:ignore>/g' vortex-sqllogictest/slt/clickbench/datafusion/plans/* .slt.no
54+ ```
3555
3656Because the harness is ` libtest-mimic ` -based, the standard test flags work, including
3757` cargo nextest ` , filtering, and listing:
@@ -103,7 +123,7 @@ engine currently produces, instead of comparing against it. This is useful after
103123change to query results or plan formatting.
104124
105125``` shell
106- # Complete every file (generate TPC-H data first if you want its result files updated):
126+ # Complete every file (generate TPC-H and ClickBench data first if you want their result files updated):
107127cargo test -p vortex-sqllogictest --test sqllogictests -- --complete
108128# Complete only the files whose name matches a substring:
109129cargo test -p vortex-sqllogictest --test sqllogictests -- --complete strings
0 commit comments