Skip to content

Commit 94c7bde

Browse files
djouallahclaude
andcommitted
Workload sail probe: keep Sail on hash joins
Run 39690013877: every model and grain test reached; fct_summary's incremental build failed in DataFusion's sort-merge join on a UTC timestamp key (apache/datafusion#26003). Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 49efb9b commit 94c7bde

1 file changed

Lines changed: 6 additions & 4 deletions

File tree

‎bench/workload/config.py‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -53,10 +53,12 @@ def files_abfss(self) -> str:
5353

5454
@property
5555
def estimated_gib(self) -> float:
56-
"""What an engine sizes its memory against (bench's Sail engine: hash joins only while the
57-
data fits half its 10 GiB pool). Not an SF: the workload's tables only grow -- fct_scada is
58-
past 380M rows -- so this says "bigger than memory", and Sail takes joins that can spill."""
59-
return 64.0
56+
"""What an engine sizes its memory against: bench's Sail engine (the sail probe) switches to
57+
sort-merge joins past half its 10 GiB pool. 5 keeps HASH joins. Sail's sort-merge join
58+
cannot key on a time-zone-aware timestamp -- fct_summary's incremental build failed
59+
"Unsupported data type in sort merge join comparator: Timestamp(us, UTC)" (run 39690013877,
60+
apache/datafusion#26003) -- so the workload stays on hash joins whatever its size."""
61+
return 5.0
6062

6163
@property
6264
def schema(self) -> str:

0 commit comments

Comments
 (0)