Skip to content

Commit ae16c5b

Browse files
committed
docs(reports): slot 3 (2026-10-17) — pin the signed-zero PARTITION BY bug with its self-inconsistency
* +1 probe (79 -> 80): window PARTITION BY over {0.0, -0.0, 1.0}. DataFusion puts the two zeros in separate partitions (n = 1 each) while its own GROUP BY merges them (n = 2), and DuckDB merges both. That contrast is the strongest evidence that the window path is the wrong side, and it is now in the probe's why. * the divergence is apache/datafusion#25766, still open, and re-verified on main with the slt-runner oracle: three partitions there too. Recorded as known_difference so it is not re-triage and so a fix shows up immediately. * triaged and recorded as non-bugs: MERGE/DELETE/UPDATE on read-only listing tables (by design, INSERT even suggests StreamTable), GROUP BY () not implemented (a tiny gap with a clean message, deliberately not filed), duplicate GROUPING SETS sets, regexp_replace flags and backreferences. * gate 7/7, scalars 391/20 and aggregates 191/13 reproduce item for item.
1 parent 71aa123 commit ae16c5b

3 files changed

Lines changed: 89 additions & 1 deletion

File tree

‎probes/semantics.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -762,6 +762,14 @@
762762
]
763763
],
764764
"why": "`GROUP BY ALL` must group by every non-aggregated projection expression -- here the COALESCE, not the bare `g` -- and the NULL group has to survive as its own group (2 rows in probe_values), because NULL is a group key like any other. Pinning both facts at once. Caution for future slots: write the plain `SELECT g, COUNT(*) ... GROUP BY ALL` form instead and the probe diverges, because DuckDB drops the NULL group there; keep the COALESCE so the two engines stay comparable."
765+
},
766+
{
767+
"id": "window_partition_signed_zero",
768+
"category": "float_semantics",
769+
"sql": "SELECT x, COUNT(*) OVER (PARTITION BY x) AS n FROM (VALUES (0.0), (-0.0), (1.0)) t(x) ORDER BY n, x",
770+
"ordered": true,
771+
"why": "Signed zero is the sharpest test of what an engine means by \"equal enough to group together\": 0.0 = -0.0 is true, so a PARTITION BY that treats them as equal gives n = 2 for both rows, and one that compares bit patterns gives n = 1 for each. The probe is here because DataFusion answers the two ways in the same engine: `GROUP BY x` merges them (n = 2), the window path does not (n = 1) -- and DuckDB merges them in both.",
772+
"known_difference": "Known upstream, still open -- do not re-file. apache/datafusion#25766 'Window PARTITION BY puts 0.0 and -0.0 in separate partitions' (open since 2026-09-26). Measured 2026-10-17 on the pinned 55.2.0 *and* on main via the sqllogictest runner: the window path returns n = 1 for 0.0, -0.0 and 1.0 (three partitions), while `GROUP BY x` on the same values returns n = 2 for the two zeros. DuckDB groups them in both cases. The asymmetry with DataFusion's own GROUP BY is the strongest evidence that the window path is the wrong side of the inconsistency; recorded here so no slot re-triages it, and so a fix upstream is visible."
765773
}
766774
]
767775
}

‎reports/datafusion-findings.md‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1169,3 +1169,57 @@ Arrow error: Cast error: Can't cast value inf to type Int64
11691169

11701170
**未发**:13:00 的规则是只在提了上游产物 / 准备了待点开的分支 / 运行失败时发。本 slot 准备的是一份
11711171
issue 文本(不在触发条件里),也没有新分支,所以不发;21:00 的日报会带上它。
1172+
1173+
## 2026-10-17 17:00 — slot 3
1174+
1175+
**结论**:80 探针(+1)· 0 个待研判候选 · Tier-A 四族逐条复现(零 diff)· 健康闸门 7/7 ·
1176+
新把 **一个仍在 main 上复现的已知上游 bug** 钉进目录(附一条很强的旁证)
1177+
1178+
| 阶段 | 结果 |
1179+
|---|---|
1180+
| Phase 0 健康闸门 | **7/7 PASS**(fresh clone,build 6m24s,duckdb 1.5.6) |
1181+
| Phase 1a 标量 / 聚合 / 窗口 | 391/20 · 191/13 · 0 —— 与基线逐条一致(`reports/df-robust*.json` 零 diff) |
1182+
| Phase 1b 语义 | **80 探针** · OK=67 · KNOWN_DIFF=11 · BOTH_ERROR=2 · **0 候选** |
1183+
| Phase 4 | +1 探针(`PARTITION BY` 的有符号零) |
1184+
| 上游产物 | 0(PR 预算仍留给待点开的 `nth-value-zero-index`) |
1185+
1186+
### Phase 0b — 仍然没有在审的 PR
1187+
1188+
`gh pr list --author SingletonYe` 只有 #26329(MERGED),`fix/nth-value-zero-index`(`dcaac3e1`)
1189+
继续等用户点开,已经等了约 14 小时。今天的时间片不再动这个分支,也不新起分支。
1190+
1191+
### 本 slot 的产出:把"同一个引擎自己不一致"这件事钉死
1192+
1193+
给有符号零找新角度时撞到一个**同引擎内部自相矛盾**的形状:
1194+
1195+
```sql
1196+
SELECT x, COUNT(*) OVER (PARTITION BY x) AS n FROM (VALUES (0.0),(-0.0),(1.0)) t(x) ORDER BY n, x;
1197+
-- DataFusion: -0.0,1 | 0.0,1 | 1.0,1 ← 0.0 与 -0.0 被分到两个分区
1198+
SELECT x, COUNT(*) AS n FROM (VALUES (0.0),(-0.0),(1.0)) t(x) GROUP BY x;
1199+
-- DataFusion: 0.0,2 | 1.0,1 ← 同一个引擎的 GROUP BY 又把它们合在一起
1200+
```
1201+
1202+
DuckDB 两种写法都合并(`n = 2`)。也就是说:**DataFusion 自己的 `GROUP BY` 与
1203+
`PARTITION BY` 对"0.0 = -0.0"给出了相反的回答**,而窗口那侧与 DuckDB 不一致。
1204+
这不是新 bug —— 对应上游 **apache/datafusion#25766**("Window PARTITION BY puts 0.0 and -0.0 in
1205+
separate partitions",2026-09-26 开,至今 OPEN)。本轮用 slt oracle 在 **main 上实跑确认它仍未修**
1206+
(同样返回三个 n=1),因此按 `known_difference` 收进目录:既避免以后重复研判,也在上游修好时
1207+
第一时间可见。`why` 里写明了那条"与自己的 GROUP BY 不对称"的旁证 —— 这是判断"哪一侧是错的"
1208+
最有力的证据。
1209+
1210+
### 顺手排掉的几个方向(都不构成 bug)
1211+
1212+
* **`MERGE` / `DELETE` / `UPDATE` 在 lakewatch 的表上报 NOT_IMPL**:这些是只读的 listing table
1213+
(CSV),不是引擎缺功能;`INSERT` 的报错还带了"用 StreamTable / CREATE UNBOUNDED EXTERNAL TABLE"
1214+
的提示,信息很清楚。`{e:?}` 那种内部 Debug 泄漏问题在上一轮已经单独记录。
1215+
* **`GROUP BY ()`(空分组集)报 `This feature is not implemented: Empty tuple not supported yet`**:
1216+
SQL 标准里有这个写法,DuckDB 也支持,属于一个**很小的缺口**,但报错干净、没有用户可误解之处,
1217+
而且 `GROUPING SETS ((g),())` 里的空集是能用的 —— 按 NOT_IMPL 的研判口径("真缺口 or 文档化限制")
1218+
记为已知限制,**不提 issue**(避免为这种边缘写法占用维护者时间)。
1219+
* **`GROUPING SETS` 里重复集合**按标准保留重复行 ✓;`regexp_replace` 的 `'g'` 标志与 `$2$1`
1220+
反向引用都正常 ✓。
1221+
1222+
### 邮件
1223+
1224+
**未发**:17:00 的规则是只在提了上游产物 / 准备了待点开的分支 / 运行失败时发。本 slot 是
1225+
"把已知上游 bug 钉进目录",三者都不是。21:00 的日报会把今天四段合起来发。

‎reports/df-diff.json‎

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
"binary": "/home/user/workspace/lakewatch-daily/target/debug/lakewatch",
44
"counts": {
55
"OK": 67,
6-
"KNOWN_DIFF": 10,
6+
"KNOWN_DIFF": 11,
77
"BOTH_ERROR": 2
88
},
99
"findings": [
@@ -164,6 +164,19 @@
164164
"documented divergence: datafusion [4 row(s)] except_all,2 | intersect_all,1 | intersect_all,1 | intersect_all,1 vs duckdb [4 row(s)] except_all,1 | except_all,2 | intersect_all,1 | intersect_all,1"
165165
],
166166
"stderr": "lakewatch: Optimizer rule 'simplify_expressions' failed\ncaused by\nArrow error: Cast error: Cannot cast string '2024-02-30' to value of Date32 type: Arrow error: Cast error: Cannot cast string '2024-02-30' to value of Date32 type: Cast error: Cannot cast string '2024-02-30' to value of Date32 type"
167+
},
168+
{
169+
"id": "window_partition_signed_zero",
170+
"category": "float_semantics",
171+
"sql": "SELECT x, COUNT(*) OVER (PARTITION BY x) AS n FROM (VALUES (0.0), (-0.0), (1.0)) t(x) ORDER BY n, x",
172+
"why": "Signed zero is the sharpest test of what an engine means by \"equal enough to group together\": 0.0 = -0.0 is true, so a PARTITION BY that treats them as equal gives n = 2 for both rows, and one that compares bit patterns gives n = 1 for each. The probe is here because DataFusion answers the two ways in the same engine: `GROUP BY x` merges them (n = 2), the window path does not (n = 1) -- and DuckDB merges them in both.",
173+
"ordered": true,
174+
"session_options": [],
175+
"status": "KNOWN_DIFF",
176+
"details": [
177+
"documented divergence: datafusion [3 row(s)] -0.0,1 | 0.0,1 | 1.0,1 vs duckdb [3 row(s)] 1.0,1 | 0.0,2 | 0.0,2"
178+
],
179+
"stderr": "lakewatch: Optimizer rule 'simplify_expressions' failed\ncaused by\nArrow error: Cast error: Cannot cast string '2024-02-30' to value of Date32 type: Arrow error: Cast error: Cannot cast string '2024-02-30' to value of Date32 type: Cast error: Cannot cast string '2024-02-30' to value of Date32 type"
167180
}
168181
],
169182
"results": [
@@ -996,6 +1009,19 @@
9961009
"session_options": [],
9971010
"status": "OK",
9981011
"details": []
1012+
},
1013+
{
1014+
"id": "window_partition_signed_zero",
1015+
"category": "float_semantics",
1016+
"sql": "SELECT x, COUNT(*) OVER (PARTITION BY x) AS n FROM (VALUES (0.0), (-0.0), (1.0)) t(x) ORDER BY n, x",
1017+
"why": "Signed zero is the sharpest test of what an engine means by \"equal enough to group together\": 0.0 = -0.0 is true, so a PARTITION BY that treats them as equal gives n = 2 for both rows, and one that compares bit patterns gives n = 1 for each. The probe is here because DataFusion answers the two ways in the same engine: `GROUP BY x` merges them (n = 2), the window path does not (n = 1) -- and DuckDB merges them in both.",
1018+
"ordered": true,
1019+
"session_options": [],
1020+
"status": "KNOWN_DIFF",
1021+
"details": [
1022+
"documented divergence: datafusion [3 row(s)] -0.0,1 | 0.0,1 | 1.0,1 vs duckdb [3 row(s)] 1.0,1 | 0.0,2 | 0.0,2"
1023+
],
1024+
"stderr": "lakewatch: Optimizer rule 'simplify_expressions' failed\ncaused by\nArrow error: Cast error: Cannot cast string '2024-02-30' to value of Date32 type: Arrow error: Cast error: Cannot cast string '2024-02-30' to value of Date32 type: Cast error: Cannot cast string '2024-02-30' to value of Date32 type"
9991025
}
10001026
]
10011027
}

0 commit comments

Comments
 (0)