You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit fa50f1f
Browse filesBrowse the repository at this point in the historyBrowse files
Add order constraints and a findings table to the extension plan
The plan now lists the five places where a change that lands alone
breaks working behavior, and maps every finding to the change or phase
that fixes it.
Signed-off-by: Connor Tsui <[email protected]>
[#10895](https://github.com/vortex-data/vortex/pull/10895)), and a check of the DuckDB literal
61
+
conversion. Otherwise a filter that compares a `Uuid` or `Json` column with a storage literal
62
+
fails when it is built, instead of being pushed down.
63
+
2. The Arrow export of phase 7 lands before U1, or together with it. U1 makes a storage-like Arrow
64
+
target type an error, for example `Int64` for a timestamp column in the Python
65
+
`to_arrow_array(arrow_type=...)`. The export with metadata gives those callers a path that loses
66
+
nothing.
67
+
3. Phase 1 builds on U4 and U5. U4 changes `ExtDType::try_with_vtable`, where phase 1 adds the check
68
+
of the claims. U5 changes the `Date`, `Time` and `Timestamp` vtables, which get their claims in
69
+
phase 1. So phase 1 waits for both to merge, or it is based on them.
70
+
4. The aggregates PR of phase 3 includes two more changes:
71
+
- The writers' part of phase 4. Once `Min` and `Max` reject types without `Ord`, the storage
72
+
fallback of `aggregate_state_dtype` would write a storage-typed state for those columns. No
73
+
version wrote that form before.
74
+
- DuckDB aggregate pushdown reads the claims. Today it excludes only `Uuid`, so `max` over a
75
+
`Json` column would fail in DuckDB. Whether DuckDB falls back to its own aggregation when
76
+
binding fails is not checked.
77
+
5. A change of `PartialOrd` for `Scalar` (XQ8) lands with the aggregates PR of phase 3, or after it.
78
+
Today `min` and `max` merge partial results with `vortex_expect`, so an earlier change can panic.
79
+
80
+
Phase 2 and U9 also need coordination with
81
+
[#10769](https://github.com/vortex-data/vortex/pull/10769), which moves the aggregate
82
+
implementations that they change into `vortex-aggregates`.
83
+
50
84
## Phase 0: urgent fixes
51
85
52
86
- Write the path-consistency helper first, because U1 and U2 use it.
53
87
- Land U1 to U8 as separate PRs. U9 needs the declaration types, so it lands in phase 2.
88
+
- Land the Arrow part of phase 7 before U1, and change the DataFusion literal path in the PR of U2
89
+
("Order constraints", items 1 and 2).
54
90
- Some existing code depends on a leak, for example a comparison of an extension column with a
55
91
storage literal. The full test suite finds these callers. Fix each one with an explicit
56
92
`ext_storage`.
@@ -137,7 +173,8 @@ check to `try_new`.
137
173
only phase 1, so it can land early.
138
174
- Writers drop the storage fallback of `aggregate_state_dtype`, so types without `Ord` get no min
139
175
and max. Types with `Ord` keep their logical `Min` and `Max`, so the format does not change. This
140
-
needs phase 3, because `Min` and `Max` reject types without `Ord` only after it.
176
+
lands in the aggregates PR of phase 3, because `Min` and `Max` reject types without `Ord` only
177
+
after it ("Order constraints", item 4).
141
178
- Pruning compares logical statistics through the claims, or unwraps them to storage bounds. The
0 commit comments