Skip to content

Commit ea1ac85

Browse files
committed
Name the nodes of the extension dependency graph after the PR plan of #11114
Each node of the graph is now one PR, with the ID that the PR plan in #11114 uses. The claims node points to #11115, the old phase 1 node splits into NU and CL, and the shared pushdown and the lift rule become separate low-priority nodes. The graph also links #10661, #10686 and #10793 as open PRs to coordinate with. Signed-off-by: Connor Tsui <[email protected]>
1 parent 96edf62 commit ea1ac85

1 file changed

Lines changed: 74 additions & 61 deletions

File tree

‎research/extension-types/05-implementation.md‎

Lines changed: 74 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -42,114 +42,127 @@ The graph shows every change of the extension type work, grouped by phase, with
4242
solid arrow is a hard dependency: the later change must not land first. A dashed arrow is an order
4343
rule or a soft dependency, and a label such as "item 3" names the rule in "Order constraints". A
4444
dotted line without an arrow links an open PR that changes the same code, so the two need
45-
coordination. Each node is one change. Phase 3 lands as the four family PRs, and the other phases
46-
do not fix their PR split yet.
45+
coordination. Each node is one PR, and its ID is the ID of that PR in the PR plan of
46+
[#11114](https://github.com/vortex-data/vortex/issues/11114). Issue
47+
[#11115](https://github.com/vortex-data/vortex/issues/11115) tracks the claims (CL).
4748

4849
```mermaid
4950
flowchart TD
5051
subgraph PH0["Phase 0"]
51-
H["Path-consistency helper<br/>#11013"]:::merged
52+
PC["PC path-consistency helper<br/>#11013"]:::merged
5253
U5["U5 temporal dtype and value errors<br/>#11015"]:::merged
5354
U4["U4 + U8 nullability without revalidation, debug output<br/>#11012"]:::merged
5455
U3["U3 + U6 + U7 storage checks and read errors<br/>#11014"]:::merged
5556
U2["U2 matching dtypes for compares<br/>#11016"]:::merged
56-
DF["DataFusion extension literals<br/>#11067"]:::merged
57-
AX["Arrow export with metadata, from phase 7<br/>X24"]
57+
DF["DF DataFusion extension literals<br/>#11067"]:::merged
58+
AX["AX Arrow export with metadata, from phase 7<br/>X24"]
5859
U1["U1 no storage casts"]
59-
TS["Timestamp accepts every i64, DateTimeParts days<br/>#10953"]:::open
60-
SP1["Spatial validators and importers<br/>EXT-N2"]
61-
SP2["Other importers fail on nested nulls<br/>X29"]
60+
TS["TS Timestamp accepts every i64, DateTimeParts days<br/>#10953"]:::open
61+
SN1["SN1 spatial validators and importers<br/>EXT-N2"]
62+
SN2["SN2 other importers fail on nested nulls<br/>X29"]
6263
end
63-
H --> U2
64+
PC --> U2
6465
U2 -->|"item 1, lands together"| DF
65-
H --> U1
66+
PC --> U1
6667
AX -.->|"item 2"| U1
6768
6869
subgraph PH1["Phase 1"]
69-
D1["Storage declaration<br/>storage_dtype, validate_metadata, facts in metadata"]
70-
D2["Strict reads<br/>declared storage equals stored storage"]
71-
D3["Claims, conformance harness, new_unchecked"]
70+
SD["SD storage declaration<br/>storage_dtype, validate_metadata, facts in metadata"]
71+
SR["SR strict reads<br/>declared storage equals stored storage"]
72+
NU["NU new_unchecked, validate_array, conformance harness"]
73+
CL["CL claims<br/>#11115"]
7274
end
73-
U4 -->|"item 3"| D1
74-
U5 --> D1
75-
D1 --> D2
76-
SP1 -.->|"item 6"| D2
77-
D1 --> D3
75+
U4 -->|"item 3"| SD
76+
U5 --> SD
77+
SD --> SR
78+
SN1 -.->|"item 6"| SR
79+
SD --> NU
80+
SD --> CL
7881
7982
subgraph PH2["Phase 2"]
80-
E1["ExtensionUse, ResultMapping, Rewrap check"]
83+
FD["FD ExtensionUse, ResultMapping, Rewrap guard"]
8184
U9["U9 wrap storage pushdown results"]
8285
end
83-
D3 --> E1
84-
E1 --> U9
86+
CL --> FD
87+
NU --> FD
88+
FD --> U9
8589
8690
subgraph PH3["Phase 3"]
87-
T0["Typing check and generic storage rewrite"]
88-
T1["Compare and between, with the DateTimeParts rule X32"]
89-
T2["Aggregates, with the phase 4 writers and DuckDB pushdown"]
90-
T3["Structural functions"]
91-
T4["list_contains and the row encoding"]
91+
TY["TY typing check and generic storage rewrite"]
92+
TY1["TY1 compare and between, with the DateTimeParts rule X32"]
93+
TY2["TY2 aggregates, with the phase 4 writers and DuckDB pushdown"]
94+
TY3["TY3 structural functions"]
95+
TY4["TY4 list_contains and the row encoding"]
9296
end
93-
U9 --> T0
94-
U1 --> T0
95-
DF --> T0
96-
T0 --> T1
97-
T0 --> T2
98-
T0 --> T3
99-
T0 --> T4
100-
TS -.-> T1
101-
T2 -.->|"items 4 and 5"| PO["Any PartialOrd change for Scalar, XQ8"]
97+
U9 --> TY
98+
U1 --> TY
99+
DF --> TY
100+
TY --> TY1
101+
TY --> TY2
102+
TY --> TY3
103+
TY --> TY4
104+
TS -.-> TY1
105+
TY2 -.->|"items 4 and 5"| PO["Any PartialOrd change for Scalar, XQ8"]
102106
103107
subgraph PH4["Phase 4"]
104-
S1["Readers ignore min and max without Ord"]
105-
S2["Pruning through the claims"]
108+
ST1["ST1 readers ignore min and max without Ord"]
109+
ST2["ST2 pruning through the claims"]
106110
end
107-
D3 --> S1
108-
T2 --> S2
111+
CL --> ST1
112+
TY2 --> ST2
109113
110114
subgraph PH5["Phase 5"]
111-
R1["ExtVTable::rules over ReduceNode"]
112-
R2["Exact unit casts and the named coarser-unit function"]
113-
R3["Converter audit for literals and SQL casts"]
115+
RU1["RU1 ExtVTable::rules over ReduceNode"]
116+
RU2["RU2 exact unit casts and the named coarser-unit function"]
117+
RU3["RU3 converter audit for literals and SQL casts"]
114118
end
115-
T0 --> R1
116-
R1 --> R2
117-
R2 --> R3
119+
TY --> RU1
120+
RU1 --> RU2
121+
RU2 --> RU3
118122
119123
subgraph PH6["Phase 6"]
120-
V1["Value check in try_new"]
121-
V2["Strict read mode and value checks on import"]
124+
VC1["VC1 value check in try_new"]
125+
VC2["VC2 strict read mode and value checks on import"]
122126
end
123-
U3 --> V1
124-
D3 --> V1
125-
V1 --> V2
127+
U3 --> VC1
128+
NU --> VC1
129+
VC1 --> VC2
126130
127131
subgraph PH7["Phase 7"]
128-
B1["Converters for DuckDB, Python and JNI"]
129-
B2["Engine pushdown through the claims"]
130-
B3["Typed array view and developer guide"]
132+
BD1["BD1 converters for DuckDB, Python and JNI"]
133+
BD2["BD2 engine pushdown through the claims"]
134+
BD3["BD3 typed array view and developer guide"]
131135
end
132-
D3 --> B2
136+
CL --> BD2
133137
134138
subgraph LOW["Low priority"]
135-
L1["Shared pushdown and lift rule<br/>#10748, X34"]
136-
L2["DateTimeParts under the wrapper"]
139+
LP1["LP1 lift structural encodings above the wrapper<br/>#10748, X5"]
140+
LP2["LP2 shared pushdown through Masked and Sparse<br/>X34"]
141+
LP3["LP3 DateTimeParts under the wrapper"]
142+
LP4["LP4 registration without silent replacement<br/>EXT-R2"]
137143
end
138-
C1["Constant fold of ext_storage<br/>#11056"]:::open -.-> L1
139-
T1 -.->|"replaces the X32 rule"| L2
144+
CF["Constant fold of ext_storage<br/>#11056"]:::open -.-> LP2
145+
TY1 -.->|"replaces the X32 rule"| LP3
140146
141147
subgraph OTHER["Open PRs to coordinate with"]
142148
AG["vortex-aggregates move<br/>#10769"]:::open
143149
BB["Spatial bounding boxes in file statistics<br/>#10837"]:::open
144150
BT["between pushdown for temporal types<br/>#10408"]:::open
145151
DI["Duration and Interval<br/>#10467, #10468"]:::open
152+
CS["IsCompoundSorted<br/>#10661"]:::open
153+
PT["Comparisons proven true from statistics<br/>#10686"]:::open
154+
OS["optimize only with a session<br/>#10793"]:::open
146155
end
147156
AG -.- U9
148-
BB -.- E1
149-
BT -.- T1
157+
AG -.- TY2
158+
BB -.- FD
159+
BT -.- TY1
150160
DI -.- U1
151161
DI -.- U2
152-
DI -.- R2
162+
DI -.- RU2
163+
CS -.- TY2
164+
PT -.- ST2
165+
OS -.- RU1
153166
154167
classDef merged fill:#d7f0dc,stroke:#2f7a45,color:#10261a
155168
classDef open fill:#fbefc7,stroke:#a87c0f,color:#2b2105

0 commit comments

Comments
 (0)