Repository navigation
Add ExtensionUse and the storage equality and order lists - #11186
connortsui20 wants to merge 2 commits into
Conversation
Scalar functions, aggregates, and row functions get an `extension_use` method that declares how they use operands whose dtypes contain extension dtypes. The default is `ExtensionUse::None`, and the erased `ScalarFnRef` and `AggregateFnRef` forward the method, so a typing check can later ask any bound function. Row functions get their own hook, because the blanket vtable would otherwise keep them from declaring anything. Nothing reads the declarations yet, so behavior does not change. Signed-off-by: Connor Tsui <[email protected]>
Extension types cannot declare yet that their storage gives their equality or their order, so `storage_gives_equality` and `storage_gives_order` answer from fixed lists of extension IDs: `Date`, `Time`, `Timestamp`, and `Uuid` for both, and `Vector` and `FixedShapeTensor` for equality only. The answers cover every depth and reject placeholders of unknown types, the same as the claims that will replace the lists. `Uuid`, `Vector`, and `FixedShapeTensor` live in other crates, so their own tests catch an ID that drifts from the lists. Signed-off-by: Connor Tsui <[email protected]>
TS (#10953) merged on 2026-11-03, and SN1 (#11183) and V10 (#11187) merged on 2026-11-04. The dependency graph marks the three merged and moves its state date to 2026-11-04. It gives SD, DP, FD and U9 their open PRs (#11184, #11185, #11186 and #11188), and it drops #10408, which closed without a merge. The text of parts 4 and 5 and the Fix lines of EXT-V9, EXT-V10, EXT-N2 and EXT-S3 now record the same state. Also record that AX, DS and SN2 are implemented on branches without a PR, and add the approved one-pass design for the masked nulls of SN2. Part 2 and decision X31 also record the fixes: #10953 made every i64 legal, #11183 made the spatial validators reject nullable nested levels, #11187 gave the Date display its fallback, and #10408 closed. Signed-off-by: Connor Tsui <[email protected]>
gatesn
left a comment
There was a problem hiding this comment.
The extension type vtable should declare its properties, not every function that uses it.
Merging this PR will improve performance by 12.97%1 warning
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
|
yeah this overloads a few things |
Record the merged AX and DS, the new RT, SF and AF nodes, and the related PRs that closed or merged. Mark the nodes that wait for a new design of the function declaration, since FD (#11186) closed after review. Signed-off-by: Connor Tsui <[email protected]>
Summary
Many functions still apply the semantics of the storage to extension values, so
min_maxover aJsoncolumn compares its raw text. A typing check can only reject calls like that once every function says how it uses extension operands, and once one place knows which types take their equality and order from their storage. This adds both without any behavior change, so the built-in functions can declare their use one family at a time (TY1 to TY5 in #11114) before the check turns on.Changes
Each function answers once per call, from its options and argument dtypes:
Once the typing check turns on, it rejects a call that answers
Nonefor an extension operand. Somin_maxover aTimestampcolumn keeps working, andmin_maxover aJsoncolumn fails.The two lists are hardcoded until types can declare their own equality and order (#11115). They name types by extension ID, since
Uuid,Vector, andFixedShapeTensorlive in crates that depend onvortex-array.RowFngets its ownextension_usehook, because the blanketScalarFnVTableimpl would otherwise keep every row function atNone.If #10467 lands after this, it needs to add
vortex.durationtoORDER_IDS.API Changes
Adds
ExtensionUse,storage_gives_equality, andstorage_gives_order, and a defaultedextension_usemethod onScalarFnVTable,AggregateFnVTable, andRowFn.