Repository navigation
Commit 3e999c4
committed
fix: reject deterministic = false when registering a Rust UDF
`CometRustUDF.register` accepted a `deterministic` flag, installed a
nondeterministic Spark stub for it, and carried it in the RustUdfCall proto,
but nothing on the native side ever read it: ImportedCScalarUdf hardcodes
Volatility::Immutable. A UDF the caller declared nondeterministic was
therefore planned as pure and could be constant-folded, evaluated once and
reused, or eliminated as a common subexpression.
Honoring the flag needs the cached per-library signature to become per
registration, which is #5249. Until then, fail the registration rather than
let the parameter lie about what Comet does with it.
Raised in review: does the proto field's value mean RustUdfs are always
immutable?1 parent f20bc31 commit 3e999c4
4 files changed
Lines changed: 63 additions & 2 deletions
File tree
- native
- core/src/execution/rust_udf
- proto/src/proto
- spark/src
- main/scala/org/apache/comet/udf
- test/scala/org/apache/comet
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
81 | 81 | | |
82 | 82 | | |
83 | 83 | | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
84 | 90 | | |
85 | 91 | | |
86 | 92 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
636 | 636 | | |
637 | 637 | | |
638 | 638 | | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
639 | 644 | | |
640 | 645 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | | - | |
| 33 | + | |
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
39 | | - | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
40 | 44 | | |
41 | 45 | | |
42 | 46 | | |
| |||
52 | 56 | | |
53 | 57 | | |
54 | 58 | | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
55 | 63 | | |
56 | 64 | | |
57 | 65 | | |
| |||
60 | 68 | | |
61 | 69 | | |
62 | 70 | | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
63 | 82 | | |
64 | 83 | | |
65 | 84 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
248 | 248 | | |
249 | 249 | | |
250 | 250 | | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
251 | 282 | | |
252 | 283 | | |
253 | 284 | | |
| |||
0 commit comments