Repository navigation
Commit e78d089
authored
[Chore](runtime-filter) refactor of try_append_late_arrival_runtime_filter (#59786)
This pull request refactors and simplifies the handling of late-arriving
runtime filters and conjunct context cloning in the scan pipeline. The
main changes include streamlining the interfaces for applying runtime
filters, improving thread safety with new locking mechanisms, and
removing redundant or duplicated code. The changes also clarify
ownership and lifecycle of conjunct expressions to prevent concurrent
access issues.
**Runtime filter and conjunct management improvements:**
* Refactored `try_append_late_arrival_runtime_filter` in
`RuntimeFilterConsumerHelper` to use references instead of pointers for
`arrived_rf_num` and conjuncts, and updated all call sites accordingly
for clearer and safer usage.
[[1]](diffhunk://#diff-ed4cbc22dcb1b5d27652827cea0f12373591a9b06e3c857f0dd0e32f58422026L99-R110)
[[2]](diffhunk://#diff-61d96ad649abf519ee21143b6a4d9ed0c1b820ba273281ca8650a1e047d10f15L47-R50)
[[3]](diffhunk://#diff-04a5f666c19cc90dfb8f222f1fadc43f08fa2d59f5ae61e2889e615e3cffc35dL101-R101)
[[4]](diffhunk://#diff-2c575051a29b06200611882d17cb4c0fea6df46659e8ea953c705aaca2c2b5a4L232-R245)
* Removed the redundant `clone_conjunct_ctxs` method from
`RuntimeFilterConsumerHelper` and moved conjunct cloning logic into
`ScanLocalStateBase`, introducing new methods
`update_late_arrival_runtime_filter` and `clone_conjunct_ctxs` for
better encapsulation.
[[1]](diffhunk://#diff-ac38473f151ccd99db4ed80dfbfa176f4f957ae2f5335ce8fbb4f5dbb0e932e9R76-R91)
[[2]](diffhunk://#diff-f5a0db9faf0b649227b0126cca493d8dce22c9d497116fd480d7d018ce1696d2R91-R94)
[[3]](diffhunk://#diff-ed4cbc22dcb1b5d27652827cea0f12373591a9b06e3c857f0dd0e32f58422026L123-R128)
[[4]](diffhunk://#diff-61d96ad649abf519ee21143b6a4d9ed0c1b820ba273281ca8650a1e047d10f15L47-R50)
[[5]](diffhunk://#diff-2c575051a29b06200611882d17cb4c0fea6df46659e8ea953c705aaca2c2b5a4L232-R245)
**Thread safety enhancements:**
* Added a `std::mutex _conjuncts_lock` to `ScanLocalStateBase` and used
it to guard access to `_conjuncts` for thread-safe updates and cloning.
[[1]](diffhunk://#diff-f5a0db9faf0b649227b0126cca493d8dce22c9d497116fd480d7d018ce1696d2R21)
[[2]](diffhunk://#diff-f5a0db9faf0b649227b0126cca493d8dce22c9d497116fd480d7d018ce1696d2R128)
[[3]](diffhunk://#diff-ac38473f151ccd99db4ed80dfbfa176f4f957ae2f5335ce8fbb4f5dbb0e932e9R76-R91)
[[4]](diffhunk://#diff-f5a0db9faf0b649227b0126cca493d8dce22c9d497116fd480d7d018ce1696d2R91-R94)
**Scanner and file scanner lifecycle fixes:**
* Changed handling of conjuncts in `FileScanner` and `Scanner` to avoid
destroying conjuncts that may still be in use, by explicitly moving them
to a `_stale_expr_ctxs` vector before clearing. Removed the
`_discard_conjuncts` method and replaced it with direct management.
[[1]](diffhunk://#diff-6afbc092caa913a8775be9772777060161de5608b9aae939202e963e8aaf4e8bL360-L366)
[[2]](diffhunk://#diff-2c575051a29b06200611882d17cb4c0fea6df46659e8ea953c705aaca2c2b5a4L232-R245)
[[3]](diffhunk://#diff-17432beb88ab0d918d9bbe41b3ba500e3f5102645896e89587a4dc43bcab2125L192-L198)
**Minor interface improvements:**
* Made `Scanner::is_open()` a `const` method for correctness.1 parent b80be52 commit e78d089
9 files changed
Lines changed: 47 additions & 54 deletions
File tree
- be
- src
- pipeline/exec
- runtime_filter
- vec/exec/scan
- test/runtime_filter
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
97 | 97 | | |
98 | 98 | | |
99 | 99 | | |
| 100 | + | |
| 101 | + | |
100 | 102 | | |
101 | | - | |
| 103 | + | |
102 | 104 | | |
103 | 105 | | |
104 | 106 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
73 | 73 | | |
74 | 74 | | |
75 | 75 | | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
76 | 94 | | |
77 | 95 | | |
78 | 96 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
| |||
87 | 88 | | |
88 | 89 | | |
89 | 90 | | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
90 | 95 | | |
91 | 96 | | |
92 | 97 | | |
| |||
120 | 125 | | |
121 | 126 | | |
122 | 127 | | |
| 128 | + | |
123 | 129 | | |
124 | 130 | | |
125 | 131 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
96 | 96 | | |
97 | 97 | | |
98 | 98 | | |
99 | | - | |
100 | | - | |
| 99 | + | |
| 100 | + | |
101 | 101 | | |
102 | | - | |
| 102 | + | |
103 | 103 | | |
104 | 104 | | |
105 | 105 | | |
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
109 | 109 | | |
110 | | - | |
| 110 | + | |
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
| |||
120 | 120 | | |
121 | 121 | | |
122 | 122 | | |
123 | | - | |
| 123 | + | |
124 | 124 | | |
125 | 125 | | |
126 | 126 | | |
127 | 127 | | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
| 128 | + | |
141 | 129 | | |
142 | 130 | | |
143 | 131 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
54 | 51 | | |
55 | 52 | | |
56 | 53 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
357 | 357 | | |
358 | 358 | | |
359 | 359 | | |
360 | | - | |
361 | | - | |
362 | | - | |
363 | | - | |
364 | | - | |
| 360 | + | |
| 361 | + | |
365 | 362 | | |
366 | | - | |
367 | 363 | | |
368 | 364 | | |
369 | 365 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
229 | 229 | | |
230 | 230 | | |
231 | 231 | | |
232 | | - | |
233 | 232 | | |
234 | | - | |
235 | | - | |
236 | | - | |
| 233 | + | |
237 | 234 | | |
238 | 235 | | |
239 | 236 | | |
240 | 237 | | |
241 | 238 | | |
242 | 239 | | |
243 | | - | |
244 | | - | |
245 | | - | |
246 | | - | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
247 | 243 | | |
248 | | - | |
249 | | - | |
250 | | - | |
251 | | - | |
252 | | - | |
| 244 | + | |
| 245 | + | |
253 | 246 | | |
254 | 247 | | |
255 | 248 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
160 | 160 | | |
161 | 161 | | |
162 | 162 | | |
163 | | - | |
| 163 | + | |
164 | 164 | | |
165 | 165 | | |
166 | 166 | | |
| |||
189 | 189 | | |
190 | 190 | | |
191 | 191 | | |
192 | | - | |
193 | | - | |
194 | | - | |
195 | | - | |
196 | | - | |
197 | | - | |
198 | | - | |
199 | 192 | | |
200 | 193 | | |
201 | 194 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
105 | | - | |
| 105 | + | |
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
| |||
0 commit comments