Repository navigation
Commit 733a74b
committed
fix: stop a broadcast read within a piece of an interrupt, and free a decode its task outlived
Arrow reads a whole message body with one read, which for a coalesced
broadcast is all of it, so the read channel now checks for an interrupt
before each piece, as Channels.newChannel did, instead of once per read.
A task whose completion listener runs on another thread can give back its
hold while its read decodes the broadcast. If it was the last holder, the
entry was freed before the decode finished, and the decode was kept where
nothing would free it. The decode is now freed instead, and the read
decodes the broadcast for itself.1 parent 2a68f4e commit 733a74b
4 files changed
Lines changed: 74 additions & 8 deletions
File tree
- spark/src
- main/scala/org/apache/spark/sql/comet
- util
- test/scala/org/apache/spark/sql/comet
- util
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | | - | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
59 | 61 | | |
60 | 62 | | |
61 | 63 | | |
| |||
132 | 134 | | |
133 | 135 | | |
134 | 136 | | |
135 | | - | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
136 | 150 | | |
137 | 151 | | |
138 | 152 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
359 | 359 | | |
360 | 360 | | |
361 | 361 | | |
362 | | - | |
363 | | - | |
| 362 | + | |
| 363 | + | |
364 | 364 | | |
365 | 365 | | |
366 | 366 | | |
| |||
370 | 370 | | |
371 | 371 | | |
372 | 372 | | |
373 | | - | |
374 | | - | |
375 | | - | |
376 | | - | |
377 | 373 | | |
378 | 374 | | |
379 | 375 | | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
380 | 380 | | |
381 | 381 | | |
382 | 382 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
211 | 211 | | |
212 | 212 | | |
213 | 213 | | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
214 | 237 | | |
215 | 238 | | |
216 | 239 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
| 23 | + | |
22 | 24 | | |
23 | 25 | | |
24 | 26 | | |
| |||
28 | 30 | | |
29 | 31 | | |
30 | 32 | | |
| 33 | + | |
31 | 34 | | |
32 | 35 | | |
33 | 36 | | |
| |||
142 | 145 | | |
143 | 146 | | |
144 | 147 | | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
145 | 174 | | |
146 | 175 | | |
147 | 176 | | |
| |||
0 commit comments