Repository navigation
Commit 1e08791
committed
docs: say where shuffle offsets are stored as lengths
The tuning guide and the config description now say that native shuffle and JVM shuffle blocks
without dictionary-encoded columns store offsets as lengths with zstd, and that Celeborn shuffle
blocks do not. native_shuffle.md describes the codec-tag flag, and the native and JVM shuffle
config tables list spark.comet.shuffle.compression.offsetsAsLengths.enabled.1 parent 3194f1a commit 1e08791
4 files changed
Lines changed: 22 additions & 10 deletions
File tree
- docs/source
- contributor-guide
- user-guide/latest/tuning
- spark/src/main/scala/org/apache/comet
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
215 | 215 | | |
216 | 216 | | |
217 | 217 | | |
218 | | - | |
219 | | - | |
220 | | - | |
221 | | - | |
222 | | - | |
223 | | - | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
191 | 191 | | |
192 | 192 | | |
193 | 193 | | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
194 | 198 | | |
195 | 199 | | |
196 | 200 | | |
| |||
270 | 274 | | |
271 | 275 | | |
272 | 276 | | |
273 | | - | |
274 | | - | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
275 | 281 | | |
276 | 282 | | |
277 | 283 | | |
| |||
510 | 516 | | |
511 | 517 | | |
512 | 518 | | |
| 519 | + | |
513 | 520 | | |
514 | 521 | | |
515 | 522 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
117 | 117 | | |
118 | 118 | | |
119 | 119 | | |
120 | | - | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
121 | 124 | | |
122 | 125 | | |
123 | 126 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
830 | 830 | | |
831 | 831 | | |
832 | 832 | | |
833 | | - | |
| 833 | + | |
| 834 | + | |
834 | 835 | | |
835 | 836 | | |
836 | 837 | | |
| |||
0 commit comments