Repository navigation
Commit 3c982c8
committed
docs: qualify direct read, explain.rules, and shuffle child rules
- plugin_overview.md: a shuffle input is a ShuffleScanExec only when it
is serialized as a ShuffleScan leaf, which needs direct read, an AQE
query stage, and supported output types. Without AQE, the shuffle is
read through ScanExec.
- understanding-comet-plans.md: spark.comet.explain.rules logs the plan
before and after CometScanRule, CometExecRule and
EliminateRedundantTransitions, not stage reverts. Point to
spark.comet.explain.fallback.log.enabled for those.
- jvm_shuffle.md: scope the columnar-to-row diagram to children that
produce Arrow batches. A CometSparkToColumnarExec child returns its
input rows directly.
- native_shuffle.md: with the Celeborn shuffle manager, native shuffle
needs a CometNativeExec child, not any CometPlan.
Part of #4419.
Generated-by: OpenCode1 parent 56396e3 commit 3c982c8
4 files changed
Lines changed: 19 additions & 11 deletions
File tree
- docs/source
- contributor-guide
- user-guide/latest
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
68 | | - | |
69 | | - | |
70 | | - | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
71 | 72 | | |
72 | 73 | | |
73 | 74 | | |
| |||
81 | 82 | | |
82 | 83 | | |
83 | 84 | | |
84 | | - | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
85 | 90 | | |
86 | 91 | | |
87 | 92 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
57 | | - | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
58 | 60 | | |
59 | 61 | | |
60 | 62 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
158 | | - | |
159 | | - | |
160 | | - | |
161 | | - | |
162 | | - | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
163 | 164 | | |
164 | 165 | | |
165 | 166 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
95 | 95 | | |
96 | 96 | | |
97 | 97 | | |
98 | | - | |
| 98 | + | |
99 | 99 | | |
100 | 100 | | |
101 | 101 | | |
| |||
0 commit comments