Repository navigation
Commit 9ef3d47
authored
fix: Address avg return type overflow for decimals (apache#24371)
## Which issue does this PR close?
- Closes apache#24369
## Rationale for this change
`AVG(Decimal(p, s))` derives a result with precision and scale increased
by four. For example, AVG(Decimal32(9, 0)) returned Decimal32(9, 4),
although the rescaled average can require 13 digits.
This caused valid averages near the input precision limit to fail with
an overflow even though Decimal64(13, 4) can represent the result.
## What changes are included in this PR?
- Preserve the input decimal variant when the derived result fits, and
widen it when necessary, while still clamping Decimal256 at MAX.
- Allow decimal AVG accumulators to use different native types for the
input, intermediate sum, and output.
- Update SQL type assertions and affected logical-plan expectations.
## Are these changes tested?
Regression tests and adjusted SLT-based tests.
## Are there any user-facing changes?
Yes. The Arrow result type of AVG(decimal) may now be wider when the
derived precision does not fit the input variant.
---------
Signed-off-by: Adam Gutglick <[email protected]>1 parent 15c097f commit 9ef3d47
4 files changed
Lines changed: 376 additions & 155 deletions
File tree
- datafusion
- functions-aggregate-common/src/aggregate/avg_distinct
- functions-aggregate/src
- sqllogictest/test_files
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
35 | | - | |
36 | | - | |
37 | | - | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
42 | 43 | | |
43 | 44 | | |
44 | 45 | | |
45 | 46 | | |
46 | 47 | | |
47 | | - | |
| 48 | + | |
48 | 49 | | |
49 | 50 | | |
50 | | - | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
51 | 57 | | |
52 | 58 | | |
53 | 59 | | |
| |||
60 | 66 | | |
61 | 67 | | |
62 | 68 | | |
63 | | - | |
| 69 | + | |
64 | 70 | | |
65 | 71 | | |
66 | 72 | | |
| |||
78 | 84 | | |
79 | 85 | | |
80 | 86 | | |
81 | | - | |
| 87 | + | |
82 | 88 | | |
83 | 89 | | |
84 | 90 | | |
| 91 | + | |
85 | 92 | | |
86 | 93 | | |
| 94 | + | |
87 | 95 | | |
88 | 96 | | |
89 | 97 | | |
| |||
98 | 106 | | |
99 | 107 | | |
100 | 108 | | |
101 | | - | |
| 109 | + | |
102 | 110 | | |
103 | 111 | | |
104 | | - | |
| 112 | + | |
105 | 113 | | |
106 | 114 | | |
107 | 115 | | |
| |||
123 | 131 | | |
124 | 132 | | |
125 | 133 | | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
130 | 137 | | |
131 | | - | |
| 138 | + | |
132 | 139 | | |
133 | 140 | | |
134 | | - | |
| 141 | + | |
135 | 142 | | |
136 | 143 | | |
137 | | - | |
| 144 | + | |
138 | 145 | | |
139 | 146 | | |
140 | 147 | | |
| |||
0 commit comments