You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit e31250d
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: research/decimal/04-implementation.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -79,7 +79,7 @@ planning review on 2026-10-23.
79
79
| CD4p | Decimal `Interleave` execution, so that CD4 can test its fallbacks | None | Medium |
80
80
| CD4 | Decimal `Widen` in memory, `WidenArray::from_stored_decimal` (D40), the decimal `Sum` and `MinMax` kernels (D36), and the serializer lowering (D26) | WD1, WD3, WD4, CD1, CD3, CD3b, CD4p | Large |
81
81
| CD4b | Export decimal `Widen` arrays to Arrow with one copy | CD4 | Small |
82
-
| LT3 | Sum, min, and max on the DecimalByteParts MSP, and the reshaped DecimalByteParts benchmarks (D19, D42) | WD3, CD1 | Medium |
82
+
| LT3 | Sum, min, and max on the DecimalByteParts MSP (D19). No benchmark changes (D42) | WD3, CD1 | Medium |
83
83
| CD5 | DecimalByteParts is the decimal over an integer child: the MSP keeps the canonical width, capped at `i64`, a narrower MSP is an integer `Widen`, and decoding needs no scan (D16). CD7 is folded in, so one PR holds both. #10905 closed on 2026-11-05 | WD1, LT3, CD1, CD3, CD3b, #11284, the CD2 fixtures in 0.91.0 | Large |
84
84
| CD6 | The `vortex.decimal` read plugin converts old nodes through `WidenArray::from_stored_decimal`| FX1, CD3, CD3b, CD4, the CD2 fixtures in 0.91.0 | Medium |
85
85
| CD7 | One DecimalByteParts scheme on the canonical array, which stores a narrower MSP as an integer `Widen`, also next to lower parts (D17). Folded into CD5 on 2026-11-05. #10992 closed on 2026-11-05 | WD1, CD1, CD3b, CD5 | Medium |
Copy file name to clipboardExpand all lines: research/decisions.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -66,9 +66,9 @@ decimal encoding.
66
66
| D37 | The `vortex.decimal` read plugin converts old narrow nodes on the host, also on CUDA read paths. A GPU scan of such a node copies its buffer to the host | Old narrow nodes are rare, and CUDA has no `Widen` kernel |
67
67
| D38 | The integer `Widen` PRs are written from scratch from their briefs. The local rewrite of #10223 and #10332 is a reference for tests and edge cases, not a base. FX3 ports the rewrite's dictionary fill fix as it is | The rewrite predates D25, D26, D28 and D36, so a cherry-pick would carry its old structure into the new PRs |
68
68
| D39 | The encoding is renamed from `Narrow` to `Widen`, after the widening cast its decode applies ([#9156](https://github.com/vortex-data/vortex/issues/9156)). The compressor scheme stays `Narrow` and produces `Widen` arrays. The wire ID is `vortex.widen`. Plan IDs NR1 to NR6 become WD1 to WD6, and benchmark input `Wide` becomes `Canonical`| Encodings are named after the function their decode applies, like `FoR` is an addition |
69
-
| D40 | `WidenArray::from_stored_decimal(parts, known_range: Precision<RangeInclusive<i256>>, ctx)` is the single owner of the choice between the canonical array and a `Widen` for stored decimal values. CD4 adds it, and CD6 uses it. Since 2026-11-05, DecimalByteParts decoding does not use it (D16). It first checks the buffer length, the alignment and the validity length, and returns an error on a mismatch. At the canonical width it returns the canonical array with no copy and no scan, and a device buffer stays on the device. Any other width copies the values to the host. `Precision::Absent` scans the valid values, and `Exact` is the smallest and the largest valid value. The child records either one as exact `Stat::Min` and `Stat::Max`. `Inexact` is only a bound, and the child records it as inexact. `WidenArray::encode_decimal` shares the core that picks the child. The function that picks the child dtype, `smallest_decimal_dtype`, is crate-private | One function owns the child label (D10), so two readers cannot choose different children for the same values. A false label gives wrong answers in the compare shortcut and in casts. A read never panics, so the checks come before any code that asserts on a length |
69
+
| D40 | `WidenArray::from_stored_decimal(parts, known_range: Precision<RangeInclusive<i256>>, ctx)` is the single owner of the choice between the canonical array and a `Widen` for stored decimal values. CD4 adds it, and CD6 uses it. Since 2026-11-05, DecimalByteParts decoding does not use it (D16). It first checks the buffer length, the alignment and the validity length, and returns an error on a mismatch. At the canonical width it returns the canonical array with no copy and no scan, and a device buffer stays on the device. Any other width copies the values on the host. A device buffer is an error until #11284 reads legacy nodes on the GPU. `Precision::Absent` scans the valid values, and `Exact` is the smallest and the largest valid value. The child records either one as exact `Stat::Min` and `Stat::Max`. `Inexact` is only a bound, and the child records it as inexact. `WidenArray::encode_decimal` shares the core that picks the child. The function that picks the child dtype, `smallest_decimal_dtype`, is crate-private | One function owns the child label (D10), so two readers cannot choose different children for the same values. A false label gives wrong answers in the compare shortcut and in casts. A read never panics, so the checks come before any code that asserts on a length |
70
70
| D41 | The PRs land in this order. CD3c, "Speed up the decimal widening copy", is a PR off `develop`. CD4 is off `develop` too and does not depend on CD3c. CD4b, "Export decimal Widen arrays to Arrow with one copy", is stacked on CD4. CD5 and CD6 are each stacked on CD4, and CD7 is stacked on CD5. CD3c, CD4 and CD4b have no release gate. CD3c merges first, so that the CodSpeed run of CD4 measures the fast copy. CD5 and CD6 wait for 0.91.0 (D31), and CD7 merges right after CD5. CD5, CD7 and CD6 aim for one release, with CD4b already merged. On 2026-11-05, #10905 (CD5) and #10992 (CD7) closed for the redesign of D16. LT3 lands on `develop` first. One PR for CD5, with CD7 folded in, stacks on LT3 and merges only after #11284. The new rule, the reader, and the writer must change together: a CD5 alone would need a temporary writer fix for the `narrowed_decimal` code that CD7 deletes, and a CD7 alone would put `Widen` MSPs in memory before the decoder and the GPU handle them. The PR can split later if review needs it. The scan design does not land in between, even if the replacements miss the release of CD6 | CD5 needs the decimal `Widen` and the helper of D40, so it moves onto CD4. Paths that need the canonical array, such as DuckDB, Arrow above precision 18 and the `canonicalize` bench, still pay one widening copy (D3), so CD3c makes that copy fast. CD4b removes one of the two copies of an Arrow export up to precision 18 |
71
-
| D42 | The DecimalByteParts benchmarks in `encodings/decimal-byte-parts/benches/dbp_compute.rs`measure the operations on the DecimalByteParts array: `sum`, `min_max`, and both compare benches run on it, and `filter`stops at it. `decode` and `canonicalize` become one `canonicalize` bench with `execute::<DecimalArray>`, which tracks the widening copy. The canonical-width control stays. These changes land with LT3. How `decompress` in `vortex/benches/common_encoding_tree_throughput.rs`changes is open| DecimalByteParts never decodes to a `Widen` (D16), so a bench that stops at a `Widen` runs to the canonical array. The narrow form shows only in the operations that run on the parts. The 2026-10-29 version stopped `decode` and `filter` at a `Widen` with `execute_until::<Widen>`|
71
+
| D42 | The DecimalByteParts benchmarks in `encodings/decimal-byte-parts/benches/dbp_compute.rs`keep their `develop` form: `decode` runs to the canonical array with `execute::<DecimalArray>`, which tracks the widening copy, `sum`, `min_max`, and both compare benches run on the DecimalByteParts array, and `filter`runs to the canonical array. The canonical-width control stays. LT3 changes no benchmark, and `decode` keeps its name, so CodSpeed keeps its history. CD5 only builds the narrow MSP inputs as a `Widen`. `decompress` in `vortex/benches/common_encoding_tree_throughput.rs`keeps its `develop` form too| DecimalByteParts never decodes to a `Widen` (D16), so a bench that stops at a `Widen` runs to the canonical array. The narrow form shows only in the operations that run on the parts. The 2026-10-29 version stopped `decode` and `filter` at a `Widen` with `execute_until::<Widen>`|
72
72
73
73
D1 to D6 date from 2026-10-19, D7 and D8 from 2026-10-22, and D12 to D38 from 2026-10-23. On
74
74
2026-10-23 the maintainer also revised D9, D10 and D11. Their 2026-10-22 versions used `P(W)`, the
0 commit comments