Repository navigation
Commit 3d37216
Add a Normalized encoding for tensor columns (#9138)
Adds `Normalized`, a norm-split array encoding for tensor columns with
`normalized` and `norms` children.
Tracking issue: #9129
`Normalized` owns the physical decomposition and validates the invariant
between its children. Scalar functions remain operations over any
well-typed input, while the encoding can guarantee that its direction
child is unit-normalized and that its stored norm is valid.
## Changes
- Renames `L2Denorm` and `encodings::l2_denorm` to `Normalized` and
`encodings::normalized`.
- `Normalized::try_new` validates exact normalized rows.
`Normalized::new_unchecked` admits structurally valid lossy children
whose stored norms remain authoritative.
- Slice and filter push down into both children, preserving the
encoding.
- `NormalizedScheme` cascades compression into both children.
- Cosine similarity, inner product, and L2 norm keep their read-through
paths for `Normalized` arrays.
## API changes
| Before | After |
| --- | --- |
| `scalar_fns::l2_denorm::*` | `encodings::normalized::*` |
| `L2Denorm::try_new_array(n, s, ctx)` | `Normalized::try_new(n, s,
ctx)` |
| `unsafe L2Denorm::new_array_unchecked(n, s)` | `unsafe
Normalized::new_unchecked(n, s)` |
| `L2Denorm::new()` | removed |
| `session.scalar_fns().register(L2Denorm)` |
`session.arrays().register(Normalized)`, unconditional |
| `ScalarFnArrayPlugin::new(L2Denorm)` | removed |
The array ID is now `vortex.tensor.normalized`; the serialized child
metadata layout is unchanged. `NormalizedScheme` remains out of
`ALL_SCHEMES`.
---------
Signed-off-by: Connor Tsui <[email protected]>
Co-authored-by: Claude <[email protected]>1 parent 3884b98 commit 3d37216
21 files changed
Lines changed: 1949 additions & 1366 deletions
File tree
- vortex-compressor/src/compressor
- vortex-tensor
- src
- encodings
- normalized
- scalar_fns
- vortex/src/editions/unstable
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
27 | 26 | | |
28 | 27 | | |
29 | 28 | | |
| |||
177 | 176 | | |
178 | 177 | | |
179 | 178 | | |
180 | | - | |
181 | | - | |
182 | | - | |
183 | | - | |
184 | 179 | | |
185 | 180 | | |
186 | 181 | | |
| |||
323 | 318 | | |
324 | 319 | | |
325 | 320 | | |
326 | | - | |
327 | | - | |
| 321 | + | |
328 | 322 | | |
329 | 323 | | |
330 | 324 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
| 38 | + | |
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
| 9 | + | |
0 commit comments