You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 1f21115
Browse filesBrowse the repository at this point in the historyBrowse files
pyarrow 26.0.0: the matrix and the Python suites run on it (venv, CI pin). Four pyarrow bugs the matrix worked around on 25.0.1 are fixed in 26.0.0 (utf8_normalize composing for NFC/NFKC by our apache/arrow#51237; pairwise_diff, the boolean fill-null kernels and winsorize honouring the offset): their pinning tests failed as designed and are retired with the _materialised copies and the unicodedata oracle; binary_slice's default-stop overflow remains (apache/arrow#51222) with its test. Matrix on 26.0.0: 38,898 pass, 1,643 documented, 0 unclassified (winsorize/float64 gained: it was the offset bug, not finding 22). Three more pinned behaviours changed and their tests say so (cumulative_max default start, out-of-range float-to-int cast now wraps as the GPU does, the fixed-size-list Parquet file pyarrow's reader used to reject); the registry carries 26.0.0's five new names: remainder on the GPU as modulo() has always computed it, remainder_checked, floored modulo/modulo_checked and search_sorted recorded as missing; the matrix's modulo checks against pc.remainder. UPSTREAM rows read fixed in 26.0.0; EVALUATION, COVERAGE, ARROW_FUNCTIONS, differential_coverage.json regenerated; FINDINGS Round 22; JSON/CSV differences re-checked on 26.0.0
Copy file name to clipboardExpand all lines: docs/COVERAGE.md
+27-23Lines changed: 27 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,7 +12,7 @@ a claim can be checked in one jump.
12
12
13
13
This file groups Arrow's functions into families and explains each one. The companion page
14
14
[**ARROW_FUNCTIONS.md**](ARROW_FUNCTIONS.md) does the complementary thing: one row per **exact Arrow
15
-
function name**, all 307 of them, generated from a registry that the test suite executes name by name
15
+
function name**, all 312 of them (Arrow v26; 307 in v25), generated from a registry that the test suite executes name by name
16
16
against `pyarrow.compute`. Go there for "is `<name>` covered?"; stay here for "how does this family work?".
17
17
18
18
## Summary
@@ -46,7 +46,9 @@ aggregations section below is likewise outside the compute-function total.
46
46
47
47
A row here covers a family, so these are not function counts. The by-name numbers are in
48
48
[ARROW_FUNCTIONS.md](ARROW_FUNCTIONS.md): of Arrow v25's 307 compute function names, **all 307 are
49
-
reachable** — 283 entirely on the GPU, 17 on the host, 7 with a stated limitation, and none missing.
49
+
reachable** — 284 entirely on the GPU, 17 on the host, 7 with a stated limitation, and none missing. Of
50
+
the five names Arrow v26 added, `remainder` runs on the GPU (it is `modulo()`'s C `%`) and the floored
51
+
`modulo`, the two checked variants and `search_sorted` are recorded as missing.
50
52
51
53
**The scope ArrowMetal 0.4.0 answers to, name by name:** every Arrow compute function name — 7 of them
52
54
with a stated limitation — over `int8/16/32/64`, `uint8/16/32/64`, `float16/32/64`, `bool`, `utf8`,
@@ -201,9 +203,10 @@ exactly what the atomic path could not: Float64 sums and means (through the soft
201
203
202
204
## Element-wise arithmetic
203
205
204
-
`modulo` (`%`, C remainder semantics, `x % 0` defined as 0) is an ArrowMetal extension rather than an
205
-
Arrow function name, so it has no row of its own; it lives beside `power` in `Kernels/Rounding.swift`
206
-
and is reachable as `am_binary(op 5)` and `.modulo()` / `__mod__` in Python.
206
+
`modulo` (`%`, C remainder semantics, `x % 0` defined as 0) lives beside `power` in `Kernels/Rounding.swift`
207
+
and is reachable as `am_binary(op 5)` and `.modulo()` / `__mod__` in Python. Since Arrow 26.0.0 it has a
208
+
function name of its own, `remainder` (the C `%`); Arrow's floored `modulo`, the checked pair and
209
+
`search_sorted`, new in 26.0.0, are not implemented and carry `missing` rows in the registry.
207
210
208
211
| Arrow function | Status | Notes |
209
212
|---|---|---|
@@ -448,11 +451,11 @@ grouped aggregates, covered above. The row below is an ArrowMetal extension.
448
451
449
452
<!-- BEGIN differential cases per function: generated by python/tests/coverage_report.py -->
450
453
451
-
The differential matrix (`python/tests/test_differential.py`, [TESTING.md](TESTING.md#3-the-differential-matrix)) runs 217 operations over 46 column types: 1,544 (operation, type) cells, each over 27 datasets, 41,688 cases. The run recorded in `docs/data/differential_coverage.json` (ArrowMetal 0.5.0, pyarrow 25.0.1, Apple M4 Max, router `gpu`): **38,897 pass, 1,644 documented divergences, 1,147 skipped, 0 unclassified**.
454
+
The differential matrix (`python/tests/test_differential.py`, [TESTING.md](TESTING.md#3-the-differential-matrix)) runs 217 operations over 46 column types: 1,544 (operation, type) cells, each over 27 datasets, 41,688 cases. The run recorded in `docs/data/differential_coverage.json` (ArrowMetal 0.5.0, pyarrow 26.0.0, Apple M4 Max, router `gpu`): **38,898 pass, 1,643 documented divergences, 1,147 skipped, 0 unclassified**.
452
455
453
456
A case counts toward an Arrow function when the ArrowMetal call it runs is that function's implementation (the call column of [ARROW_FUNCTIONS.md](ARROW_FUNCTIONS.md)) or, for an ArrowMetal call with no Arrow name of its own (`sort`, `argsort`, `to_strings`), when its answer is compared with that function's result. One case can check several functions — a `compare_scalar` case runs all six comparisons — so the rows below add up to more than the case total. A documented divergence is a failing case that matches an open finding in [EVALUATION.md](EVALUATION.md); a skipped case is an operation that does not implement that column type.
454
457
455
-
280 of the 307 Arrow function names have matrix cases. Every name, with or without them, is also run through `call_function` against `pyarrow.compute` in `python/tests/test_functions.py`.
458
+
281 of the 312 Arrow function names have matrix cases. Every name, with or without them, is also run through `call_function` against `pyarrow.compute` in `python/tests/test_functions.py`.
|`percent_rank_and_cume_dist`| window functions, against a reference written in the harness | 270 | 270 | 0 | 0 |
@@ -879,7 +882,7 @@ included.
879
882
880
883
## What ArrowMetal 0.4.0 claims, and what it does not
881
884
882
-
**The claim.** ArrowMetal 0.4.0 answers to **all 307 of the Apache Arrow v25 compute function names** —
885
+
**The claim.** ArrowMetal 0.5.0 answers to **all 307 of the Apache Arrow v25 compute function names** (and to `remainder` of the five v26 added; the other four are recorded as missing) —
883
886
the 283 in the C++ docs plus the 24 `hash_*` grouped aggregates — over `int8/16/32/64`, `uint8/16/32/64`,
0 commit comments