Repository navigation
Commit e8ffc1a
authored
fix(vane): track Windows distributed mutation artifacts (#56)
Windows drive paths such as `C:\...` and `D:/...` were parsed as
non-file URI schemes in mutation workers. They consequently reopened the
dataset through Lance's direct local writer, bypassing the object-store
wrapper that tracks mutation files. UPDATE replacements and
partial-DELETE files could be written and then rejected by transaction
artifact validation.
Use upstream `uri_to_url()` and OS-aware `to_file_path()` before
selecting `file-object-store`. Resolve the C++ error-redaction policy
before the mutation FFI call so path normalization cannot erase its Rust
error. Keep artifact validation and diagnostic redaction in place.
Regression coverage now includes local paths/file URIs with spaces,
Unicode and literal percent escapes, plus the existing remote URI
behavior. The existing Windows/macOS qualification script now exercises
UPDATE, whole-fragment and partial DELETE, and full-table mutations in
both local and Ray modes. The failed-UPDATE test requires the underlying
nullability error rather than accepting only the generic transaction
prefix.
### Validation
- `cargo test --locked --offline --lib --features vane-distributed
ffi::distributed_mutation::tests`: **5 passed**; compiled the changed
Rust source.
- C++ syntax compilation of `src/lance_distributed_write.cpp` with the
released Vane 0.3.0 SDK headers and `LANCE_VANE_DISTRIBUTED`:
**passed**.
- `python -m pytest -q python/tests/test_vane_platform_release.py`: **23
passed**.
- `uv run make format`, `cargo fmt --all --check`, Black, Python syntax
checks and `git diff --check`: **passed**.
- Updated qualification script: **local and Ray passed on Linux using
the existing signed 0.3.0/0.3.0.1 wheels**. This checks the expanded
script, not a newly linked provider. The Ray run used one node/two CPUs;
inspection of 38 Ray log files found no mutation error or Python fatal.
Windows verification of the rebuilt provider and the end-to-end
error-detail regression remain pending. No TestPyPI/PyPI publication is
part of this PR.
### Windows verification
Use the matched Windows base/provider wheels built for this PR (the
`vane-lance-full-windows` CI artifact), in a fresh CPython 3.12 x64
environment. The already published `0.3.0.1` wheel and the old test
kit's pinned `install` step still select the old native code.
From this PR checkout, after installing those CI wheels:
```powershell
$env:VANE_EXPECTED_EXTENSION_TRUST_IDENTITY = "vane-ci-test-key"
$env:VANE_DISTRIBUTED_NODE_COUNT = "1"
$env:VANE_DISTRIBUTED_WORKER_SLOTS = "2"
python scripts/verify_vane_platform.py --runner local --directory C:/vane-pr-test/local
python scripts/verify_vane_platform.py --runner ray --directory C:/vane-pr-test/ray
```
Use fresh output directories. The script verifies signed loading, exact
mutated rows and the empty result after full DELETE. Inspect actor
stderr as well as command exit status.
The separate GIL abort is tracked in
[AstroVela/vane#1146](AstroVela/vane#1146),
with the related upstream
[duckdb-python#456](duckdb/duckdb-python#456)
and [#462](duckdb/duckdb-python#462). That
crash's exact native caller still needs a Windows stack; this mutation
fix does not claim to fix it.1 parent b3efe20 commit e8ffc1a
4 files changed
Lines changed: 98 additions & 22 deletions
File tree
- python/tests
- rust/ffi
- scripts
- src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
821 | 821 | | |
822 | 822 | | |
823 | 823 | | |
824 | | - | |
| 824 | + | |
| 825 | + | |
825 | 826 | | |
826 | 827 | | |
827 | 828 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| |||
700 | 700 | | |
701 | 701 | | |
702 | 702 | | |
703 | | - | |
704 | | - | |
705 | | - | |
706 | | - | |
707 | | - | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
708 | 706 | | |
709 | 707 | | |
710 | | - | |
711 | | - | |
712 | | - | |
713 | | - | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
714 | 711 | | |
715 | 712 | | |
716 | 713 | | |
| |||
2297 | 2294 | | |
2298 | 2295 | | |
2299 | 2296 | | |
| 2297 | + | |
| 2298 | + | |
| 2299 | + | |
| 2300 | + | |
| 2301 | + | |
| 2302 | + | |
| 2303 | + | |
| 2304 | + | |
| 2305 | + | |
| 2306 | + | |
| 2307 | + | |
| 2308 | + | |
| 2309 | + | |
| 2310 | + | |
| 2311 | + | |
| 2312 | + | |
| 2313 | + | |
| 2314 | + | |
| 2315 | + | |
| 2316 | + | |
| 2317 | + | |
| 2318 | + | |
| 2319 | + | |
| 2320 | + | |
| 2321 | + | |
| 2322 | + | |
| 2323 | + | |
| 2324 | + | |
| 2325 | + | |
| 2326 | + | |
| 2327 | + | |
| 2328 | + | |
| 2329 | + | |
| 2330 | + | |
2300 | 2331 | | |
2301 | 2332 | | |
2302 | 2333 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | | - | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
48 | 50 | | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
55 | 58 | | |
56 | 59 | | |
57 | 60 | | |
58 | 61 | | |
59 | 62 | | |
60 | 63 | | |
61 | 64 | | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
62 | 104 | | |
63 | 105 | | |
64 | 106 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2693 | 2693 | | |
2694 | 2694 | | |
2695 | 2695 | | |
| 2696 | + | |
| 2697 | + | |
| 2698 | + | |
| 2699 | + | |
2696 | 2700 | | |
2697 | 2701 | | |
2698 | 2702 | | |
| |||
2716 | 2720 | | |
2717 | 2721 | | |
2718 | 2722 | | |
2719 | | - | |
2720 | | - | |
2721 | | - | |
2722 | | - | |
| 2723 | + | |
| 2724 | + | |
2723 | 2725 | | |
2724 | 2726 | | |
2725 | 2727 | | |
| |||
0 commit comments