Skip to content

fix(vane): track Windows distributed mutation artifacts - #56

Merged
kaka11chen merged 1 commit into
main_vanefrom
codex/fix-windows-distributed-mutations-20261108
Nov 8, 2026
Merged

kaka11chen merged 1 commit into
main_vanefrom
codex/fix-windows-distributed-mutations-20261108

Conversation

@kaka11chen

Copy link
Copy Markdown

Windows drive paths such as C:\... and D:/... were parsed as non-file URI schemes in mutation workers. They consequently reopened the dataset through Lance's direct local writer, bypassing the object-store wrapper that tracks mutation files. UPDATE replacements and partial-DELETE files could be written and then rejected by transaction artifact validation.

Use upstream uri_to_url() and OS-aware to_file_path() before selecting file-object-store. Resolve the C++ error-redaction policy before the mutation FFI call so path normalization cannot erase its Rust error. Keep artifact validation and diagnostic redaction in place.

Regression coverage now includes local paths/file URIs with spaces, Unicode and literal percent escapes, plus the existing remote URI behavior. The existing Windows/macOS qualification script now exercises UPDATE, whole-fragment and partial DELETE, and full-table mutations in both local and Ray modes. The failed-UPDATE test requires the underlying nullability error rather than accepting only the generic transaction prefix.

Validation

  • cargo test --locked --offline --lib --features vane-distributed ffi::distributed_mutation::tests: 5 passed; compiled the changed Rust source.
  • C++ syntax compilation of src/lance_distributed_write.cpp with the released Vane 0.3.0 SDK headers and LANCE_VANE_DISTRIBUTED: passed.
  • python -m pytest -q python/tests/test_vane_platform_release.py: 23 passed.
  • uv run make format, cargo fmt --all --check, Black, Python syntax checks and git diff --check: passed.
  • Updated qualification script: local and Ray passed on Linux using the existing signed 0.3.0/0.3.0.1 wheels. This checks the expanded script, not a newly linked provider. The Ray run used one node/two CPUs; inspection of 38 Ray log files found no mutation error or Python fatal.

Windows verification of the rebuilt provider and the end-to-end error-detail regression remain pending. No TestPyPI/PyPI publication is part of this PR.

Windows verification

Use the matched Windows base/provider wheels built for this PR (the vane-lance-full-windows CI artifact), in a fresh CPython 3.12 x64 environment. The already published 0.3.0.1 wheel and the old test kit's pinned install step still select the old native code.

From this PR checkout, after installing those CI wheels:

$env:VANE_EXPECTED_EXTENSION_TRUST_IDENTITY = "vane-ci-test-key"
$env:VANE_DISTRIBUTED_NODE_COUNT = "1"
$env:VANE_DISTRIBUTED_WORKER_SLOTS = "2"
python scripts/verify_vane_platform.py --runner local --directory C:/vane-pr-test/local
python scripts/verify_vane_platform.py --runner ray --directory C:/vane-pr-test/ray

Use fresh output directories. The script verifies signed loading, exact mutated rows and the empty result after full DELETE. Inspect actor stderr as well as command exit status.

The separate GIL abort is tracked in AstroVela/vane#1146, with the related upstream duckdb-python#456 and #462. That crash's exact native caller still needs a Windows stack; this mutation fix does not claim to fix it.

@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-08T16:01:42.328494Z
We couldn’t complete this review. Comment @codex review to try again.
efe4181 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@kaka11chen
kaka11chen merged commit e8ffc1a into main_vane Nov 8, 2026
43 checks passed
kaka11chen added a commit that referenced this pull request Nov 9, 2026
Stage the next Lance provider release on TestPyPI so the Windows
distributed mutation fix from #56 can be qualified with a
production-signed provider.

ane-extension-lance==0.3.0.1 cannot be rebuilt: TestPyPI's Warehouse
ile_registry permanently records every uploaded filename, and
ane_provider_release.py requires elease_number to increase beyond
indexed releases for the same Vane X.Y.Z.

This bump stages ane-extension-lance==0.3.0.2 against the unchanged
ane-ai==0.3.0 runtime:

- 
elease_number: 1 -> 2
- provider version:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant