Repository navigation
fix(java): release borrowed batch schemas in address-based callers - #84
Open
ChristopherWirt wants to merge 2 commits into
Open
ChristopherWirt wants to merge 2 commits into
ChristopherWirt wants to merge 2 commits into
Conversation
Signed-off-by: Christopher Wirt <[email protected]>
Signed-off-by: Christopher Wirt <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Address-based
writeBatchcallers export an Arrow schema for each batch but close its outer struct without releasing the export. Native import takes the array and only borrows the schema; Arrow 19'sArrowSchema.close()does not invoke its release callback. This leaves the exported schema's child allocations alive.Closes #53.
Upstream comparison
The issue remains in both repositories, reviewed on 2026-11-04:
9f21db7954c7: native import moves the array withFFI_ArrowArray::from_rawand borrowsFFI_ArrowSchema.bc58587293a3: the address-based native import is identical. ItstestWriteBatchand other fixtures also omit schema release.The managed
writeBatch(VectorSchemaRoot, BufferAllocator)overload exports no per-batch schema and already cleans up its temporary array. Current Spark production writes use that overload. #18's writer-creation cleanup does not cover these per-batch exports.Changes
finallyblocks at eleven leaking call sites across JNI and Spark fixtures. Also release arrays that were not consumed if export/import fails, before closing their outer structs.Validation
Completed independent reuse, maintainability, efficiency and correctness reviews. No substantive defects were found; applied the repository formatter to the changed Java files and reviewed the final diff.
-Werrorpassed.JNIWriterTest,NativeIOBridgeTest,ListContainsFilterTest,SpatialTypesTest,TestMinimal,IndependentBuffersTest).VortexNestedSchemaResolutionTestandVortexReadTypeParityTest.git diff --checkpassed.testWriteBatchmade the address-based case fail with 294 bytes still allocated; the managed-root case passed. Restoring the release made both cases pass again.Tests used a cached JNI build with identical writer/runtime sources and Cargo dependency files; its only Rust source delta is PR #82's unrelated file-listing/deletion utilities. No Rust code changed in this PR. Spark 3.5 variants and the full repository suite were not run.