Skip to content

ArrowArray::try_from_raw should not assume the pointers are from Arc #1333

@viirya

Description

@viirya

Describe the bug

ArrowArray::try_from_raw now uses Arc::from_raw to construct the two pointers to the C Data Interface (ArrowArray and ArrowSchema) back to the structs.

But the two pointers could be from other language runtime (e.g. JVM). We've been using the ffi API to pass Arrow Arrays from JVM to Rust. Sometimes we see fatal error reported at JVM (SIGILL). After digging it deeper, we found that sometimes Arc::from_raw on the pointers gives a Arc pointer with strong reference count reaching the maximum limit.

As the pointer is from JVM, it is not produced by Arc::into_raw, the offset calculation in Arc::from_raw simply doesn't work and can get into wrong values for the strong value in ArcInner.

I think that ArrowArray::try_from_raw should not assume the pointers are from Arc. They should be considered as just raw pointers to the C Data Interface.

To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    arrowChanges to the arrow cratebug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions