Skip to content

Union Layout Should Not Support Separate Validity Mask #1590

@tustvold

Description

@tustvold

Describe the bug

Unlike other data types, unions do not have their own validity bitmap. Instead, the nullness of each slot is determined exclusively by the child arrays which are composed to create the union.

From https://arrow.apache.org/docs/format/Columnar.html#union-layout

To Reproduce

We currently support this, which aside from adding a lot of complexity, is incorrect

Expected behavior

We shouldn't support something not in the arrow specification

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions