Add ListView support to `arrow-row` and `arrow-ord` by brancz · Pull Request #9176 · apache/arrow-rs

brancz · 2026-01-14T14:31:17Z

Which issue does this PR close?

Closes #9174

What changes are included in this PR?

Implementation and tests. It's mostly copied from List.

Are these changes tested?

Yes, see unit tests.

Are there any user-facing changes?

No, purely additive.

@alamb @Jefffrey

friendlymatthew

Hi, do you mind rebasing? I think CI is failing because this needs to be rebased on the latest master- the base branch is missing the encoded_len fn that was added recently

Otherwise, the implementation makes sense to me

brancz · 2026-01-15T09:22:13Z

Very confused because both this PR and #9175 are based off of latest main, and the tip of the other PR seems to work fine.

brancz · 2026-01-16T09:29:40Z

Figured it out, I did actually need to change something after the rebase here, also refactored the use in both list-types.

arrow-row/src/list.rs

arrow-row/src/lib.rs

rluvaton · 2026-01-19T09:37:13Z

arrow-row/src/list.rs

+        _ => unreachable!(),
+    };
+
+    let null_buffer = NullBuffer::new(BooleanBuffer::new(nulls.into(), 0, rows.len()));


Maybe use new_unchecked as you already have the null count and we want to avoid calculating it twice

Suggested change

let null_buffer = NullBuffer::new(BooleanBuffer::new(nulls.into(), 0, rows.len()));

let null_buffer = NullBuffer::new_unchecked(BooleanBuffer::new(nulls.into(), 0, rows.len()), null_count);

good catch, done!

rluvaton · 2026-01-19T09:39:28Z

arrow-row/src/lib.rs

+
+        if size > 0 {
+            min_offset = min_offset.min(offset);
+            max_end = max_end.max(end);


Maybe you can break if you reached maximum bounds (0 and maximum value that can be)

done (although I wonder how often this is better in practice while adding an additional branch for a case that might happen incredibly rarely)

rluvaton · 2026-01-19T09:40:29Z

arrow-row/src/lib.rs

+    #[test]
+    fn test_list_view() {
+        test_single_list_view::<i32>();
+    }
+
+    #[test]
+    fn test_large_list_view() {
+        test_single_list_view::<i64>();
+    }


Please add nested tests like the regular list

rluvaton · 2026-01-19T09:43:54Z

arrow-row/src/lib.rs

        test_nested_list::<i64>();
    }

+    fn test_single_list_view<O: OffsetSizeTrait>() {


Please add more tests that take advantage of the fact that this is a view, namely

both list point to the same value.

unordered offsets (one item is from offset x and some item after that is from offset y and y is before x)

list 1 items cover list 2 items and a little more (e.g. list 1 offset is 10 and size 5 and list 2 offset is 12 and size 2).

done (added all cases in one test let me know if you prefer separate tests)

rluvaton · 2026-01-19T09:45:07Z

arrow-row/src/lib.rs

Please add the list view and large list view to here as well similar to how list and large list are here.
don't forget to increase the random range so it will cover the new values

this was a great idea, it actually caught something

brancz · 2026-01-20T13:53:30Z

@rluvaton thoughts on the assertion vs. returning an error? I don't feel too strongly one way or another, but I do think we should be consistent for both List and ListView.

alamb

Could we also please update the documentation here with how ListView works:

https://docs.rs/arrow-row/57.2.0/arrow_row/struct.RowConverter.html#list-encoding

It would help to review it more carefully

brancz · 2026-01-27T17:03:59Z

Added! (it's not different from Lists, but one wouldn't know that without already knowing the implementation, so I agree this was important!)

rluvaton · 2026-01-28T12:23:57Z

@rluvaton thoughts on the assertion vs. returning an error? I don't feel too strongly one way or another, but I do think we should be consistent for both List and ListView.

consistent

alamb

Thanks @brancz and @rluvaton and @Jefffrey -- other than the uncovered code in arrow-ord I think this PR is ready to merge.

I also reviewed this code and the test coverage, and I think it is quite good

I reviewed the code coverage here

cargo llvm-cov --html test -p arrow-row

alamb · 2026-01-28T22:44:22Z

arrow-row/src/lib.rs

+///                          └──── 2_u8 ────┘   └──── 3_u8 ────┘
+///
+///```
+///


arrow-row/src/list.rs

alamb · 2026-01-28T23:01:39Z

arrow-ord/src/ord.rs

    Ok(f)
 }

+fn compare_list_view<O: OffsetSizeTrait>(


While useful, I didn't see any test coverage for this code

I think we should either break it into its own PR, or add tests. Here is a PR to do so:

Add arrow-row tests brancz/arrow-rs#1

(I am also happy to review a separate PR)

Merged, thank you!

I can't remember now anymore why it was, but I needed to implement compare support for something I needed for tests or fuzzing. Sorry, I've been working on so much lately that I don't remember anymore what exactly it was but it was needed to address other review comments.

Sorry, I've been working on so much lately that I don't remember anymore what exactly it was

I TOTALLY understand

Add arrow-row tests

alamb

Thank you @brancz @rluvaton and @friendlymatthew

alamb · 2026-01-30T14:46:10Z

I merged up to resolve a conflict.

alamb · 2026-01-30T15:08:37Z

Thanks again everyone

github-actions bot added the arrow Changes to the arrow crate label Jan 14, 2026

brancz mentioned this pull request Jan 14, 2026

Support aggregating by ListView apache/datafusion#19782

Open

friendlymatthew approved these changes Jan 14, 2026

View reviewed changes

brancz force-pushed the row-list-view branch 2 times, most recently from cd2a465 to 29f13ef Compare January 15, 2026 09:13

arrow-row: Add ListView support

ba24e9d

brancz force-pushed the row-list-view branch from 29f13ef to ba24e9d Compare January 15, 2026 12:42

arrow-row: Refactor to deduplicate code for encoded list length

a9564b7

Jefffrey reviewed Jan 16, 2026

View reviewed changes

arrow-row/src/list.rs Show resolved Hide resolved

arrow-row/src/list.rs Show resolved Hide resolved

arrow-row/src/lib.rs Outdated Show resolved Hide resolved

arrow-row: Move bounds check function further down in the file

d36eaa9

rluvaton reviewed Jan 19, 2026

View reviewed changes

brancz added 5 commits January 20, 2026 11:28

arrow-row: Use unchecked new since the null count is already retrieved

80c24de

arrow-row: Return list bounds early if full range is already found

1441826

arrow-ord: Add ability to compare ListView

5bf5fc5

arrow-row: Add various cases with offset and size orders for ListView

b2d8491

arrow-row: Add ListView to fuzz tests and fix failure

57a8d3f

arrow-row: Add tests with nested ListView

3f9da4d

brancz force-pushed the row-list-view branch from 2bc5cda to 3f9da4d Compare January 20, 2026 13:54

alamb reviewed Jan 25, 2026

View reviewed changes

arrow-row: Add ListView row format documentation

1cdba6d

alamb changed the title ~~arrow-row: Add ListView support~~ Add ListView support to arrow-row and arrow-ord Jan 28, 2026

alamb changed the title ~~Add ListView support to arrow-row and arrow-ord~~ Add ListView support to arrow-row Jan 28, 2026

Add arrow-row tests

e53ec69

alamb mentioned this pull request Jan 28, 2026

Add arrow-row tests brancz/arrow-rs#1

Merged

alamb reviewed Jan 28, 2026

View reviewed changes

Merge pull request #1 from alamb/alamb/list_view_ord_tests

c9b3dc0

Add arrow-row tests

alamb approved these changes Jan 29, 2026

View reviewed changes

alamb changed the title ~~Add ListView support to arrow-row~~ Add ListView support to arrow-row and arrow-ord Jan 29, 2026

Jefffrey approved these changes Jan 30, 2026

View reviewed changes

alamb added 2 commits January 30, 2026 09:39

Merge remote-tracking branch 'apache/main' into row-list-view

5f083ae

Fix merge

85935de

Fix range

840a0ce

alamb merged commit 05cb6b0 into apache:main Jan 30, 2026
17 checks passed

brancz deleted the row-list-view branch January 30, 2026 15:23

alamb mentioned this pull request Jan 30, 2026

fix: fix [[NULL]] array doesn't roundtrip in arrow-row bug #9275

Merged

	let null_buffer = NullBuffer::new(BooleanBuffer::new(nulls.into(), 0, rows.len()));
	let null_buffer = NullBuffer::new_unchecked(BooleanBuffer::new(nulls.into(), 0, rows.len()), null_count);

Comments

Conversation

brancz commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

friendlymatthew left a comment

Choose a reason for hiding this comment

Uh oh!

brancz commented Jan 15, 2026

Uh oh!

brancz commented Jan 16, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brancz Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brancz commented Jan 20, 2026

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

brancz commented Jan 27, 2026

Uh oh!

rluvaton commented Jan 28, 2026

Uh oh!

alamb left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

alamb commented Jan 30, 2026

Uh oh!

Uh oh!

alamb commented Jan 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

brancz commented Jan 14, 2026 •

edited

Loading

brancz Jan 20, 2026 •

edited

Loading

alamb left a comment •

edited

Loading