Skip to content

[Variant] [Shredding] Support typed_access for Utf8 and BinaryView #8364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
alamb merged 5 commits into from
Sep 17, 2025
Merged

[Variant] [Shredding] Support typed_access for Utf8 and BinaryView #8364

alamb merged 5 commits into from
Sep 17, 2025

Conversation

@petern48
Copy link
Contributor

@petern48 petern48 commented Sep 16, 2025

Which issue does this PR close?

We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax.

Rationale for this change

See Issue

What changes are included in this PR?

Support typed_access for Utf8 and BinaryView

Are these changes tested?

Yes

Are there any user-facing changes?

N/A. Variant support still in development

@github-actions github-actions bot added the parquet-variant parquet-variant* crates label Sep 16, 2025
@petern48 petern48 marked this pull request as ready for review September 16, 2025 15:58
klion26
klion26 approved these changes Sep 17, 2025
View reviewed changes
Copy link
Member

@klion26 klion26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the contribution.

@alamb
Copy link
Contributor

alamb commented Sep 17, 2025

I just ran the new integration test and it fails (as expected) with this PR as we have enabled some new features.

cargo test --all-features variant_integration

Fails like this:

failures:

---- test_variant_integration_case_31 stdout ----
VariantTestCase {
    case_number: 31,
    test: Some(
        "testShreddedVariantPrimitives",
    ),
    parquet_file: Some(
        "case-031.parquet",
    ),
    variant_file: Some(
        "case-031_row-0.variant.bin",
    ),
    variant_files: None,
    error_message: None,
    variant_description: None,
}
note: test did not panic as expected at parquet/tests/variant_integration.rs:58:16
---- test_variant_integration_case_30 stdout ----
VariantTestCase {
    case_number: 30,
    test: Some(
        "testShreddedVariantPrimitives",
    ),
    parquet_file: Some(
        "case-030.parquet",
    ),
    variant_file: Some(
        "case-030_row-0.variant.bin",
    ),
    variant_files: None,
    error_message: None,
    variant_description: None,
}
note: test did not panic as expected at parquet/tests/variant_integration.rs:58:16

failures:
    test_variant_integration_case_30
    test_variant_integration_case_31

I just pushed a commit to fix the CI to run the tests, and will push another one to fix the tests

@github-actions github-actions bot added the parquet Changes to the parquet crate label Sep 17, 2025
@alamb alamb merged commit b06996b into apache:main Sep 17, 2025
31 checks passed
@alamb
Copy link
Contributor

alamb commented Sep 17, 2025

Thank you @petern48 and @klion26 🙏

@petern48 petern48 deleted the utf8_binary_view_variant branch September 17, 2025 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@klion26 klion26 klion26 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

parquet Changes to the parquet crate parquet-variant parquet-variant* crates

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Variant] [Shredding] Support typed_access for Utf8 and BinaryView

3 participants

@petern48 @alamb @klion26