Skip to content

Move cloud compatibility gating into a shared helper beside the object store configuration code #5658

Open

Description

@dwsmith1983

What is the problem the feature request solves?

Follow-up from the native Delta Lake scan review (#5365). The Delta contrib carries its own cloud compatibility gating (S3 option divergence checks) next to Delta admission logic. Since native Parquet and native Delta share the same underlying object store client, that policy could live in a focused helper beside the shared object store configuration code so compatibility fixes apply consistently, and Delta admission can concentrate on protocol features, schemas, and Delta metadata.

The helper must preserve Hadoop's distinct rules for plain options, password-backed values, variable substitution, and bucket overrides, without introducing another provider abstraction.

Describe the potential solution

No response

Additional context

Raised in the #5365 review discussion.

Activity

  1. dwsmith1983 commented on Oct 9, 2026

    @dwsmith1983
    ContributorAuthor

    take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:scanParquet scan / data readingenhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions