You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Support anonymous S3 access through the CometS3CredentialProvider SPI #6298
The SPI returns CometS3Credentials with non-null access/secret keys, so it cannot express
"anonymous". When the built-in Hadoop adapter is enabled globally and a bucket's chain resolves to
AnonymousAWSCredentialsProvider (common for public datasets), the adapter fails with a clear error
and users must opt the bucket out with an empty fs.s3a.bucket..comet.credential.provider.class.
Supporting anonymous through the SPI (a sentinel credential or an isAnonymous flag the native reader
maps to skip-signature) would let the adapter match Spark for these buckets. See #6023.
Activity