You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit c0aeca8
Browse filesBrowse the repository at this point in the historyBrowse files
fix: apply Hadoop's default SharedKey when an account key is set without an auth type
`AbfsConfiguration.getAuthType` falls back to `SharedKey` when no
`fs.azure.account.auth.type` is set, and `initializeClient` then reads only the
account key. The native scan read every mechanism's keys in that case, so a
valid account key beside an unused OAuth client secret was rejected for the
missing client id and tenant.
An account key entry without an auth type now selects `SharedKey`: the OAuth
and SAS keys beside it are neither translated nor validated, a blank key is
still reported, `fs.azure.account.keyprovider` is still rejected, and the
environment contributes nothing. With neither an auth type nor a key, every
mechanism is read as before, so OAuth-only and environment-only configurations
keep working.
| SAS token |`fs.azure.sas.<container>.<account>`, `fs.azure.sas.fixed.token`|
376
376
377
-
This authority extends only to the mechanisms Comet recognises. An explicit `fs.azure.account.auth.type` is Hadoop choosing a mechanism, so it is validated against the translated keys rather than ignored. It also decides which keys are read, as it does for Hadoop: only the selected mechanism's keys are translated and validated, so an OAuth client secret left in the configuration under an account-scoped `SharedKey`, or an account key under `OAuth`, is left alone, neither rejected nor passed to the store. Hadoop ABFS mechanisms that have no native counterpart are neither translated nor allowed to fall through to the environment or to managed identity. In each of these cases no `AZURE_*` variable is consulted, and when the translated keys do not satisfy the mechanism the scan fails with an error naming the key (and, for a class name, the value) instead:
377
+
This authority extends only to the mechanisms Comet recognises. An explicit `fs.azure.account.auth.type` is Hadoop choosing a mechanism, so it is validated against the translated keys rather than ignored. It also decides which keys are read, as it does for Hadoop: only the selected mechanism's keys are translated and validated, so an OAuth client secret left in the configuration under an account-scoped `SharedKey`, or an account key under `OAuth`, is left alone, neither rejected nor passed to the store. With no auth type at all, Hadoop defaults to `SharedKey` whenever `fs.azure.account.key` (or its account-scoped form) is present, and the native scan follows suit: only the account key is read, a blank key is still an error, and OAuth or SAS keys beside it are skipped in the same way. With neither an auth type nor an account key, every mechanism's keys are read, as described above. Hadoop ABFS mechanisms that have no native counterpart are neither translated nor allowed to fall through to the environment or to managed identity. In each of these cases no `AZURE_*` variable is consulted, and when the translated keys do not satisfy the mechanism the scan fails with an error naming the key (and, for a class name, the value) instead:
Hadoop keys beyond these are not translated. `fs.azure.account.oauth.provider.type` is validated whenever it is set and OAuth is the mechanism read, that is with `fs.azure.account.auth.type=OAuth` or with no auth type at all: a class ending in `MsiTokenProvider` stands alone, `WorkloadIdentityTokenProvider` needs the client id and tenant, `ClientCredsTokenProvider` needs the client secret principal, any other class is an error naming the key and the class, and a blank value is an error naming the key. A class matches only by its exact simple name, bare or as the last segment of a qualified name. It, `fs.azure.account.auth.type`, and the unsupported mechanism keys listed under [Supported Authentication](#supported-authentication) are read only to decide whether Hadoop configures an auth mechanism, and whether it is one the native scan can build. The storage account and container always come from the URL.
411
+
Hadoop keys beyond these are not translated. `fs.azure.account.oauth.provider.type` is validated whenever it is set and OAuth is the mechanism read, that is with `fs.azure.account.auth.type=OAuth`, or with no auth type and no `fs.azure.account.key` (a key with no auth type selects `SharedKey`, Hadoop's default, and the class is then ignored): a class ending in `MsiTokenProvider` stands alone, `WorkloadIdentityTokenProvider` needs the client id and tenant, `ClientCredsTokenProvider` needs the client secret principal, any other class is an error naming the key and the class, and a blank value is an error naming the key. A class matches only by its exact simple name, bare or as the last segment of a qualified name. It, `fs.azure.account.auth.type`, and the unsupported mechanism keys listed under [Supported Authentication](#supported-authentication) are read only to decide whether Hadoop configures an auth mechanism, and whether it is one the native scan can build. The storage account and container always come from the URL.
0 commit comments