Skip to content

Commit c2a7adb

Browse files
committed
fix: keep a URL password out of Hadoop messages in the ABFS fallback reason
Strip the password from the user info wherever a Failed message quotes it, cover the supported OAuth and SAS mechanisms staying native, and add end-to-end fallback tests for Parquet and CSV scans.
1 parent c33edf3 commit c2a7adb

4 files changed

Lines changed: 112 additions & 5 deletions

File tree

‎docs/source/user-guide/latest/datasources.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -479,7 +479,7 @@ hadoop-azure must be on the driver classpath. It already is whenever Spark can l
479479
| `WorkloadIdentityTokenProvider` with a `fs.azure.account.oauth2.client.assertion.provider.type` (hadoop-azure 3.5.0 and later) | Not supported | |
480480
| `UserboundSASWithOAuth` (hadoop-azure 3.5.0 and later) | Not supported | |
481481

482-
Custom provider classes hand each token request to Java code the native scan cannot call, and the refresh token and user password flows have no `object_store` counterpart. When Hadoop selects one of these, the scan falls back to Spark's own scan, which authenticates through hadoop-azure, and the fallback reason names the auth type and, where one is configured, the class. Configure a supported mechanism for that account to read it natively. Values arrive as Hadoop resolved them, trimmed and defaulted where the ABFS driver trims and defaults them, so the store sees what the driver would have used.
482+
Custom provider classes hand each token request to Java code the native scan cannot call, and the refresh token and user password flows have no `object_store` counterpart. When Hadoop selects one of these, the scan falls back to Spark's own scan, which authenticates through hadoop-azure, and the fallback reason names the auth type and, for SAS, OAuth and Workload Identity, the provider class (`Custom` names only the auth type). Configure a supported mechanism for that account to read it natively. Values arrive as Hadoop resolved them, trimmed and defaulted where the ABFS driver trims and defaults them, so the store sees what the driver would have used.
483483

484484
Client credentials and Workload Identity token requests go over HTTPS only. The managed identity request goes to the IMDS endpoint over HTTP, as Hadoop's does. For client credentials, Hadoop posts to `fs.azure.account.oauth2.client.endpoint` as given, while `object_store` posts to `<authority host>/<tenant>/oauth2/v2.0/token`. Comet takes the tenant from the path segment before the last `oauth2` and everything before that segment as the authority host, so a v1 `/oauth2/token` endpoint and a proxy with a path prefix both resolve. The token request path is always the v2.0 one. An `http://` endpoint fails.
485485

‎spark/src/main/scala/org/apache/comet/objectstore/AbfsAuthResolver.scala‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -480,9 +480,10 @@ private[comet] object AbfsAuthResolver extends Logging {
480480
// The container and host only: a password in the user info stays out of the reason.
481481
val authority = Option(uri.getRawAuthority).getOrElse("")
482482
val at = authority.lastIndexOf('@')
483-
val location = s"${uri.getScheme}://" +
484-
(if (at < 0) authority
485-
else authority.substring(0, at).takeWhile(_ != ':') + authority.substring(at))
483+
val safeAuthority =
484+
if (at < 0) authority
485+
else authority.substring(0, at).takeWhile(_ != ':') + authority.substring(at)
486+
val location = s"${uri.getScheme}://$safeAuthority"
486487
def unsupported(mechanism: String): String =
487488
s"Native scan does not support ABFS authentication $mechanism for $location"
488489
outcome match {
@@ -496,7 +497,10 @@ private[comet] object AbfsAuthResolver extends Logging {
496497
_) =>
497498
Some(unsupported(s"auth type ${AuthTypes.OAUTH} with provider $c"))
498499
case Failed(_, message) =>
499-
Some(s"Hadoop ABFS authentication failed on the driver for $location: $message")
500+
// Some of Hadoop's own messages quote the whole URI, user info included.
501+
val safeMessage =
502+
if (authority.isEmpty) message else message.replace(authority, safeAuthority)
503+
Some(s"Hadoop ABFS authentication failed on the driver for $location: $safeMessage")
500504
case _: Resolved | Unconfigured => None
501505
}
502506
}

‎spark/src/test/scala/org/apache/comet/objectstore/AbfsScanFallbackSuite.scala‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,41 @@ class AbfsScanFallbackSuite extends CometTestBase {
112112
}
113113
}
114114

115+
private def abfsKeys(pairs: (String, String)*): Seq[(String, String)] =
116+
pairs.map { case (key, value) => s"$key.accta.dfs.core.windows.net" -> value }
117+
118+
test("native Parquet scan with a Custom ABFS auth type falls back to Spark") {
119+
writeParquet("parquet-custom", 0)
120+
withSQLConf(abfsKeys("fs.azure.account.auth.type" -> "Custom"): _*) {
121+
val df = spark.read.parquet(abfss("data", "accta", "parquet-custom"))
122+
checkSparkAnswerAndFallbackReason(df, "auth type Custom")
123+
}
124+
}
125+
126+
test("native Parquet scan with an unknown ABFS auth type falls back to Spark") {
127+
writeParquet("parquet-unknown-auth", 0)
128+
withSQLConf(abfsKeys("fs.azure.account.auth.type" -> "NotAnAuthType"): _*) {
129+
val df = spark.read.parquet(abfss("data", "accta", "parquet-unknown-auth"))
130+
checkSparkAnswerAndFallbackReason(df, "failed on the driver")
131+
}
132+
}
133+
134+
test("native CSV scan with a Custom ABFS auth type falls back to Spark") {
135+
writeCsv("csv-custom", 0)
136+
withSQLConf(csvConfs ++ abfsKeys("fs.azure.account.auth.type" -> "Custom"): _*) {
137+
val df = readCsv(abfss("data", "accta", "csv-custom"))
138+
checkSparkAnswerAndFallbackReason(df, "auth type Custom")
139+
}
140+
}
141+
142+
test("native CSV scan with an unknown ABFS auth type falls back to Spark") {
143+
writeCsv("csv-unknown-auth", 0)
144+
withSQLConf(csvConfs ++ abfsKeys("fs.azure.account.auth.type" -> "NotAnAuthType"): _*) {
145+
val df = readCsv(abfss("data", "accta", "csv-unknown-auth"))
146+
checkSparkAnswerAndFallbackReason(df, "failed on the driver")
147+
}
148+
}
149+
115150
test("native Parquet scan over two directories of one container stays native") {
116151
writeParquet("parquet-e", 0)
117152
writeParquet("parquet-f", 100)

‎spark/src/test/scala/org/apache/comet/objectstore/NativeConfigSuite.scala‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -793,6 +793,37 @@ class NativeConfigSuite extends AnyFunSuite with Matchers {
793793
}
794794
}
795795

796+
test("extractScanObjectStoreOptions - a URL password stays out of a Hadoop message") {
797+
// Hadoop reads `data:url-password` as the container and rejects the short account name with
798+
// a message that quotes the whole URI.
799+
val hadoopConf = new Configuration()
800+
hadoopConf.set("fs.azure.account.key.accta", "a2V5LWE=")
801+
val uri = new URI("abfss://data:url-password@accta/path/a.parquet")
802+
NativeConfig.extractScanObjectStoreOptions(hadoopConf, Seq(uri)) match {
803+
case Left(reason) =>
804+
reason should include("account name is not fully qualified")
805+
reason should include("abfss://data@accta")
806+
reason should not include "url-password"
807+
case Right(options) => fail(s"expected a fallback, got $options")
808+
}
809+
}
810+
811+
test("nativeFallbackReason - names the classes of a declined Workload Identity and SAS") {
812+
val uri = abfsFile("data", "accta", "a.parquet")
813+
val assertion = AbfsAuthResolver.Declined(
814+
"OAuth",
815+
Some(Providers.WORKLOAD_IDENTITY),
816+
Some("com.example.Assertion"))
817+
val reason = AbfsAuthResolver.nativeFallbackReason(uri, assertion).get
818+
reason should include("auth type OAuth")
819+
reason should include(s"with provider ${Providers.WORKLOAD_IDENTITY}")
820+
reason should include("and client assertion provider com.example.Assertion")
821+
822+
val userbound = AbfsAuthResolver.Declined("UserboundSASWithOAuth", None)
823+
AbfsAuthResolver.nativeFallbackReason(uri, userbound).get should include(
824+
"auth type UserboundSASWithOAuth")
825+
}
826+
796827
test("extractScanObjectStoreOptions - a custom SAS token provider falls back to Spark") {
797828
val providerClass = classOf[NeverInstantiatedSasTokenProvider].getName
798829
assertFallsBack(
@@ -846,6 +877,43 @@ class NativeConfigSuite extends AnyFunSuite with Matchers {
846877
options(Markers.RESOLUTION) shouldBe Markers.RESOLVED
847878
}
848879

880+
test("extractScanObjectStoreOptions - every supported OAuth and SAS mechanism stays native") {
881+
val handles = AbfsReflection.handles(getClass.getClassLoader).get
882+
val clientEndpoint = "https://login.microsoftonline.com/tenant/oauth2/token"
883+
val supported = Seq(
884+
"ClientCredsTokenProvider" -> Seq(
885+
Keys.AUTH_TYPE -> "OAuth",
886+
Keys.OAUTH_PROVIDER_TYPE -> Providers.CLIENT_CREDS,
887+
Keys.CLIENT_ENDPOINT -> clientEndpoint,
888+
Keys.CLIENT_ID -> "client-123",
889+
Keys.CLIENT_SECRET -> "client-secret-value"),
890+
"MsiTokenProvider" -> Seq(
891+
Keys.AUTH_TYPE -> "OAuth",
892+
Keys.OAUTH_PROVIDER_TYPE -> Providers.MSI,
893+
Keys.MSI_TENANT -> "tenant-abc",
894+
Keys.CLIENT_ID -> "client-123"))
895+
val needsNewerHadoop = Seq(
896+
"WorkloadIdentityTokenProvider" -> Seq(
897+
Keys.AUTH_TYPE -> "OAuth",
898+
Keys.OAUTH_PROVIDER_TYPE -> Providers.WORKLOAD_IDENTITY,
899+
Keys.MSI_TENANT -> "tenant-abc",
900+
Keys.CLIENT_ID -> "client-123"),
901+
"fixed SAS token" -> Seq(
902+
Keys.AUTH_TYPE -> "SAS",
903+
Keys.SAS_FIXED_TOKEN -> "sv=2020-08-04&sig=fixed-signature"))
904+
val cases = supported ++ (if (handles.hasFixedSasTokenProvider) needsNewerHadoop else Nil)
905+
val uris = Seq(abfsFile("data", "accta", "a.parquet"))
906+
cases.foreach { case (name, pairs) =>
907+
withClue(s"$name: ") {
908+
val hadoopConf = abfsScanConf(pairs: _*)
909+
NativeConfig.extractScanObjectStoreOptions(hadoopConf, uris) match {
910+
case Right(options) => options(Markers.RESOLUTION) shouldBe Markers.RESOLVED
911+
case Left(reason) => fail(s"expected native options, got a fallback: $reason")
912+
}
913+
}
914+
}
915+
}
916+
849917
test("extractScanObjectStoreOptions - non-abfs scans forward every scheme's options") {
850918
val hadoopConf = new Configuration()
851919
hadoopConf.set("fs.s3a.access.key", "s3-access-key")

0 commit comments

Comments
 (0)