Skip to content

Commit f21fdf4

Browse files
holdenksfc-gh-hkaraucloud-fan
committed
[SPARK-59118][4.X][SQL] Warn when doAs is enabled but not enforced
### What changes were proposed in this pull request? This is a softer follow on of #58414 for 4.X branch which warns when doAs is enabled but not enforced. We should log rather than fail a previously running job. ### Why are the changes needed? People might expect doAs to run as a different user, which is not supported. ### Does this PR introduce _any_ user-facing change? The Spark Thrift Server logs a startup warning when `hive.server2.enable.doAs` is enabled with an authenticating auth type, unless the new static conf `spark.sql.hive.thriftServer.allowIneffectiveDoAs` is set to `true`. ### How was this patch tested? New unit test. ### Was this patch authored or co-authored using generative AI tooling? Yes, claude Closes #58438 from holdenk/SPARK-59118-doAs-warn-4.x-r2. Lead-authored-by: Holden Karau <[email protected]> Co-authored-by: Holden Karau <[email protected]> Co-authored-by: Wenchen Fan <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent f3c676c commit f21fdf4

5 files changed

Lines changed: 193 additions & 0 deletions

File tree

‎docs/sql-migration-guide.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ license: |
5757
- Since Spark 4.4, with whole-stage codegen enabled, a filter condition, including the `FILTER` clause of an aggregate function, evaluates the remaining top-level conjuncts of the condition on a row after one of them evaluates to `NULL`, as the SQL standard requires, and does so in the order they appear in the condition, as Spark already did with whole-stage codegen disabled, which also evaluates a subexpression shared by several conjuncts before the condition. Only a `FALSE` conjunct stops the evaluation. One exception: an `IS NOT NULL` check on a column, or on an expression over it, that a later conjunct references is still evaluated before that conjunct and the conjuncts up to the check's own position, even when it is written after them, and can drop the row before an error one of them raises. Once a condition has a conjunct that can be `NULL`, the `IS NOT NULL` checks after that conjunct are evaluated in the order of the condition on every row, not only on rows where a conjunct is `NULL`. A row whose condition is `NULL` is still dropped, but an error raised by a later conjunct on that row, such as `ARITHMETIC_OVERFLOW` in ANSI mode, now fails the query. For example, `SELECT a FROM t WHERE NOT (a > 0 AND b > 0) AND y + 1 > 0` can fail with `ARITHMETIC_OVERFLOW` on a row where `a` is `NULL`, `b` is `1`, and `y` is `2147483647`, unless the row is dropped earlier by a `FALSE` conjunct or `IS NOT NULL` check that precedes `y + 1 > 0` in the condition, or skipped before the filter, for example by data source filter pushdown such as Parquet row group statistics. Previously the query skipped such a row with whole-stage codegen enabled. The later conjuncts also run on more rows than before, so a nondeterministic conjunct such as `rand(seed)` can return different values on the remaining rows, and a side-effecting user-defined function is called on rows that are dropped. To restore the previous behavior, set `spark.sql.legacy.filterShortCircuitOnNullConjunct` to `true`.
5858

5959
- Since Spark 4.4, when a format is specified, `to_timestamp`, `try_to_timestamp`, and `to_timestamp_ltz` cast a numeric first argument to a string and parse it with the format, for example `to_timestamp(20201231, 'yyyyMMdd')` returns `2020-12-31 00:00:00`. This matches the existing behavior when `spark.sql.timestampType` is `TIMESTAMP_NTZ`. Previously, with the default `TIMESTAMP_LTZ` timestamp type, such calls failed with `INTERNAL_ERROR`. Note that the string form of the number is parsed, so a DECIMAL `1.5` is parsed as `"1.5"`.
60+
- Since Spark 4.4, the Spark Thrift Server logs a warning at startup when `hive.server2.enable.doAs` is `true` and `hive.server2.authentication` verifies user identities (any recognized type other than `NONE`/`NOSASL`), because executor-side data access still runs as the server's own service identity rather than the connecting user (SPARK-5159). Set `spark.sql.hive.thriftServer.allowIneffectiveDoAs` to `true` to acknowledge the limitation and silence the warning. Spark 5.0 is expected to refuse to start on this configuration unless that option is set.
6061

6162
## Upgrading from Spark SQL 4.2 to 4.3
6263

‎sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,23 @@ object StaticSQLConf {
147147
.booleanConf
148148
.createWithDefault(true)
149149

150+
val HIVE_THRIFT_SERVER_ALLOW_INEFFECTIVE_DOAS =
151+
buildStaticConf("spark.sql.hive.thriftServer.allowIneffectiveDoAs")
152+
.doc("With hive.server2.enable.doAs=true the Spark Thrift Server impersonates the " +
153+
"connecting user on the driver, but executor-side data access still runs as the " +
154+
"server's own service identity (SPARK-5159), so storage-level permissions are " +
155+
"checked against the privileged service principal instead of the impersonated " +
156+
"user. Because that can silently grant users access to data they could not read " +
157+
"with their own credentials, the server warns at startup when " +
158+
"hive.server2.enable.doAs=true and hive.server2.authentication verifies user " +
159+
"identities (any recognized type other than NONE/NOSASL), unless this option is " +
160+
"set to true to acknowledge the limitation. Spark 5.0 is expected to refuse to " +
161+
"start on that configuration unless this option is set.")
162+
.version("4.4.0")
163+
.withBindingPolicy(ConfigBindingPolicy.NOT_APPLICABLE)
164+
.booleanConf
165+
.createWithDefault(false)
166+
150167
val SPARK_SESSION_EXTENSIONS = buildStaticConf("spark.sql.extensions")
151168
.doc("A comma-separated list of classes that implement " +
152169
"Function1[SparkSessionExtensions, Unit] used to configure Spark Session extensions. The " +

‎sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,12 +23,15 @@ import java.util.concurrent.atomic.AtomicBoolean
2323
import org.apache.hadoop.hive.common.ServerUtils
2424
import org.apache.hadoop.hive.conf.HiveConf
2525
import org.apache.hadoop.hive.conf.HiveConf.ConfVars
26+
import org.apache.hive.service.auth.HiveAuthFactory.AuthTypes
2627
import org.apache.hive.service.cli.thrift.{ThriftBinaryCLIService, ThriftHttpCLIService}
2728
import org.apache.hive.service.server.HiveServer2
2829

2930
import org.apache.spark.SparkContext
3031
import org.apache.spark.annotation.{DeveloperApi, Since}
32+
import org.apache.spark.internal.LogEntry
3133
import org.apache.spark.internal.Logging
34+
import org.apache.spark.internal.LogKeys.{CONFIG, CONFIG2}
3235
import org.apache.spark.internal.config.UI.UI_ENABLED
3336
import org.apache.spark.sql.{SparkSession, SQLContext}
3437
import org.apache.spark.sql.hive.HiveUtils
@@ -105,6 +108,48 @@ object HiveThriftServer2 extends Logging {
105108
}
106109
}
107110

111+
// Sessions are impersonated on the driver (metastore calls, driver-side file system access),
112+
// but executor-side data access runs as the service identity (SPARK-5159), so storage ACLs
113+
// are checked against the wrong principal. NONE/NOSASL are exempt: they establish no verified
114+
// identity (the client picks any username it likes, and doAs then impersonates that
115+
// unverified name for metastore calls), so there is no security boundary to warn about --
116+
// and Hive's out-of-the-box config (auth NONE, doAs true) must stay quiet. Unrecognized auth
117+
// types are left alone too: HiveAuthFactory rejects them with its own "Unsupported
118+
// authentication type" error, which points at the actual problem.
119+
private[thriftserver] def ineffectiveDoAsWarning(
120+
hiveConf: HiveConf,
121+
allowIneffectiveDoAs: Boolean): Option[LogEntry] = {
122+
val authType = hiveConf.getVar(ConfVars.HIVE_SERVER2_AUTHENTICATION)
123+
val verifyingAuthTypes =
124+
AuthTypes.values().filterNot(Set(AuthTypes.NONE, AuthTypes.NOSASL)).map(_.getAuthName)
125+
// getVar returns the default (NONE) when unset and "" when set empty, never null.
126+
val authVerifiesUser = verifyingAuthTypes.exists(_.equalsIgnoreCase(authType))
127+
if (authVerifiesUser && hiveConf.getBoolVar(ConfVars.HIVE_SERVER2_ENABLE_DOAS) &&
128+
!allowIneffectiveDoAs) {
129+
Some(log"${MDC(CONFIG, ConfVars.HIVE_SERVER2_ENABLE_DOAS.varname)} is set to true, " +
130+
log"but the Spark Thrift Server impersonates the connecting user only on the driver, " +
131+
log"for Hive metastore calls and driver-side file system access: executor-side data " +
132+
log"access still runs as the server's own service identity, so storage permissions " +
133+
log"are checked against the service principal rather than the connecting user " +
134+
log"(SPARK-5159). That can expose data the connecting user is not authorized to " +
135+
log"read. Setting it to false silences this, but note that it stops impersonating " +
136+
log"metastore calls too, so it is not a no-op (and unsetting it does not help -- " +
137+
log"Hive's own default is true). To acknowledge the limitation and silence this " +
138+
log"instead, set " +
139+
log"${MDC(CONFIG2, StaticSQLConf.HIVE_THRIFT_SERVER_ALLOW_INEFFECTIVE_DOAS.key)} to " +
140+
log"true; it is a static conf, so it must be set before the SparkSession is created. " +
141+
log"Spark 5.0 is expected to refuse to start on this configuration unless it is set.")
142+
} else {
143+
None
144+
}
145+
}
146+
147+
private[thriftserver] def warnIfIneffectiveDoAs(
148+
hiveConf: HiveConf,
149+
allowIneffectiveDoAs: Boolean): Unit = {
150+
ineffectiveDoAsWarning(hiveConf, allowIneffectiveDoAs).foreach(logWarning(_))
151+
}
152+
108153
def main(args: Array[String]): Unit = {
109154
// If the arguments contains "-h" or "--help", print out the usage and exit.
110155
if (args.contains("-h") || args.contains("--help")) {
@@ -154,6 +199,8 @@ private[hive] class HiveThriftServer2(sparkSession: SparkSession)
154199
private val started = new AtomicBoolean(false)
155200

156201
override def init(hiveConf: HiveConf): Unit = {
202+
HiveThriftServer2.warnIfIneffectiveDoAs(
203+
hiveConf, sparkSession.conf.get(StaticSQLConf.HIVE_THRIFT_SERVER_ALLOW_INEFFECTIVE_DOAS))
157204
val sparkSqlCliService = new SparkSQLCLIService(this, sparkSession)
158205
setSuperField(this, "cliService", sparkSqlCliService)
159206
addService(sparkSqlCliService)
Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.spark.sql.hive.thriftserver
19+
20+
import org.apache.hadoop.hive.conf.HiveConf
21+
import org.apache.hadoop.hive.conf.HiveConf.ConfVars
22+
import org.apache.hive.service.auth.HiveAuthFactory.AuthTypes
23+
24+
import org.apache.spark.SparkFunSuite
25+
import org.apache.spark.sql.internal.StaticSQLConf
26+
27+
/**
28+
* Tests for the SPARK-59118 startup warning: `hive.server2.enable.doAs` is accepted but
29+
* impersonation does not reach executor-side data access. Spark 5.0 is expected to refuse
30+
* to start instead, on the same configuration and with the same opt-out conf.
31+
*/
32+
class HiveThriftServer2DoAsSuite extends SparkFunSuite {
33+
34+
private def hiveConf(authType: String, doAs: Boolean): HiveConf = {
35+
val conf = new HiveConf()
36+
conf.setVar(ConfVars.HIVE_SERVER2_AUTHENTICATION, authType)
37+
conf.setBoolVar(ConfVars.HIVE_SERVER2_ENABLE_DOAS, doAs)
38+
conf
39+
}
40+
41+
/**
42+
* The warning text (if any) that checking `conf` would log. These tests assert on the
43+
* returned entry rather than captured log output, which is hostage to JVM-global logging
44+
* state when the whole module shares one forked JVM (SERIAL_SBT_TESTS=1 in CI).
45+
*/
46+
private def doAsWarnings(conf: HiveConf, allowIneffectiveDoAs: Boolean = false): Seq[String] = {
47+
HiveThriftServer2.ineffectiveDoAsWarning(conf, allowIneffectiveDoAs).map(_.message).toSeq
48+
}
49+
50+
// The auth types that establish a user identity worth impersonating.
51+
private val verifyingAuthTypes =
52+
AuthTypes.values().filterNot(Set(AuthTypes.NONE, AuthTypes.NOSASL)).map(_.getAuthName)
53+
54+
test("SPARK-5159 / SPARK-59118 warn when doAs is enabled but not enforced") {
55+
val warnings = doAsWarnings(hiveConf("KERBEROS", doAs = true))
56+
assert(warnings.length == 1)
57+
assert(warnings.head.contains("SPARK-5159"))
58+
// Auth type matching is case-insensitive.
59+
assert(doAsWarnings(hiveConf("kerberos", doAs = true)).length == 1)
60+
}
61+
62+
test("SPARK-59118 allowIneffectiveDoAs=true silences an otherwise-warned config") {
63+
assert(doAsWarnings(hiveConf("KERBEROS", doAs = true), allowIneffectiveDoAs = true).isEmpty)
64+
}
65+
66+
test("SPARK-59118 the warning names the conf that silences it") {
67+
val warning = doAsWarnings(hiveConf("KERBEROS", doAs = true)).head
68+
assert(warning.contains(StaticSQLConf.HIVE_THRIFT_SERVER_ALLOW_INEFFECTIVE_DOAS.key))
69+
}
70+
71+
test("SPARK-59118 warn for every auth type that verifies the user") {
72+
// Guards against a new AuthTypes value silently landing on the quiet side.
73+
assert(verifyingAuthTypes.nonEmpty)
74+
verifyingAuthTypes.foreach { authType =>
75+
assert(doAsWarnings(hiveConf(authType, doAs = true)).length == 1, s"no warning: $authType")
76+
}
77+
}
78+
79+
test("SPARK-59118 does not warn when auth type establishes no user identity") {
80+
Seq("NONE", "NOSASL", "none", "noSasl").foreach { authType =>
81+
assert(doAsWarnings(hiveConf(authType, doAs = true)).isEmpty, s"warned: $authType")
82+
}
83+
}
84+
85+
test("SPARK-59118 does not warn when doAs is disabled") {
86+
verifyingAuthTypes.foreach { authType =>
87+
assert(doAsWarnings(hiveConf(authType, doAs = false)).isEmpty, s"warned: $authType")
88+
}
89+
}
90+
91+
test("SPARK-59118 does not warn on an untouched Hive config") {
92+
// Hive defaults doAs to true and auth to NONE, so a stock config must stay quiet.
93+
assert(doAsWarnings(new HiveConf()).isEmpty)
94+
}
95+
96+
test("SPARK-59118 an unrecognized or empty auth type is left to Hive's own error") {
97+
// HiveAuthFactory rejects these with "Unsupported authentication type", where the
98+
// warning's advice would not help. getVar returns "" (never null) for an explicitly
99+
// empty value, so this must not NPE.
100+
Seq("NOT_AN_AUTH_TYPE", "").foreach { authType =>
101+
assert(doAsWarnings(hiveConf(authType, doAs = true)).isEmpty, s"warned: $authType")
102+
}
103+
}
104+
}

‎sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/SparkExecuteStatementOperationSuite.scala‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,9 +23,11 @@ import java.util.concurrent.Semaphore
2323
import scala.concurrent.duration._
2424

2525
import org.apache.hadoop.hive.conf.HiveConf
26+
import org.apache.hadoop.hive.conf.HiveConf.ConfVars
2627
import org.apache.hive.service.cli.OperationState
2728
import org.apache.hive.service.cli.session.{HiveSession, HiveSessionImpl}
2829
import org.apache.hive.service.rpc.thrift.{TProtocolVersion, TTypeId}
30+
import org.apache.logging.log4j.Level
2931
import org.mockito.Mockito.{doReturn, mock, spy, when, RETURNS_DEEP_STUBS}
3032
import org.mockito.invocation.InvocationOnMock
3133

@@ -135,6 +137,28 @@ class SparkExecuteStatementOperationSuite extends SharedSparkSession {
135137
}
136138
}
137139

140+
test("SPARK-59118 HiveThriftServer2.init warns when doAs is not enforced") {
141+
// The warning is only worth anything if init() actually calls it.
142+
val hiveConf = new HiveConf()
143+
hiveConf.setVar(ConfVars.HIVE_SERVER2_AUTHENTICATION, "KERBEROS")
144+
hiveConf.setBoolVar(ConfVars.HIVE_SERVER2_ENABLE_DOAS, true)
145+
// init() boots Hive's service graph; operation logging attaches an appender to the root
146+
// logger that nothing ever removes, so keep it off rather than leak it into this JVM.
147+
hiveConf.setBoolVar(ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED, false)
148+
// Attach to the root logger, not the HiveThriftServer2 logger by name: withLogAppender on
149+
// a named logger leaves behind a non-additive LoggerConfig (log4j2 copies root's additivity)
150+
// that swallows that logger's output for every later suite in this JVM.
151+
val appender = new LogAppender("doAs impersonation warning")
152+
withLogAppender(appender, level = Some(Level.WARN)) {
153+
new HiveThriftServer2(spark).init(hiveConf)
154+
}
155+
val warnings = appender.loggingEvents.map(_.getMessage.getFormattedMessage)
156+
assert(warnings.exists { warning =>
157+
warning.contains(ConfVars.HIVE_SERVER2_ENABLE_DOAS.varname) &&
158+
warning.contains("SPARK-5159")
159+
})
160+
}
161+
138162
private class MySparkExecuteStatementOperation(
139163
session: SparkSession,
140164
hiveSession: HiveSession,

0 commit comments

Comments
 (0)