Skip to content

Commit be929ac

Browse files
morningmanclaude
andcommitted
[fix](flight-sql) Finish the previous request before checking a prepared handle
### What problem does this PR solve? Issue Number: None Related PR: #68631 Problem Summary: A prepared GetFlightInfo checks its handle before it executes anything: the handle must exist, the session must still be in the namespace it was prepared in, and the statement re-analyzed now must have the schema that Prepare advertised. `executePreparedStatement` ran these checks before `executeQueryStatement` reached `FlightProtocolAdapter.beginRequest()`, where a request drops what the previous one left: its deferred coordinator, the frontend-side result no DoGet took, and its endpoints. A handle that the checks refused therefore left all three in place. The previous query's deferred coordinator, which a batch-mode external-table scan or a remote Doris scan leaves, kept its workload group queue slot and its query registration until the session's next statement, the timeout checker or teardown, and the previous frontend-side result could still be fetched. Before #69867, every prepared GetFlightInfo reached `executeQueryStatement`, and a missing handle failed inside it, where the catch finalized the previous coordinator. `executePreparedStatement` now calls `beginRequest()` before the checks: a prepared execution is the session's next request whatever they find. Prepare and GetSchema are metadata requests and still leave the previous request alone, so the call is neither in the session's command lock nor in `preparedQuery()`, which GetSchema goes through too. A checked handle whose statement could no longer be described when it was re-analyzed answered UNIMPLEMENTED and was kept, so every later execution failed the same way. That happens after a column grant is revoked so that the analyzed plan is denied, or after a DDL that turns a result column into an untyped NULL. The check now compares the schema Prepare would advertise now, the placeholder for such a statement: the handle is removed and the request answers NOT_FOUND "Prepared statement schema changed; prepare again", as for any other schema change, and the statement prepared again keeps the placeholder and runs unchecked. Prepare and the check share `advertisedSchema` for this. ### Release note A Flight SQL prepared execution refused before it runs no longer keeps the previous query's deferred coordinator and result. A prepared statement whose result schema can no longer be determined before it runs answers NOT_FOUND and must be prepared again. ### Check List (For Author) - Test: Unit Test, Regression test - `DorisFlightSqlSchemaTest` adds one test for each way the checks refuse a handle: a missing handle, a namespace change, a sql_mode drift, a dropped view (INVALID_ARGUMENT, the handle is kept) and a revoked column grant (NOT_FOUND, the handle is removed, and preparing again gives the placeholder). Each sends GetSchema and GetFlightInfo after a previous request left a deferred executor, a frontend-side result and an endpoint, and checks that only GetFlightInfo finished it. - Without the new `beginRequest()` call the five tests fail at GetFlightInfo; with the call in `preparedQuery()` they fail at GetSchema; with the previous UNIMPLEMENTED only the revoked-grant test fails. - 45 FE test classes (555 tests) pass and fe-core Checkstyle is clean. - Local cluster: all 21 suites under `arrow_flight_sql_p0`, `test_uuid_arrow_flight` and `test_sort_operator` over Flight SQL pass. - Behavior changed: Yes. A prepared GetFlightInfo that its checks refuse finishes the previous request first. A checked handle that can no longer be described is removed and answers NOT_FOUND instead of UNIMPLEMENTED. - Does this need documentation: No Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 36772dc commit be929ac

2 files changed

Lines changed: 162 additions & 16 deletions

File tree

‎fe/fe-core/src/main/java/org/apache/doris/arrowflight/DorisFlightSqlProducer.java‎

Lines changed: 31 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -401,6 +401,10 @@ public FlightInfo getFlightInfoPreparedStatement(final CommandPreparedStatementQ
401401

402402
private FlightInfo executePreparedStatement(String peerIdentity, ConnectContext connectContext,
403403
String preparedStatementId, FlightDescriptor descriptor) {
404+
// A prepared execution is the session's next request even when its handle fails the checks below, so
405+
// what the previous request left is dropped before them, as executeQueryStatement does for a statement
406+
// (which then finds nothing left). Prepare and GetSchema are metadata requests and leave it.
407+
FlightProtocolAdapter.of(connectContext).beginRequest();
404408
PreparedQuery prepared = preparedQuery(connectContext, preparedStatementId);
405409
if (prepared.getSchema() == null) {
406410
// Prepare could not tell this statement's result schema and advertised the placeholder, so
@@ -489,24 +493,45 @@ private static PreparedQuery preparedQuery(ConnectContext connectContext, String
489493
}
490494

491495
// Execution parses the SQL again in the current session. Never silently replace the schema Prepare
492-
// advertised when a setting such as sql_mode or time_zone changes the result.
496+
// advertised when a setting such as sql_mode or time_zone changes the result. The check compares what
497+
// Prepare would advertise now, so a handle also expires when only running its statement could tell the
498+
// schema any more (a column grant revoked since Prepare, a DDL that turned a result column into an untyped
499+
// NULL); prepared again, the statement gets the placeholder and runs unchecked.
493500
private static Schema checkPreparedSchema(ConnectContext connectContext, String preparedStatementId,
494501
PreparedQuery prepared) {
495-
Schema schema = discoverSchema(connectContext, prepared.getSql());
496-
if (!schema.equals(prepared.getSchema())) {
502+
Schema schema = advertisedSchema(connectContext, prepared.getSql());
503+
if (!prepared.getSchema().equals(schema)) {
497504
connectContext.removePreparedQuery(preparedStatementId);
498505
throw CallStatus.NOT_FOUND.withDescription("Prepared statement schema changed; prepare again")
499506
.toRuntimeException();
500507
}
501508
return schema;
502509
}
503510

511+
// The schema Prepare advertises and records for query: null stands for the placeholder of a statement
512+
// whose result schema only running it can tell.
513+
private static Schema advertisedSchema(ConnectContext connectContext, String query) {
514+
try {
515+
return analyzeSchema(connectContext, query);
516+
} catch (UndiscoverableSchemaException e) {
517+
return null;
518+
}
519+
}
520+
521+
// The schema GetSchema answers for query.
504522
private static Schema discoverSchema(ConnectContext connectContext, String query) {
505523
try {
506-
return FlightSqlQuerySchema.analyze(connectContext, query);
524+
return analyzeSchema(connectContext, query);
507525
} catch (UndiscoverableSchemaException e) {
508526
throw CallStatus.UNIMPLEMENTED.withDescription(e.getMessage()).withCause(e).toRuntimeException();
509-
} catch (FlightRuntimeException e) {
527+
}
528+
}
529+
530+
private static Schema analyzeSchema(ConnectContext connectContext, String query)
531+
throws UndiscoverableSchemaException {
532+
try {
533+
return FlightSqlQuerySchema.analyze(connectContext, query);
534+
} catch (UndiscoverableSchemaException | FlightRuntimeException e) {
510535
throw e;
511536
} catch (Exception e) {
512537
throw CallStatus.INVALID_ARGUMENT.withDescription("Cannot determine query schema: " + e.getMessage())
@@ -564,17 +589,7 @@ private void createPreparedStatement(final ActionCreatePreparedStatementRequest
564589
// before registering a handle so that a statement that cannot run leaves none behind. A
565590
// statement whose schema only running it can tell keeps the placeholder: the Arrow Flight
566591
// SQL JDBC driver prepares every statement and does not fall back when Prepare refuses one.
567-
Schema schema;
568-
try {
569-
schema = FlightSqlQuerySchema.analyze(connectContext, query);
570-
} catch (UndiscoverableSchemaException e) {
571-
schema = null;
572-
} catch (FlightRuntimeException e) {
573-
throw e;
574-
} catch (Exception e) {
575-
throw CallStatus.INVALID_ARGUMENT.withDescription("Cannot determine query schema: " + e.getMessage())
576-
.withCause(e).toRuntimeException();
577-
}
592+
Schema schema = advertisedSchema(connectContext, query);
578593
final ByteString handle = ByteString.copyFromUtf8(peerIdentity + ":" + preparedStatementId);
579594
Result result = new Result(Any.pack(buildCreatePreparedStatementResult(handle,
580595
new Schema(Collections.emptyList()), schema == null ? PLACEHOLDER_DATASET_SCHEMA : schema))

‎fe/fe-core/src/test/java/org/apache/doris/arrowflight/DorisFlightSqlSchemaTest.java‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919

2020
import org.apache.doris.analysis.UserIdentity;
2121
import org.apache.doris.arrowflight.protocol.FlightProtocolAdapter;
22+
import org.apache.doris.arrowflight.results.FlightSqlEndpointsLocation;
2223
import org.apache.doris.arrowflight.sessions.FlightSessionsManager;
2324
import org.apache.doris.catalog.Column;
2425
import org.apache.doris.catalog.Env;
@@ -33,7 +34,9 @@
3334
import org.apache.doris.nereids.StatementContext;
3435
import org.apache.doris.nereids.exceptions.AnalysisException;
3536
import org.apache.doris.nereids.glue.LogicalPlanAdapter;
37+
import org.apache.doris.nereids.parser.NereidsParser;
3638
import org.apache.doris.nereids.parser.SqlDialectHelper;
39+
import org.apache.doris.nereids.trees.plans.commands.RevokeTablePrivilegeCommand;
3740
import org.apache.doris.nereids.util.PlanChecker;
3841
import org.apache.doris.qe.ConnectContext;
3942
import org.apache.doris.qe.SessionVariable;
@@ -42,6 +45,7 @@
4245
import org.apache.doris.qe.SqlModeHelper;
4346
import org.apache.doris.qe.StmtExecutor;
4447
import org.apache.doris.qe.VariableMgr;
48+
import org.apache.doris.thrift.TNetworkAddress;
4549
import org.apache.doris.thrift.TUniqueId;
4650
import org.apache.doris.utframe.TestWithFeService;
4751

@@ -924,4 +928,131 @@ void executionSchemaDriftExpiresHandleAndCleansResults() throws Exception {
924928
flightContext.setExecutor(previous);
925929
}
926930
}
931+
932+
// Sends GetSchema for the first handle and GetFlightInfo for the second, both refused by the checks a prepared
933+
// request runs before it executes, each after the previous request left a deferred coordinator, a result on
934+
// this frontend and an endpoint. GetSchema is a metadata request and leaves them; GetFlightInfo is the next
935+
// request of the session whatever its checks find, so it drops them first, as every GetFlightInfo does.
936+
private void assertRefusedBeforeExecution(List<ActionCreatePreparedStatementResult> handles,
937+
FlightStatusCode code, boolean handleKept) {
938+
for (boolean execute : Arrays.asList(false, true)) {
939+
ActionCreatePreparedStatementResult handle = handles.get(execute ? 1 : 0);
940+
String request = (execute ? "GetFlightInfo of " : "GetSchema of ") + preparedId(handle);
941+
StmtExecutor previous = Mockito.mock(StmtExecutor.class);
942+
flightContext.addFlightSqlDeferredExecutor(previous);
943+
flightContext.getFlightSqlChannel().addOKResult("previous", "SELECT 1");
944+
flightContext.addFlightSqlEndpointsLocation(new FlightSqlEndpointsLocation(new TUniqueId(48, 3),
945+
new TNetworkAddress("127.0.0.1", 8070), new TNetworkAddress("127.0.0.1", 8060),
946+
new ArrayList<>()));
947+
try {
948+
FlightRuntimeException error = Assertions.assertThrows(FlightRuntimeException.class, () -> {
949+
if (execute) {
950+
producer.getFlightInfoPreparedStatement(command(handle), callContext,
951+
FlightDescriptor.command(new byte[0]));
952+
} else {
953+
producer.getSchemaPreparedStatement(command(handle), callContext,
954+
FlightDescriptor.command(new byte[0]));
955+
}
956+
}, request);
957+
Assertions.assertEquals(code, error.status().code(), request);
958+
Assertions.assertEquals(handleKept, flightContext.getPreparedQuery(preparedId(handle)) != null,
959+
request);
960+
Assertions.assertEquals(execute ? Collections.emptyList() : Collections.singletonList(previous),
961+
flightContext.getFlightSqlDeferredExecutors(), request);
962+
Assertions.assertEquals(execute ? 0 : 1, flightContext.getFlightSqlChannel().resultNum(), request);
963+
Assertions.assertEquals(execute ? 0 : 1, flightContext.getFlightSqlEndpointsLocations().size(),
964+
request);
965+
Mockito.verify(previous, Mockito.times(execute ? 1 : 0)).finalizeArrowFlightQuery();
966+
} finally {
967+
FlightProtocolAdapter.of(flightContext).beginRequest();
968+
}
969+
}
970+
}
971+
972+
@Test
973+
void missingHandleFinishesThePreviousRequestOnExecution() {
974+
ActionCreatePreparedStatementResult missing = ActionCreatePreparedStatementResult.newBuilder()
975+
.setPreparedStatementHandle(ByteString.copyFromUtf8("schema-peer:never-prepared")).build();
976+
assertRefusedBeforeExecution(Arrays.asList(missing, missing), FlightStatusCode.NOT_FOUND, false);
977+
}
978+
979+
@Test
980+
void namespaceChangeFinishesThePreviousRequestOnExecution() throws Exception {
981+
List<ActionCreatePreparedStatementResult> handles = Arrays.asList(prepare("SELECT id FROM schema_input"),
982+
prepare("SELECT id FROM schema_input"));
983+
String database = flightContext.getDatabase();
984+
// Moved outside the session's requests, as SetSessionOptions moves it.
985+
flightContext.setDatabase("information_schema");
986+
try {
987+
assertRefusedBeforeExecution(handles, FlightStatusCode.NOT_FOUND, false);
988+
} finally {
989+
flightContext.setDatabase(database);
990+
handles.forEach(handle -> flightContext.removePreparedQuery(preparedId(handle)));
991+
}
992+
}
993+
994+
@Test
995+
void sqlModeDriftFinishesThePreviousRequestOnExecution() throws Exception {
996+
long mode = flightContext.getSessionVariable().getSqlMode();
997+
List<ActionCreatePreparedStatementResult> handles = new ArrayList<>();
998+
try {
999+
flightContext.getSessionVariable().setSqlMode(0);
1000+
handles.add(prepare("SELECT 1 || 2 AS x"));
1001+
handles.add(prepare("SELECT 1 || 2 AS x"));
1002+
// Changed outside the session's requests, as SetSessionOptions changes it.
1003+
flightContext.getSessionVariable().setSqlMode(SqlModeHelper.MODE_PIPES_AS_CONCAT);
1004+
assertRefusedBeforeExecution(handles, FlightStatusCode.NOT_FOUND, false);
1005+
} finally {
1006+
flightContext.getSessionVariable().setSqlMode(mode);
1007+
handles.forEach(handle -> flightContext.removePreparedQuery(preparedId(handle)));
1008+
}
1009+
}
1010+
1011+
@Test
1012+
void failedReanalysisFinishesThePreviousRequestOnExecution() throws Exception {
1013+
createView("CREATE VIEW schema_dropped_view AS SELECT id FROM schema_input");
1014+
List<ActionCreatePreparedStatementResult> handles = Arrays.asList(
1015+
prepare("SELECT id FROM schema_dropped_view"), prepare("SELECT id FROM schema_dropped_view"));
1016+
try {
1017+
// Another session drops the view: the statement fails as its execution would, and the handle stays.
1018+
dropView("DROP VIEW schema_dropped_view");
1019+
assertRefusedBeforeExecution(handles, FlightStatusCode.INVALID_ARGUMENT, true);
1020+
} finally {
1021+
handles.forEach(handle -> flightContext.removePreparedQuery(preparedId(handle)));
1022+
}
1023+
}
1024+
1025+
@Test
1026+
void checkedHandleExpiresOnceOnlyExecutionCanTellItsSchema() throws Exception {
1027+
addUser("schema_revoked_reader", true);
1028+
grantPriv("GRANT SELECT_PRIV ON schema_test.schema_input TO 'schema_revoked_reader'@'%'");
1029+
UserIdentity originalUser = flightContext.getCurrentUserIdentity();
1030+
flightContext.setCurrentUserIdentity(UserIdentity.createAnalyzedUserIdentWithIp("schema_revoked_reader", "%"));
1031+
// The analyzed plan sorts on name, the executed plan does not (columnGrantsAreCheckedAsExecutionChecksThem).
1032+
String sorted = "SELECT id FROM (SELECT id, name FROM schema_input ORDER BY name) d";
1033+
List<ActionCreatePreparedStatementResult> handles = new ArrayList<>();
1034+
try {
1035+
handles.add(prepare(sorted));
1036+
handles.add(prepare(sorted));
1037+
for (ActionCreatePreparedStatementResult handle : handles) {
1038+
Assertions.assertNotNull(flightContext.getPreparedQuery(preparedId(handle)).getSchema());
1039+
}
1040+
// Another session narrows the grant to the column the executed plan reads. The analyzed plan is denied
1041+
// now, so only execution can decide, and nothing is left to check the advertised schema against.
1042+
((RevokeTablePrivilegeCommand) new NereidsParser().parseSingle(
1043+
"REVOKE SELECT_PRIV ON schema_test.schema_input FROM 'schema_revoked_reader'@'%'"))
1044+
.run(connectContext, null);
1045+
grantPriv("GRANT SELECT_PRIV(id) ON schema_test.schema_input TO 'schema_revoked_reader'@'%'");
1046+
assertRefusedBeforeExecution(handles, FlightStatusCode.NOT_FOUND, false);
1047+
// Prepared again, the statement keeps the placeholder and its execution decides.
1048+
ActionCreatePreparedStatementResult again = prepare(sorted);
1049+
handles.add(again);
1050+
Assertions.assertEquals(Collections.singletonList("ResultMeta"), datasetSchema(again).getFields()
1051+
.stream().map(Field::getName).collect(Collectors.toList()));
1052+
Assertions.assertNull(flightContext.getPreparedQuery(preparedId(again)).getSchema());
1053+
} finally {
1054+
flightContext.setCurrentUserIdentity(originalUser);
1055+
handles.forEach(handle -> flightContext.removePreparedQuery(preparedId(handle)));
1056+
}
1057+
}
9271058
}

0 commit comments

Comments
 (0)