Skip to content

Commit 099d5bd

Browse files
authored
[fix](arrow-flight) Do not take the point-query short circuit on an Arrow Flight connection (apache#67487)
### What problem does this PR solve? Issue Number: close apache#67368 Related PR: apache#67381 (sibling fix in the same tracking series), apache#62259 Problem Summary: A `UNIQUE KEY` point query that qualifies for the short-circuit path returned no Flight endpoint over Arrow Flight SQL. The client failed with ``` fetch arrow flight schema failed, no FlightSqlEndpointsLocations ``` and the row was silently dropped. The identical query with `SET_VAR(enable_short_circuit_query=false)` returned it on the same connection. **Root cause** — the short circuit produces no Arrow result at either end, and nothing prevented an Arrow Flight connection from planning one: * It executes on `PointQueryExecutor`, not a `Coordinator`, and `Coordinator`/`NereidsCoordinator` are the only places that register a `FlightSqlEndpointsLocation`. `StmtExecutor.executeAndSendResult` then returns early through its Arrow Flight branch with nothing registered and without ever calling `getNext()`, so `GetFlightInfo` found an empty endpoint list. * The BE cannot be pointed at either. `tablet_fetch_data` serializes with `VMysqlResultWriter` into `PTabletKeyLookupResponse.row_batch` and runs no fragment, so the `ArrowFlightResultBlockBuffer` that `fetch_arrow_flight_schema` looks up by finst id never exists. `LogicalResultSinkToShortCircuitPointQuery` did not look at the connect type, and `enable_short_circuit_query` defaults to `true`, so every ADBC / Arrow Flight JDBC point query on a MoW + light-schema-change + `store_row_column` table hit this. Prepared statements go through the same `executeQueryStatement` and failed identically. **Fix** — keep Arrow Flight SQL on the normal execution path. This has to be decided at plan time rather than when picking the executor: `OlapScanNode.computeTabletInfo` and several rewrite/property rules (`ChildOutputPropertyDeriver`, `ShuffleKeyPruner`, `NestedColumnPruning`, `PruneOlapScanPartition`) read `StatementContext.isShortCircuitQuery()` while the plan is being built, so flipping the flag later would run a coordinator over a plan shaped for a different execution mode. MySQL connections keep the short circuit unchanged. Returning the point-query result from the FE instead was considered and rejected for now: `FlightSqlChannel.addResult` builds varchar vectors only, so every column would come back as `Utf8`, inconsistent with the normal Flight path. Full support (Arrow serialization in the BE lookup RPC plus a result buffer to hand out an endpoint) is a larger change and out of scope here. Also refreshes a now-stale comment in `StmtExecutor` that said point queries reach the Arrow Flight deferral gate.
1 parent 219c619 commit 099d5bd

4 files changed

Lines changed: 182 additions & 3 deletions

File tree

‎fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/LogicalResultSinkToShortCircuitPointQuery.java‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@
3131
import org.apache.doris.nereids.trees.plans.logical.LogicalFilter;
3232
import org.apache.doris.nereids.trees.plans.logical.LogicalOlapScan;
3333
import org.apache.doris.qe.ConnectContext;
34+
import org.apache.doris.qe.ConnectContext.ConnectType;
3435

3536
import com.google.common.annotations.VisibleForTesting;
3637
import com.google.common.collect.ImmutableList;
@@ -64,7 +65,20 @@ private boolean filterMatchShortCircuitCondition(LogicalFilter<LogicalOlapScan>
6465

6566
@VisibleForTesting
6667
boolean scanMatchShortCircuitCondition(LogicalOlapScan olapScan) {
67-
if (!ConnectContext.get().getSessionVariable().isEnableShortCircuitQuery()) {
68+
ConnectContext connectContext = ConnectContext.get();
69+
if (!connectContext.getSessionVariable().isEnableShortCircuitQuery()) {
70+
return false;
71+
}
72+
// The short circuit produces no Arrow result at either end. PointQueryExecutor is not a
73+
// Coordinator, and Coordinator/NereidsCoordinator are the only places that register a
74+
// FlightSqlEndpointsLocation, so GetFlightInfo found none and failed the query with
75+
// "no FlightSqlEndpointsLocations"; the BE side cannot be pointed at either, since the lookup rpc
76+
// serializes with VMysqlResultWriter into PTabletKeyLookupResponse.row_batch and never creates the
77+
// ArrowFlightResultBlockBuffer that fetch_arrow_flight_schema looks up. Keep Arrow Flight SQL on
78+
// the normal execution path. This has to be decided here at plan time rather than when picking the
79+
// executor: OlapScanNode.computeTabletInfo and several rewrite and property rules read
80+
// StatementContext.isShortCircuitQuery() while building the plan. See #67368.
81+
if (connectContext.getConnectType() == ConnectType.ARROW_FLIGHT_SQL) {
6882
return false;
6983
}
7084
// Lazy point-query pruning does not preserve explicit PARTITION/TABLET restrictions.

‎fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1550,8 +1550,10 @@ public void executeAndSendResult(boolean isOutfileQuery, boolean isSendFields,
15501550
// need deferral (the BE buffers their result independently) but are captured by the
15511551
// same gate; the trade-off is their coordinator, query queue slot and query
15521552
// registration stay held until the next query / teardown instead of being released
1553-
// at the end of GetFlightInfo. Point queries use a different coordBase (not
1554-
// deferred). See #62259.
1553+
// at the end of GetFlightInfo. A short-circuit point query is the one case with a
1554+
// different coordBase, and it can no longer reach here: it has no Arrow result on
1555+
// either side, so LogicalResultSinkToShortCircuitPointQuery keeps Arrow Flight SQL
1556+
// on the normal execution path. See #62259 and #67368.
15551557
if (coordBase == coord) {
15561558
deferredForArrowFlight = true;
15571559
context.addFlightSqlDeferredExecutor(this);

‎fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/ShortCircuitPointQueryTest.java‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,11 +29,15 @@
2929
import org.apache.doris.nereids.trees.plans.logical.LogicalOlapScan;
3030
import org.apache.doris.nereids.util.MemoPatternMatchSupported;
3131
import org.apache.doris.nereids.util.PlanChecker;
32+
import org.apache.doris.qe.ConnectContext;
33+
import org.apache.doris.qe.ConnectContext.ConnectType;
3234
import org.apache.doris.utframe.TestWithFeService;
3335

3436
import org.junit.jupiter.api.Assertions;
3537
import org.junit.jupiter.api.Test;
3638

39+
import java.lang.reflect.Field;
40+
3741
/**
3842
* Regression test:
3943
* For short-circuit point query, we should not rewrite LogicalOlapScan to LogicalEmptyRelation
@@ -154,6 +158,34 @@ void testRemoteOlapTableDoesNotUseShortCircuit() throws Exception {
154158
.scanMatchShortCircuitCondition(scan));
155159
}
156160

161+
@Test
162+
void testArrowFlightSqlConnectionDoesNotUseShortCircuit() throws Exception {
163+
// The short circuit hands its rows back through PointQueryExecutor, which registers no
164+
// FlightSqlEndpointsLocation and leaves no Arrow result on the BE, so GetFlightInfo used to fail
165+
// with "fetch arrow flight schema failed, no FlightSqlEndpointsLocations" and drop the row.
166+
// An Arrow Flight SQL connection has to plan the normal execution path. See #67368.
167+
Field connectTypeField = ConnectContext.class.getDeclaredField("connectType");
168+
connectTypeField.setAccessible(true);
169+
ConnectType originConnectType = (ConnectType) connectTypeField.get(connectContext);
170+
try {
171+
connectTypeField.set(connectContext, ConnectType.ARROW_FLIGHT_SQL);
172+
Plan plan = rewrite("select * from tbl_point_query where `key` = 1");
173+
174+
Assertions.assertFalse(connectContext.getStatementContext().isShortCircuitQuery());
175+
// And the plan really is the ordinary one: tbl_point_query is empty, so it prunes to a
176+
// LogicalEmptyRelation, which is exactly what the short circuit suppresses in
177+
// testShortCircuitPointQueryKeepOlapScanWhenTableEmpty above.
178+
Assertions.assertTrue(plan.anyMatch(p -> p instanceof LogicalEmptyRelation));
179+
Assertions.assertFalse(plan.anyMatch(p -> p instanceof LogicalOlapScan));
180+
} finally {
181+
connectTypeField.set(connectContext, originConnectType);
182+
}
183+
184+
// The very same statement still short circuits on a MySQL connection.
185+
rewrite("select * from tbl_point_query where `key` = 1");
186+
Assertions.assertTrue(connectContext.getStatementContext().isShortCircuitQuery());
187+
}
188+
157189
private long getTabletId(String partitionName) throws Exception {
158190
Database database = Env.getCurrentInternalCatalog().getDbOrMetaException("test");
159191
OlapTable table = (OlapTable) database.getTableOrMetaException("tbl_partitioned_point_query");
Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
// Licensed to the Apache Software Foundation (ASF) under one
2+
// or more contributor license agreements. See the NOTICE file
3+
// distributed with this work for additional information
4+
// regarding copyright ownership. The ASF licenses this file
5+
// to you under the Apache License, Version 2.0 (the
6+
// "License"); you may not use this file except in compliance
7+
// with the License. You may obtain a copy of the License at
8+
//
9+
// http://www.apache.org/licenses/LICENSE-2.0
10+
//
11+
// Unless required by applicable law or agreed to in writing,
12+
// software distributed under the License is distributed on an
13+
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
// KIND, either express or implied. See the License for the
15+
// specific language governing permissions and limitations
16+
// under the License.
17+
18+
import org.apache.doris.regression.util.JdbcUtils
19+
20+
// Regression for https://github.com/apache/doris/issues/67368
21+
//
22+
// A UNIQUE KEY point query that matches the short circuit is executed by PointQueryExecutor instead of
23+
// a Coordinator, and neither end of that path can serve an Arrow Flight result:
24+
//
25+
// * Coordinator and NereidsCoordinator are the only places that register a FlightSqlEndpointsLocation,
26+
// so GetFlightInfo found no endpoint and failed with
27+
// "fetch arrow flight schema failed, no FlightSqlEndpointsLocations", dropping the row.
28+
// * The BE could not be pointed at either. tablet_fetch_data serializes with VMysqlResultWriter into
29+
// PTabletKeyLookupResponse.row_batch and runs no fragment, so the ArrowFlightResultBlockBuffer that
30+
// fetch_arrow_flight_schema looks up by finst id never exists.
31+
//
32+
// The fix keeps Arrow Flight SQL connections on the normal execution path, decided at plan time in
33+
// LogicalResultSinkToShortCircuitPointQuery. The table below is the one from the issue.
34+
suite("test_point_query_over_arrow_flight") {
35+
def mysqlConn = context.getConn()
36+
def flightConn = context.getArrowFlightSqlConnection()
37+
38+
def runOnMysql = { String stmt ->
39+
def (result, meta) = JdbcUtils.executeToList(mysqlConn, stmt)
40+
return result
41+
}
42+
def runOnFlight = { String stmt ->
43+
def (result, meta) = JdbcUtils.executeToList(flightConn, stmt)
44+
return result
45+
}
46+
// The suite level explain{} action always runs on the MySQL connection, but the whole point here is
47+
// which protocol asked for the plan, so read the explain text off each connection explicitly.
48+
def explainOn = { conn, String stmt ->
49+
def (rows, meta) = JdbcUtils.executeToList(conn, "explain " + stmt)
50+
return rows.collect { row -> row.get(0).toString() }.join("\n")
51+
}
52+
53+
def dbName = context.dbName
54+
runOnMysql "USE `${dbName}`"
55+
runOnFlight "USE `${dbName}`"
56+
57+
def tblName = "test_point_query_over_arrow_flight_tbl"
58+
runOnMysql "DROP TABLE IF EXISTS ${tblName}"
59+
runOnMysql """
60+
CREATE TABLE ${tblName} (
61+
`col1` SMALLINT NOT NULL,
62+
`col2` INT NOT NULL,
63+
`loc3` CHAR(10) NOT NULL,
64+
`value` CHAR(10) NOT NULL,
65+
INDEX col3 (`loc3`) USING INVERTED,
66+
INDEX col2_idx (`col2`) USING INVERTED
67+
) ENGINE=OLAP
68+
UNIQUE KEY(`col1`, `col2`, `loc3`)
69+
DISTRIBUTED BY HASH(`col1`, `col2`, `loc3`) BUCKETS 1
70+
PROPERTIES (
71+
"replication_allocation" = "tag.location.default: 1",
72+
"disable_auto_compaction" = "true",
73+
"bloom_filter_columns" = "col1",
74+
"store_row_column" = "true",
75+
"enable_mow_light_delete" = "false"
76+
)
77+
"""
78+
runOnMysql "INSERT INTO ${tblName} VALUES (10, 20, 'aabc', 'value')"
79+
80+
def pointQuery = "SELECT * FROM ${tblName} WHERE col1 = 10 AND col2 = 20 AND loc3 = 'aabc'"
81+
82+
// The short circuit is still taken on a MySQL connection: the fix is scoped to one protocol, it does
83+
// not disable the optimization. Assert this first, so a table that stopped qualifying for the short
84+
// circuit (schema or session variable drift) fails loudly here instead of making the flight
85+
// assertions below pass for the wrong reason.
86+
def mysqlExplain = explainOn(mysqlConn, pointQuery)
87+
assertTrue(mysqlExplain.contains("SHORT-CIRCUIT"),
88+
"the point query must still short circuit on a mysql connection, but got:\n" + mysqlExplain)
89+
90+
// The same statement must be planned on the normal path over Arrow Flight SQL.
91+
def flightExplain = explainOn(flightConn, pointQuery)
92+
assertFalse(flightExplain.contains("SHORT-CIRCUIT"),
93+
"the point query must not short circuit on an arrow flight connection, but got:\n" + flightExplain)
94+
95+
// This is the call that used to fail with "no FlightSqlEndpointsLocations".
96+
def (flightRows, flightMeta) = JdbcUtils.executeToList(flightConn, pointQuery)
97+
assertEquals(1, flightRows.size())
98+
assertEquals(10, flightRows[0][0] as int)
99+
assertEquals(20, flightRows[0][1] as int)
100+
assertEquals("aabc", flightRows[0][2].toString())
101+
assertEquals("value", flightRows[0][3].toString())
102+
103+
// Both protocols must see the same row, one through the short circuit and one through the normal
104+
// plan.
105+
def mysqlRows = runOnMysql(pointQuery)
106+
assertEquals(1, mysqlRows.size())
107+
assertEquals(mysqlRows[0].collect { it.toString() }, flightRows[0].collect { it.toString() })
108+
109+
// The BE produces the arrow batch, so the column types survive. Serving the point query result from
110+
// the FE instead would hand every column back as a string, because FlightSqlChannel.addResult builds
111+
// varchar vectors only.
112+
assertTrue(flightRows[0][0] instanceof Number,
113+
"col1 must stay numeric over arrow flight, but got: " + flightRows[0][0].getClass())
114+
assertTrue(flightRows[0][1] instanceof Number,
115+
"col2 must stay numeric over arrow flight, but got: " + flightRows[0][1].getClass())
116+
assertEquals(4, flightMeta.getColumnCount())
117+
118+
// A key that matches no row is planned the same way and used to fail with the same error, so it is
119+
// not enough for the statement above to be the only shape that works.
120+
def emptyRows = runOnFlight("SELECT * FROM ${tblName} WHERE col1 = 11 AND col2 = 20 AND loc3 = 'aabc'")
121+
assertEquals(0, emptyRows.size())
122+
123+
// The workaround reported in the issue keeps working, and a plain non point query on the same table
124+
// is unaffected.
125+
def hintRows = runOnFlight("SELECT /*+ SET_VAR(enable_short_circuit_query=false) */ * FROM ${tblName} "
126+
+ "WHERE col1 = 10 AND col2 = 20 AND loc3 = 'aabc'")
127+
assertEquals(1, hintRows.size())
128+
assertEquals(1, runOnFlight("SELECT col1 FROM ${tblName} ORDER BY col1").size())
129+
130+
runOnMysql "DROP TABLE IF EXISTS ${tblName}"
131+
}

0 commit comments

Comments
 (0)