Repository navigation
[Bug] Arrow Flight SQL: short-circuit point query returns no Flight endpoint (no FlightSqlEndpointsLocations) #67368
Description
Search before asking
- I had searched in the issues and found no similar issues.
Version
Apache Doris 4.1.3-rc02, commit 31263df4dc1d4d3a27517d264802cd4d6b92c874
Client: Python + ADBC Flight SQL driver (adbc_driver_flightsql), FE arrow_flight_sql_port = 41070.
The MySQL/JDBC protocol is used as the control path for comparison.
What's Wrong?
A UNIQUE KEY point query that qualifies for the point-query short-circuit path returns no Flight endpoint when executed over Python ADBC Flight SQL. The client fails with:
fetch arrow flight schema failed, no FlightSqlEndpointsLocations
The identical query with SET_VAR(enable_short_circuit_query=false) returns the full row over the same ADBC connection.
What You Expected?
With the point-query short circuit enabled, Flight SQL should return an endpoint and the query result, just like the normal execution path does.
How to Reproduce?
- Create the
UNIQUE KEYtable below and insert one row. - Run the default point query over Python ADBC; it fails with the no-endpoint error.
- Run the same point query with the short circuit disabled on the same ADBC connection; the row is returned.
DROP TABLE IF EXISTS table_3821461;
CREATE TABLE table_3821461 (
col1 SMALLINT NOT NULL,
col2 INT NOT NULL,
loc3 CHAR(10) NOT NULL,
value CHAR(10) NOT NULL,
INDEX col3 (loc3) USING INVERTED,
INDEX col2_idx (col2) USING INVERTED
) ENGINE=OLAP
UNIQUE KEY(col1,col2,loc3)
DISTRIBUTED BY HASH(col1,col2,loc3) BUCKETS 1
PROPERTIES (
"replication_allocation"="tag.location.default: 1",
"disable_auto_compaction"="true",
"bloom_filter_columns"="col1",
"store_row_column"="true",
"enable_mow_light_delete"="false"
);
INSERT INTO table_3821461 VALUES (10,20,'aabc','value');
-- Fails through Arrow Flight SQL.
SELECT * FROM table_3821461
WHERE col1=10 AND col2=20 AND loc3='aabc';
-- Succeeds through Arrow Flight SQL.
SELECT /*+ SET_VAR(enable_short_circuit_query=false) */ *
FROM table_3821461
WHERE col1=10 AND col2=20 AND loc3='aabc';Client side:
import adbc_driver_flightsql.dbapi as flight_sql
conn = flight_sql.connect(uri="grpc://127.0.0.1:41070",
db_kwargs={"username": "root", "password": ""})
cur = conn.cursor()
cur.execute("SELECT * FROM table_3821461 WHERE col1=10 AND col2=20 AND loc3='aabc'")
cur.fetch_arrow_table()Anything Else?
The short-circuit point query does not go through the normal coordinator/result-sink path, so no Flight endpoint location is ever registered for the query; the Flight SQL path appears not to be wired into that execution mode at all.
Workaround: set enable_short_circuit_query=false for the query. This is only a diagnostic/temporary workaround; it should not be applied silently by clients, since it changes the execution plan.
The table above is taken from the public point_query_p0/test_point_query regression suite, and the A/B comparison was confirmed on the same table and the same connection.
Tracking issue: #65615
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
Doris-Breakwater commentedon Sep 1, 2026
Breakwater-GitHub-Analysis-Slot: slot_f074cb282cd2
Initial assessment
This is a confirmed server-side Arrow Flight SQL compatibility bug in Apache Doris 4.1.3-rc02 (
31263df4dc1d4d3a27517d264802cd4d6b92c874), rather than an ADBC-only error. The report is sufficiently detailed for implementation: the exact commit and deterministic same-connection A/B test isolate the failure to the short-circuit execution mode, and the source at that commit explains the empty endpoint list.Impact is limited to queries that are eligible for the point-query short-circuit path and are executed through Arrow Flight SQL. It prevents results from being fetched, but there is no evidence here of data loss or incorrect data. The stated
enable_short_circuit_query=falseworkaround is valid as a temporary, explicit workaround. I did not independently start a 4.1.3-rc02 cluster; the conclusion below is based on the supplied reproducer plus a trace of the exact reported source revision.Verified code path and root cause
DorisFlightSqlProducer.executeQueryStatement()clears prior endpoint locations, executes the statement, and for a remote result immediately callsfetchArrowFlightSchema()before constructingFlightInfo: DorisFlightSqlProducer.java#L187-L284.returnResultFromLocal=false. When Nereids marks the query as short-circuit eligible,StmtExecutor.executeAndSendResult()selectsPointQueryExecutor, callsexec(), then takes the Arrow Flight branch and returns before the normalgetNext()loop: StmtExecutor.java#L1381-L1439.PointQueryExecutor.exec()is intentionally a no-op; the tablet lookup and row retrieval happen only ingetNext(): PointQueryExecutor.java#L247-L300. Therefore the Flight branch returns before the point lookup is even requested.CoordinatorandNereidsCoordinatorresult-sink paths (Coordinator.java#L853-L886, NereidsCoordinator.java#L463-L484).PointQueryExecutordoes not create a result sink or calladdFlightSqlEndpointsLocation().FlightSqlConnectProcessor.fetchArrowFlightSchema()reaches its deterministic empty-list guard and throws the exact reported error: FlightSqlConnectProcessor.java#L112-L116.This is more than missing endpoint metadata. The point-query RPC response is explicitly a MySQL row batch (
is_binary_rowin internal_service.proto#L378-L406), and the BE path usesVMysqlResultWriter: point_query_executor.cpp#L645-L666. Adding aFlightSqlEndpointsLocationalone would advertise an endpoint for which no Flight result stream or Arrow schema exists.Both direct and prepared-statement FlightInfo paths call the same
executeQueryStatement()method, so the gap is not specific to one ADBC statement mode.Missing information
No information is blocking initial triage or root-cause confirmation. For the interoperability/version matrix only, it would be useful—but not required for the fix—to record the installed versions of
adbc_driver_flightsql,adbc_driver_manager, andpyarrow. An FE stack trace/query ID would provide runtime confirmation of the guard above, but should not be required before accepting this issue.The issue currently has no labels, assignee, milestone, or linked development work. Suggested labels are
kind/bugandarea/rpc, consistent with the Arrow Flight SQL tracking issue.Recommended next steps
LogicalResultSinkToShortCircuitPointQueryfrom selecting the short-circuit mode for Arrow Flight SQL connections, so these queries use the existing coordinator/result-sink path and produce a valid Flight endpoint. The eligibility decision is made here: LogicalResultSinkToShortCircuitPointQuery.java#L63-L89. This is the smallest safe server-side compatibility fix, at the cost of the point-query optimization for Flight clients.VectorSchemaRoot. This is a larger change because the existing response is MySQL-encoded. Do not implement an endpoint-only patch.arrow_flight_sql_p0, using this UNIQUE KEY MOW row-store table. Cover both a matching key and an empty result with short-circuit enabled by default, plus the disabled-short-circuit control. If the correctness-first gate is chosen, add an FE rule test proving Flight connections are not marked short-circuit; if full support is chosen, add a test proving the optimized path itself was used.[fix](arrow-flight) Do not take the point-query short circuit on an A…
[fix](arrow-flight) Release a finished Flight query's coordinator ins…
branch-4.1: [fix](arrow-flight) Release a finished Flight query's coo…
branch-4.1: [fix](arrow-flight) Release a finished Flight query's coo…
[refactor](session) Replace the execution layer's ConnectType branche…