You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 8608b8e
Browse filesBrowse the repository at this point in the historyBrowse files
[fix](remote-doris) Close the Flight SQL session a remote Doris scan opens on the remote frontend
A Remote Doris catalog with use_arrow_flight reads the other cluster over
Arrow Flight SQL: for every scan RemoteDorisScanNode performs a handshake
against a remote frontend, runs the query there and hands the endpoints to
the BE, which reads the rows from the remote BEs with DoGet. The handshake
opens a session on the remote frontend, and the scan closed the gRPC
channel but never sent CloseSession, so every scan left one session behind
until wait_timeout. Since apache#68101 a Flight session is a connection of the
remote frontend's one pool, counted against the catalog user's
max_user_connections; since apache#68266 nothing caps them any more. A hundred
scans within eight hours and the catalog user, MySQL clients included, is
refused there with "Reach limit of connections". The external regression
hit exactly that on 2026-09-21: the remote_doris suites point the catalog
at the frontend under test as root, and 48 leaked sessions took half of
root's quota.
RemoteDorisFlightSession is the session as an object: handshake, execute,
and close() = CloseSession (bounded to five seconds, so a remote frontend
that stopped answering cannot hang the local query's teardown) followed by
the channel and the allocator. RemoteDorisScanNode keeps it from getSplits
until stop(), which the coordinator calls when the local query closes or
is cancelled - when the BE is done with the remote query's endpoints. It
cannot be closed right after GetFlightInfo: the remote frontend cancels
whatever a closed session was still running, and when the remote table is
itself an external table in batch mode the remote query is deferred there
while the BE reads. A query that fails closes its session at once, so the
retry on the next node leaves nothing behind either.
For the same reason the local coordinator has to outlive dispatch when the
local query is an Arrow Flight SQL query, or apache#67503 would close it, and
the session with it, right after exec() while the local BE may still be
reading. The deferral gate's predicate is generalized from "has a batch
split source" to "the BE still depends on this scan after dispatch":
ScanNode.coordinatorMustOutliveDispatch() and
Coordinator.mustOutliveDispatch(), with the open session as the second
reason. Batch-mode external scans behave as before.
RemoteDorisScanNodeTest drives the scan node against an in-process Flight
SQL server that counts the sessions it is asked to close; the regression
suite test_remote_doris_flight_session scans through a catalog logging in
as a user of its own and asserts after each scan that the processlist
holds no ArrowFlightSQL session of that user.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
0 commit comments