Repository navigation
Description
This issue tracks the work that makes MySQL and Arrow Flight SQL equal front ends over one session
layer in the FE: one session object, one result-encoding abstraction, one Doris-to-Arrow type
mapping, one connection pool and quota, and one prepared-statement registry.
Part of #65615. The design is being written up as a DSIP; this issue is the implementation tracker.
If you hit one of the problems listed here, please open a separate bug report and link it back.
#67578 builds on this one: completing the Arrow Flight SQL server and shipping an official Doris
ADBC driver needs the session lifecycle, the result path and the prepared-statement registry that
this issue unifies. The two can be discussed in parallel, but that work only lands cleanly on top
of what is tracked here.
Relationship to the earlier protocol SPI work. #60355 / #60361 (both closed) decoupled the
listeners: how a protocol server is discovered, configured and started. This issue covers
everything behind the listener - session state, the result path, type mapping, connection governance
and prepared statements - which is still written against the MySQL protocol and special-cased for
Arrow Flight SQL.
Why
An Arrow Flight SQL session already is a ConnectContext, SHOW PROCESSLIST lists it and KILL
works on it. What is not shared is most of what the execution layer does afterwards. A read of the
FE session/connection/result paths and the BE result sinks turned up the following; items with a PR
have been verified locally, the rest are code-reading conclusions and are marked as such.
- The execution layer branches on the protocol in roughly 40 places (
StmtExecutor18,
ConnectProcessor8, nineCommands, plusCoordinator,NereidsCoordinator,
StatementContext,FrontendServiceImpl) and reaches forMysqlChannelin roughly 50. A Flight
session throws fromgetMysqlChannel(), so every unguarded reach is a live failure rather than a
style problem - see [fix](arrow-flight) Forward a statement to the master FE without touching the MySQL channel #67569 for one that reached production. - There are three Doris-to-Arrow type mappings: the BE's
convert_to_arrow_type, the FE's
FlightSqlSchemaHelper.getArrowType(used byCommandGetTables), andFlightSqlChannel, which
encodes every FE-side result asutf8. The FE mirror already disagrees with the BE in four types. - Flight connections are governed by their own pool with their own limit
(arrow_flight_max_connections) plus a per-user token LRU, soqe_max_connectionand
max_user_connectionsdo not apply to them, and a live session that the LRU evicts surfaces to the
client asinvalid bearer token. - Flight prepared statements do not use the FE's prepared-statement machinery:
createPreparedStatementstores the SQL text and returns a placeholder schema, and parameter
binding is not implemented.
Status convention
Same as #65615: [x] means merged or confirmed complete; [ ] means open or needs follow-up.
Stage 0 - independent bug fixes
Defects found by the survey that do not depend on the refactor. All three are merged and carry
dev/4.2.x, so they reach the 4.2 branch ahead of the refactor.
- A Flight session cannot forward a statement to the master FE: [fix](arrow-flight) Forward a statement to the master FE without touching the MySQL channel #67569, merged 2026-09-09.
FEOpExecutor.buildStmtForwardParams()readsCLIENT_DEPRECATE_EOFstraight off the MySQL channel
(introduced by [fix](protocol) Support CLIENT_DEPRECATE_EOF to fix empty result with MySQL driver 9.5.0 #61050), so on a multi-FE deployment every statement a Flight connection has to
forward - any DDL issued to a follower or observer, or anything at all under
force_forward_all_queries- fails withgetMysqlChannel not in mysql connection. -
ConnectTypedeclared twice, plus an unreachable dispatch on the forward path: [refactor](qe) Merge the duplicate ConnectType enum and drop the unreachable forward branch #67572,
merged 2026-09-08. Cleanup, no behavior change. One leftover, deliberately deferred to the
ProtocolAdapterPR:ConnectProcessor.connectTypeis still a redundant mirror of
ctx.getConnectType(). - A Flight session reports
0.0.0.0:0as its client address: [fix](arrow-flight) Report the real client address of an Arrow Flight SQL session #67576, merged 2026-09-10.
TheHostcolumn of
SHOW PROCESSLISTandinformation_schema.processlist, the audit log'sclient_ip, and the kill
and timeout warnings all showed the placeholder, although the real address is resolved when the
bearer token is issued.
Stage 1 - one session, one result path
Behavior-preserving refactor, guarded by golden tests that pin the MySQL packet bytes. Complete:
all five PRs merged between 2026-09-11 and 2026-09-14, and the performance baseline shows no change.
- Golden tests first: [test](protocol) Record a golden baseline of MySQL packets and Arrow Flight results #67789, merged 2026-09-11. A recording
MysqlChanneldrives a statement set through
MysqlConnectProcessor/StmtExecutorand the produced packets are compared byte for byte; the same
statements then run through the Flight processor and their schema and rows are compared. This is the
safety net for everything below, so it lands before any refactor. 27 MySQL cases and 8 Flight
statements, recorded as annotated hexdumps rather than binary blobs so a diff is reviewable. Two
responses are recorded as shape rather than bytes, because their payload moves with unrelated
changes: a parser error carries the grammar's whole keyword list, and anEXPLAINcarries the
current plan text.COM_STMT_EXECUTEis out of scope for the golden files because
supportHandleByFe()is false for it, so its result always comes from a BE - the cursor-fetch
packet boundaries stay covered byprepared_stmt_p0instead. -
ProtocolAdapter: [refactor](session) Bind a ConnectContext to one ProtocolAdapter instead of a Flight subclass #67835, merged 2026-09-11. The per-protocol half ofConnectContext
(channel, capabilities, accept loop, close, theCOM_STMT_EXECUTEstate, the Flight result cache,
endpoints and deferred executors) moves behind an interface with a MySQL and a Flight
implementation;ConnectContext.forMysql/forMysqlProxy/forFlightcreate the bound
context.ConnectContextkeeps its class name and getter signatures - it is referenced by ~900
files - andFlightSqlConnectContextgoes away. Flight commands get a per-session lock:
ConnectContextis not thread safe and Flight did not serialize a session's calls. Verified
against the [test](protocol) Record a golden baseline of MySQL packets and Arrow Flight results #67789 baseline: not a byte changed. The internal (no-client) context stays a MySQL
context over aDummyMysqlChannelfor now, because the execution layer still keys its result
path onConnectType; a distinct internal adapter follows once those branches are gone. - Package move: [refactor](arrow-flight) Move Arrow Flight SQL out of the service package into org.apache.doris.arrowflight #67866, merged 2026-09-11.
org.apache.doris.service.arrowflightbecomes
org.apache.doris.arrowflight. Flight was filed underservicein 2023 ([feature-wip](arrow-flight)(step3) Support authentication and user session #24772) next to the thrift
FrontendServiceImpl; it is now a peer of the top-levelmysqlpackage, and [refactor](session) Bind a ConnectContext to one ProtocolAdapter instead of a Flight subclass #67835 already had to
mirror aprotocolsub-package on both sides. A puregit mv(36 files, onlypackage/import
lines change, nothing references the old name by string), sequenced beforeResultSenderso the
new implementations land in the final place. -
ResultSender: [refactor](session) Move result encoding behind ResultSender #67883, merged 2026-09-12. The result-encoding half moves out of
StmtExecutor/ConnectProcessor:qe.protocol.ResultSenderwithMysqlResultSender/
FlightResultSender,StmtExecutorwithout a serializer field orMysqlChannelparameters, the
three text-result paths (EXPLAIN,EXPLAIN PLAN PROCESS,REPLAY) through the one
sendResultSet, andConnectProcessorwithout itsconnectTypefield (the per-statement
protocol work of a multi-statement request isadapter.finishStatement). The MySQL golden is
byte-identical; the Flight golden changes in exactly one entry,EXPLAIN PLAN PROCESS, which
had no Flight branch before and now returns its rows. Also carries the leftovers of the [refactor](session) Bind a ConnectContext to one ProtocolAdapter instead of a Flight subclass #67835
review (lock wait bound =getExecTimeoutS(),FlightRuntimeExceptionpassed through the
producer's catch-alls, a WARN when the lock is given up). - Remaining protocol branches in the execution layer go to zero: [refactor](session) Replace the execution layer's ConnectType branches with protocol capabilities #67900, merged 2026-09-14.
StmtExecutor,ConnectContext.supportHandleByFe, the nine insert / transactionCommands, both
coordinators, the short-circuit rule andFEOpExecutorstop askinggetConnectType();
ProtocolAdaptergains one method per use:canReplayForwardedQueryResult,supportsFeSideResult,
supportsShortCircuitPointQuery,canRetryQuery, the lifecycle hooksbeforeStatement/
beforeAttempt/beforeQuery/returnsResultFromLocal, andfillForwardRequest(with
MysqlProtocolAdapter.restoreFromForwardRequestas the master's side). The Flight-only
returnResultFromLocalflag is no longer flipped from outside the adapter, andFlightResultSender
no longer has to undo a flip forEXPLAIN. The behavior changes, all recorded in the MySQL golden
(28 -> 33 cases; the Flight golden is byte-identical): the channel is reset once, when a statement
starts, instead of in the query path and in nine commands, so a request from a client without
CLIENT_MULTI_STATEMENTSwhose last statement is not a query (select 1; set @a = 1) delivers
only the last response instead of the buffered result set of theSELECTfollowed by anOK-- a
stream no client parses;MysqlChannel.reset()now also rewinds the sequence id to the last packet
the client saw, so such a response (select 1; select 2,select 1; select * from no_such_table)
starts at sequence id 1 instead of leaving the hole pymysql and libmysqlclient rejected (the
pre-existing bug noted on 2026-09-13, fixed here because the reset-per-statement had widened it);
ProxyMysqlChanneldrops a failed attempt's packets on reset, so a forwarded query the master
retries no longer delivers both attempts' packets; and a Flight query replanned after a cloud
NEED_REPLANerror starts every attempt throughbeforeAttempt, which withdraws exactly the
endpoints the failed attempt registered. The recording channel of the golden models the send buffer
and the sequence rewind, so the golden shows what reaches the client. Wired end to end by
ProtocolCapabilityWiringTest(forward refusal, retry, an internal executor answering on the
caller's connection) and three new regression suites:test_arrow_flight_session_lifecycle,
test_multi_statement_response(a raw MySQL-protocol client covers all four
CLIENT_MULTI_STATEMENTSxCLIENT_DEPRECATE_EOFcombinations) and the two-FE
test_mysql_forward_to_master. - Performance baseline, run 2026-09-13: three FE builds - before Stage 1 (
60042611fea), after
the package move (af333cc583d) and the [refactor](session) Replace the execution layer's ConnectType branches with protocol capabilities #67900 branch - against the same BE, data and metadata.
A 10M-row wide table drained overmysql --quickand over a raw socket,select 1and one-row BE
queries over Connector/J at 1 and 8 threads, ADBCfetch_arrow_table/select 1/
show variables. Every difference is within the run-to-run noise of a single build (FE-side paths
within +/-1%; the few cells above 3% changed sign on interleaved re-runs).
#67520 (merged 2026-09-10) lands in the middle of this surface: it touches ConnectProcessor,
MysqlConnectProcessor, StmtExecutor, FEOpExecutor, MysqlProto and ConnectContext, and adds
MysqlCursorFetchCompatibility, a client-capability branch that decides where a cursor result's
terminators go. The adapter extraction absorbs it rather than works around it: that logic belongs to
the MySQL adapter (#67835 moved it there as MysqlProtocolAdapter.clientConsumesCursorMetadataTerminator).
#67753 (merged 2026-09-11, right after #67835) adds one more thing the ResultSender step has to
carry: an internal query run on behalf of a client (the IVM dry run) streams its rows to that
client's MySQL channel, so StmtExecutor grew channel-taking overloads of its send methods. The
result sender is handed over by the caller in that case rather than taken from the executor's own
session.
Stage 2 - one type mapping
Deferred (2026-09-14). The BE Arrow type layer is being refactored separately - the refactor
#67530 was told to wait for - and changing the mapping now would collide with it, so #67530 and
#65789 were closed unmerged and this stage lands after that refactor, on top of its mapping. Until
then every new interface uses the current mapping exactly as it is, through one FE entry point
(FlightSqlSchemaHelper.getArrowTypeextracted intoDorisArrowTypeMappingwith no value
changed), and the known-wrong cells are kept deliberately rather than fixed piecemeal:GetTables
reports TIMESTAMPTZ with a fixedUTCzone andNullfor TIMEV2 / VARBINARY / AGG_STATE;
AGG_STATE travels asutf8;doris_typemetadata covers only top-level LARGEINT / IPV4 / IPV6, no
ARROW:extension:nameanywhere; FE-side results are allutf8; the oversized-columnlarge_utf8
mismatch of #65789 stays. The server-side items of #67578 Part A (session actions, metadata
commands, prepared statements, cancellation, parallel endpoints, ingest) go first instead, with
Stage 3 between the first two, and the BE-touching ones stay clear of the type layer.
- The one FE entry point first, done now:
FlightSqlSchemaHelper.getArrowTypeand the field
builder it feeds extracted intoorg.apache.doris.arrow.DorisArrowTypeMappingwith no value changed,
the mapping pinned as it stands by a table-driven unit test (one row perPrimitiveType, the
known-wrong cells included and marked) and by a regression suite that reads theGetTablesschema
of a table of every type through a raw Flight SQL client. [refactor](arrow-flight) Extract the Doris-to-Arrow type mapping into DorisArrowTypeMapping #68315, opened 2026-09-21. - One enumerable Doris-to-Arrow mapping, shared by a table-driven BE unit test and an FE test that
validates the FE mirror against the same golden file, and the four known FE/BE disagreements fixed.
[feature](timestamp_ns) Add end-to-end TIMESTAMP_NS support #66761 added TIMESTAMP_NS to both mappings separately - one more line inarrow_row_batch.cppand
one inFlightSqlSchemaHelper- which is exactly the drift this stage removes; the golden file
covers it. - Field metadata completed on the BE:
doris_typefor every non-native type plus
ARROW:extension:name(doris.largeint,doris.ipv4,doris.ipv6,doris.bitmap,doris.hll,
doris.quantile_state,doris.agg_state,doris.time), with JSONB and VARIANT using the canonical
arrow.jsonextension. Storage types do not change:convert_to_arrow_typealso serves the
Spark and Flink connectors' read path and Python UDFs, so changing them would change those
protocols. In particular LARGEINT stays a decimal string rather than becomingdecimal128(38,0),
which cannot represent an int128 (39 decimal digits; Arrow caps decimal128 precision at 38), and
IPV4/IPV6 stayint32/utf8rather thanfixed_size_binary. - FE-side results (
SHOW,EXPLAIN, replayed proxy results) built as typed vectors instead of
all-utf8. - A cross-protocol regression suite: the same query over JDBC and over Flight, compared column by
column across the whole type matrix.
Stage 3 - one connection pool and quota
Builds on #67504 (merged 2026-09-07), which stopped a finished Flight query from holding its
coordinator until wait_timeout.
- Single pool:
FlightSqlConnectPoolMgrfolds intoConnectPoolMgr, and Flight registration goes
throughqe_max_connectionandmax_user_connectionslike any other connection. [refactor](arrow-flight) One connection pool for MySQL connections and Arrow Flight SQL sessions #68101, merged
2026-09-20 (46e1c671220);arrow_flight_max_connectionsis the sub-quota, defaulting to half of
qe_max_connection. - The bearer token becomes a session credential: no token cache, no per-user LRU, no separate
lifetime.arrow_flight_token_cache_sizeandarrow_flight_token_alive_time_secondare deprecated
with a warning for one major version. [refactor](arrow-flight) Make the bearer token the credential of exactly one session #68266, merged 2026-09-21 (4de617b58e0). - A
Protocolcolumn onSHOW PROCESSLISTandinformation_schema.processlist(FE and the BE
schema scanner), plus aprotocolfield in the audit log. In [refactor](arrow-flight) Make the bearer token the credential of exactly one session #68266 as well, merged 2026-09-21
(4de617b58e0), together with the connection governance regression suite
test_connection_governanceand a fix the audit column needs on an upgraded cluster: an alter job
created by a frontend daemon (InternalSchemaInitializeradding a column toaudit_log) sent an
emptyTQueryGlobalssince [Chore](alter) add TQueryGlobals/TQueryOptions params to TAlterTabletReqV2 #58085 and the backend cancelled it.
Stage 4 - prepared statements
- The prepared-statement registry is extracted from the MySQL
COM_STMT_*handling so both front
ends share it. - Flight
createPreparedStatement/getFlightInfoPreparedStatement/closePreparedStatementgo
through it and return the real result and parameter schemas. -
acceptPutPreparedStatementQuerybinds Arrow parameter batches, so Flight SQL JDBC with
useServerPrepStmts=trueand the ADBC default path work.
Known gaps, not yet scheduled
Code-reading conclusions from the same survey. They are not blocking the stages above; each needs its
own report or PR.
- A forwarded statement's result set is dropped over Flight. Even after [fix](arrow-flight) Forward a statement to the master FE without touching the MySQL channel #67569, a statement
that is forwarded to the master and returns rows gives the Flight client nothing, because the proxy
result is replayed byConnectProcessor.finalizeCommand(), which is MySQL-only. Affects only
statements that both need forwarding and carry a result set. -
arrow_flight_token_alive_time_secondis applied in the wrong unit.createToken()converts
the configured seconds as if they were minutes, so the recorded expiry is 60x too far out; the token
is actually evicted by the Guava cache's own expiry, which makes the explicit expiry check dead code.
Gone with [refactor](arrow-flight) Make the bearer token the credential of exactly one session #68266 (merged 2026-09-21): the setting is a deprecated no-op andFlightTokenDetailsis
deleted. -
AGG_STATEis advertised asutf8while carrying non-UTF-8 bytes - the same shape as the
IcebergBINARYproblem in [Bug] Arrow Flight SQL: Iceberg BINARY column is declared as Arrow string but carries invalid UTF-8 #67371. -
select 1andselect @@varalways go to the BE on a Flight session, because
supportHandleByFe()is hard-coded false for Arrow Flight SQL. Enabling FE-side results for Flight
is deliberately out of scope until the FE result path is typed (Stage 2). -
SHOW TABLES / DATABASES / COLUMNS / TABLE STATUS ... WHEREhang on a Flight session. The
four commands run an internal query on the session's own context, so the plan gets an Arrow result
sink while the frontend pulls the rows overfetch_data; the request waits until it times out. The
LIKEform takes the metadata path and is unaffected. Found while writing [refactor](session) Replace the execution layer's ConnectType branches with protocol capabilities #67900, which
deliberately leaves that path as it is (the coordinators keep deciding on the per-statement flag,
not on a protocol capability). - The master answers a forwarded query with
statusCode1105.ConnectProcessor.proxyExecute
reports 0 only when the state isOK; a query ends inEOF, so the follower sees a failure code
with a null message and its multi-statement loop stops after the first statement:select 1; select 2on a follower withforce_forward_all_queriesruns onlyselect 1. - The handshake does not advertise
CLIENT_MULTI_STATEMENTS/CLIENT_MULTI_RESULTS.
Connector/J only asks forCLIENT_MULTI_STATEMENTSwhen the server advertised it, so a JDBC client
withallowMultiQueries=truenever receives the intermediate responses of a multi-statement
request, although the server side handles them (the golden shows it, and the mysql CLI, which sets
the flag regardless, gets them).test_multi_statement_responsereaches that path with a raw
protocol client instead.
Upstream PRs this work depends on
- [fix](arrow-flight) Release a finished Flight query's coordinator instead of holding it until wait_timeout #67504 - release a finished Flight query's coordinator instead of holding it until
wait_timeout. Merged 2026-09-07; unblocks Stage 1. - [fix](arrow-flight) Keep the Doris type of a nested LARGEINT in the Arrow schema #67530 - keep the Doris type of a nested LARGEINT in the Arrow schema. Closed unmerged on
2026-09-14: the BE type-layer refactor it was told to wait for is going ahead separately, and
Stage 2 now lands after that refactor instead (note under Stage 2). The nested and JSON / VARIANT
metadata it carried is part of that stage. - [fix](arrow-flight) Fix utf8/large string schema-array mismatch for oversized columns #65789 - utf8/large-string schema-array mismatch for oversized columns. Closed unmerged on
2026-09-14; the mismatch itself remains and gets its own fix after the type-layer refactor, and
arrow_block_convertor.cppis left alone until then.
Progress
- 2026-09-07: survey and design complete; Stage 0 PRs submitted.
- 2026-09-08 .. 2026-09-10: Stage 0 merged in full ([refactor](qe) Merge the duplicate ConnectType enum and drop the unreachable forward branch #67572, [fix](arrow-flight) Forward a statement to the master FE without touching the MySQL channel #67569, [fix](arrow-flight) Report the real client address of an Arrow Flight SQL session #67576), each labelled
dev/4.2.x. - 2026-09-10: Stage 1 opened with the golden baseline, [test](protocol) Record a golden baseline of MySQL packets and Arrow Flight results #67789.
- 2026-09-11: [test](protocol) Record a golden baseline of MySQL packets and Arrow Flight results #67789 merged. The recorded packets are now the gate for the rest of Stage 1: each
refactor PR runs both golden tests before and after, and since those PRs are behavior-preserving,
a changed byte means the refactor changed something it should not have. Stage 2 remains parked
on [fix](arrow-flight) Keep the Doris type of a nested LARGEINT in the Arrow schema #67530. - 2026-09-11: [refactor](session) Bind a ConnectContext to one ProtocolAdapter instead of a Flight subclass #67835 opened and merged the same day - the
ProtocolAdapterextraction, first
refactor PR of Stage 1. Both golden tests pass unchanged on it. Next is theResultSender
extraction, which also has to absorb the channel-taking send overloads [fix](ivm) Answer FE-computable dry runs on the frontend instead of a placeholder backend #67753 added. - 2026-09-11: [refactor](arrow-flight) Move Arrow Flight SQL out of the service package into org.apache.doris.arrowflight #67866 opened and merged the same day - the
service.arrowflight->arrowflight
package move, a mechanical PR slotted in beforeResultSender. Flight now lives in the top-level
org.apache.doris.arrowflightpackage, a peer oforg.apache.doris.mysql;ResultSenderis next
and its implementations go tomysql.protocol/arrowflight.protocol. - 2026-09-12: [refactor](session) Move result encoding behind ResultSender #67883 opened - the
ResultSenderextraction, second refactor PR of Stage 1. Only
the expected Flight golden entry changed (EXPLAIN PLAN PROCESSgains a result); everything
MySQL is byte for byte. What remains for Stage 1 is the capability-predicate PR (the branches
outside the result path) and the performance baseline. - 2026-09-12: [refactor](session) Move result encoding behind ResultSender #67883 merged.
StmtExecutorandConnectProcessorno longer encode anything
themselves; every protocol branch left in the execution layer is now a candidate for a capability
predicate onProtocolAdapter, and that PR is next. The list, on the merged master: five
ConnectTypechecks inStmtExecutor(including the Flight forward refusal that [fix](arrow-flight) Forward a statement to the master FE without touching the MySQL channel #67569 added),
supportHandleByFe(), the nineCommands that reset the MySQL channel, the two coordinators and
QueryProcessor,StatementContext.close, the short-circuit rule andFEOpExecutor. The
performance baseline is still pending. - 2026-09-13: [refactor](session) Replace the execution layer's ConnectType branches with protocol capabilities #67900 opened - the capability-predicate PR, last refactor PR of Stage 1. After it,
grep 'ConnectType\.\|getMysqlChannel()'overqe/**andnereids/**(outside*/protocol/)
finds only theConnectContext.getMysqlChannel()delegate and two lines ofMysqlConnectProcessor
that read the client's packets. Two things found on the way, filed for their own fixes rather than
folded in: on a Flight session,SHOW TABLES / DATABASES / COLUMNS / TABLE STATUS ... WHERErun an
internal query on the session's own context, so the plan gets an Arrow result sink while the
frontend tries to pull the rows overfetch_data(the request hangs until it times out); and
MysqlChannel.reset()does not rewind the sequence id, so the response a client without
CLIENT_MULTI_STATEMENTSgets forselect 1; select 2starts at sequence id 4, which pymysql and
libmysqlclient reject (Connector/J does not check). The internal adapter for the no-client context
and the performance baseline remain. - 2026-09-14: [refactor](session) Replace the execution layer's ConnectType branches with protocol capabilities #67900 merged; Stage 1 is complete. The review added a per-attempt hook,
beforeAttempt, called at the top ofStmtExecutor.executefor the first attempt and for each
replan retry: a Flight attempt that failed afterbeforeQueryused to leave the next attempt in the
"result on the backends" state, and its endpoints next to the retry's. The sequence-id fix for
MysqlChannel.reset()went into the same PR rather than its own, because the reset-per-statement
had widened it fromselect 1; select 2to any request withoutCLIENT_MULTI_STATEMENTSwhose
later statement fails. The performance baseline (three FE builds against one BE, MySQL and Flight
paths) shows no measurable change. Left for their own PRs, listed under Known gaps: the Flight
SHOW ... WHEREhang, the master's 1105 for a forwarded query, and the un-advertised
CLIENT_MULTI_STATEMENTS. The internal adapter for the no-client context is no longer a
prerequisite for anything - nothing in the execution layer keys on the context type now - so it is
not scheduled. Stage 2 still waits on [fix](arrow-flight) Keep the Doris type of a nested LARGEINT in the Arrow schema #67530. - 2026-09-14: [fix](arrow-flight) Keep the Doris type of a nested LARGEINT in the Arrow schema #67530 and [fix](arrow-flight) Fix utf8/large string schema-array mismatch for oversized columns #65789 closed unmerged; Stage 2 deferred behind the separate BE type-layer
refactor (note under Stage 2). The order from here: session actions ([Tracking] Complete the Arrow Flight SQL server and ship an official Doris ADBC driver #67578 A1), Stage 3, the FE
mapping entry point (no value changes) together with the metadata commands ([Tracking] Complete the Arrow Flight SQL server and ship an official Doris ADBC driver #67578 A3), Stage 4
withExecuteSchema([Tracking] Complete the Arrow Flight SQL server and ship an official Doris ADBC driver #67578 A5), cancellation / renewal / polling (A6), then the BE-touching items
that stay clear of the type layer - parallel endpoints with self-contained tickets, ingest over the
existing Arrow load format, IPC compression - and Stage 2 last, once the refactored mapping is in.
PRs for the [Tracking] Complete the Arrow Flight SQL server and ship an official Doris ADBC driver #67578 items reference both trackers. - 2026-09-14: [feature](arrow-flight) Serve the Flight SQL session actions on the Doris session #67966 opened - the session actions ([Tracking] Complete the Arrow Flight SQL server and ship an official Doris ADBC driver #67578 A1), first of the Stage 5 items. An option
is the statement that sets it (catalog->SWITCH,schema->USE, any other name ->
SET SESSION), run throughFlightSqlConnectProcessoras a command of the session, answered per
name;GetSessionOptionsisSHOW VARIABLESas strings;CloseSessionwas already invalidating
the token synchronously, so that half of A1 was verified rather than changed. No type mapping
involved. Next: Stage 3. - 2026-09-17: [feature](arrow-flight) Serve the Flight SQL session actions on the Doris session #67966 merged (
76d9221dacc): Stage 5 item A1 done. [refactor](arrow-flight) One connection pool for MySQL connections and Arrow Flight SQL sessions #68101 opened - Stage 3, item 1: one
connection pool.FlightSqlConnectPoolMgris deleted;ConnectPoolMgrregisters either protocol
againstqe_max_connectionandmax_user_connections, keeps a Flight sub-quota
(arrow_flight_max_connections, now defaulting to -1 = followqe_max_connection, capped at it) and
a peer-identity index, and itsunregisterConnectionreleases the protocol's session state through
the newProtocolAdapter.releaseSession(the [feature](arrow-flight) Serve the Flight SQL session actions on the Doris session #67966 teardown contract kept); a Flight session
refused for a limit gets MySQL's sentence asRESOURCE_EXHAUSTEDand its token invalidated;
ProtocolAdapter.connectPoolis gone. Regressiontest_connection_quotauses
max_user_connections = 4because the per-user token LRU still evicts the oldest session at
max_user_connections / 2tokens - that cache goes with the next item. Next: the bearer token as
the session credential. - 2026-09-20: [refactor](arrow-flight) One connection pool for MySQL connections and Arrow Flight SQL sessions #68101 merged (
46e1c671220): Stage 3, item 1 done. [refactor](arrow-flight) Make the bearer token the credential of exactly one session #68266 opened - Stage 3, item 2:
the bearer token is the peer identity a session is registered in the pool under and nothing else.
The session opens at the handshake that authenticates the password (a session that does not fit is
refused there, in MySQL's words, and no token is issued), a token is valid exactly as long as its
session, and the session ends only withCloseSession,KILL CONNECTIONorwait_timeout- the
very next call under the token isUNAUTHENTICATED. Decided for it: a full quota refuses rather
than evicting the least recently used session (the eviction was a side effect of the Guava cache,
not a design), and there is no idle timeout of Flight's own -wait_timeout, settable through the
session options, governs both protocols. Thetokenspackage, both caches, the created-session
flag and the creation lock are deleted; the two token settings are deprecated no-ops reported at
startup;test_connection_quotais back tomax_user_connections = 1, and
test_bearer_token_lifecyclecovers the three ways a session ends. Next: theProtocolcolumn. - 2026-09-20: Stage 3, items 3 and 4 pushed onto [refactor](arrow-flight) Make the bearer token the credential of exactly one session #68266 (still open) as three more commits. A
Protocolcolumn (MySQL/ArrowFlightSQL) is the last column ofSHOW PROCESSLIST,
information_schema.processlistand/rest/v1/session; the audit event carries it asProtocol
andaudit_loggains aprotocolcolumn; a row of a frontend on the other version is padded or
cut to the reader's columns (FE and BE scanner alike).test_connection_governanceis the
operator's view of a MySQL connection and a Flight session together: both processlists agree
column for column, the Flight session's ownSHOW PROCESSLIST, the shared quota's refusal over
either protocol,KILL CONNECTIONin both directions,wait_timeout. Found on the way and fixed
in its own commit (worth picking): since [Chore](alter) add TQueryGlobals/TQueryOptions params to TAlterTabletReqV2 #58085 an alter job created without a session sent an
emptyTQueryGlobals, soInternalSchemaInitializer's upgrade ofaudit_logwas cancelled by the
backend (now_stringmissing) and every audit column added since only existed on clusters created
after it. Next:DorisArrowTypeMappingentry (Stage 2, the one item done now) and the metadata
commands (Stage 5 A3). - 2026-09-21: Stage 3 complete - [refactor](arrow-flight) Make the bearer token the credential of exactly one session #68266 merged as
4de617b58e0. Stage 2's one present item opened as
[refactor](arrow-flight) Extract the Doris-to-Arrow type mapping into DorisArrowTypeMapping #68315: the mapping moves out ofFlightSqlSchemaHelperintoDorisArrowTypeMappingunchanged,
with a table-driven unit test recording every cell as it stands (the first commit runs the same
table against the old code) and a regression suite recording theGetTablesschema a raw Flight SQL
client sees for a column of every type; theUTCzone on TIMESTAMPTZ andNullfor AGG_STATE are
pinned there deliberately, per the deferral above. Next: the metadata commands (Stage 5 A3).
Activity
[test](protocol) Record a golden baseline of MySQL packets and Arrow …
[refactor](session) Bind a ConnectContext to one ProtocolAdapter inst…
[refactor](arrow-flight) Move Arrow Flight SQL out of the service pac…
[refactor](session) Move result encoding behind ResultSender
[refactor](session) Move result encoding behind ResultSender (#67883)
[refactor](session) Replace the execution layer's ConnectType branche…
[feature](arrow-flight) Serve the Flight SQL session actions on the D…
[refactor](arrow-flight) One connection pool for MySQL connections an…
[refactor](arrow-flight) Make the bearer token the credential of exac…
[test](arrow-flight) Pin the Doris-to-Arrow type mapping as it stands…
[refactor](arrow-flight) Extract the Doris-to-Arrow type mapping into…
[test](arrow-flight) Pin the GetTables schema a Flight SQL client see…
[refactor](arrow-flight) Extract the Doris-to-Arrow type mapping into…