Repository navigation
Commit 83ebddd
authored
[refactor](session) Bind a ConnectContext to one ProtocolAdapter instead of a Flight subclass (#67835)
### What problem does this PR solve?
Issue Number: #67577 -- the tracking issue for the protocol-agnostic
session and execution
layer. This is the second PR of its Stage 1 (after the golden baseline
#67789) and does not close it.
The shape after this PR:
```
+---------------------------+ +-----------------------------+
| MySQL client | | Arrow Flight SQL client |
+-------------+-------------+ +--------------+--------------+
| |
v v
+---------------------------+ +-----------------------------+
| MysqlServer | | DorisFlightSqlProducer |
| AcceptListener | | every call goes through |
| ReadListener | | adapter.runCommand() |
| (xnio: one command | | (lock: one command |
| at a time) | | at a time) |
+-------------+-------------+ +--------------+--------------+
| |
v v
+---------------------------+ +-----------------------------+
| MysqlConnectProcessor | | FlightSqlConnectProcessor |
+-------------+-------------+ +--------------+--------------+
| |
+----------------------------+-----------------------------+
|
v
+-------------------------------------------------------------------------------------------+
| ConnectContext -- the session, one per connection |
| |
| user, catalog / db, SessionVariable, transaction, prepared statements, |
| queryId / stmtId, executor, audit, ... |
| |
| protocolAdapter : ProtocolAdapter |
| bound once, by forMysql() | forMysqlProxy() | forFlight() | new ConnectContext() |
| |
| getMysqlChannel() getCapability() getFlightSqlChannel() isReturnResultFromLocal() .. |
| same signatures as before, now delegate to protocolAdapter |
+---------------------------------------------+---------------------------------------------+
|
v
+-----------------------------------------------+
| <<interface>> qe.protocol.ProtocolAdapter |
| |
| type() |
| remoteHostPortString(ctx) |
| resultSinkType() |
| connectPool(scheduler) |
| afterStatement(ctx) |
| closeConnection(ctx) |
+-----------------------+-----------------------+
|
+--------------------+----------------------------+
| |
+-----------------------+----------------------+ +-----------------------+----------------------+
| mysql.protocol.MysqlProtocolAdapter | | service.arrowflight.protocol |
| | | .FlightProtocolAdapter |
| MysqlChannel: | | |
| a socket | | peerIdentity (the bearer token) |
| ProxyMysqlChannel (statement forwarded | | FlightSqlChannel (FE-side result cache) |
| to the master) | | endpoints of the last query |
| DummyMysqlChannel (internal context) | | returnResultFromLocal |
| server / negotiated MysqlCapability | | prepared queries |
| handshake packet, MysqlSslContext | | deferred executors + idle bound |
| COM_STMT_EXECUTE packet, cursor flag | | (#62259, #67503) |
| clientConsumesCursorMetadataTerminator() | | per-session command lock: |
| accept-query loop | | runCommand() / callCommand() |
| (start / suspend / resume / stop) | | |
+----------------------------------------------+ +----------------------------------------------+
Not touched by this PR: ConnectProcessor / StmtExecutor / Coordinator keep calling the
ConnectContext getters. The result-encoding half (ResultSender) and the capability
predicates that replace the remaining ConnectType branches come in the next PRs.
```
**In plain terms.** A client session in the frontend is a
`ConnectContext`. Today that one class
holds the state of both wire protocols at the same time: the MySQL
socket, the capabilities
negotiated with the MySQL client, the handshake and SSL state, the
prepared-statement packet being
executed -- and, next to them, the Arrow Flight SQL result cache, the
backend endpoints of the last
Flight query, the prepared queries and the deferred coordinators. Which
half is real is decided by a
subclass, `FlightSqlConnectContext`, that overrides six methods and
leaves every other Flight member
sitting on the base class, where a MySQL connection carries it as dead
weight and a Flight session
throws from the MySQL ones. This PR gives each protocol its own object,
a `ProtocolAdapter`, and
binds a session to exactly one of them when it is created. Nothing a
client sees changes: the golden
byte-for-byte baseline recorded in #67789 is identical before and after.
Problem Summary:
`ConnectContext` mixes three things: the session (user, catalog and
database, session variables,
transaction, prepared statements, the running statement), the MySQL
protocol state, and the Arrow
Flight SQL protocol state. The next steps of #67577 move the result path
of both protocols onto one
shared implementation, which needs a place for "what only this protocol
knows" that is not the
session itself. This PR creates that place and moves the state, without
touching the execution layer
yet: `StmtExecutor`, `ConnectProcessor` and the coordinators still call
the same `ConnectContext`
getters, which now delegate.
### What is changed?
**`qe/protocol/ProtocolAdapter`** -- the wire-protocol half of a
connection: `type()`, the client
address for processlist and the audit log, the result sink type the
backend must use, the pool the
connection is registered in (there is still one per protocol), a
per-statement cleanup hook and
`closeConnection`. `qe/protocol` holds only the interface; each front
end implements it in a
`protocol` subpackage of its own package (`mysql/protocol`,
`service/arrowflight/protocol`), which is
also where the result senders of the next step go.
**`mysql/protocol/MysqlProtocolAdapter`** -- owns the `MysqlChannel` (a
socket, the
`ProxyMysqlChannel` of a forwarded statement on the master, or the
`DummyMysqlChannel` of an
internal context), the server and negotiated capabilities, the handshake
packet, the SSL context,
the `COM_STMT_EXECUTE` packet and its cursor flag, and the xnio
accept-query loop that
`AcceptListener` / `ReadListener` drive. It also owns the decision
`StmtExecutor` and `FEOpExecutor`
used to compute from `ConnectContext` fields -- whether the Connector/J
release on the other end
consumes the metadata terminator of a cursor result (#67520) -- as
`clientConsumesCursorMetadataTerminator`.
**`service/arrowflight/protocol/FlightProtocolAdapter`** -- owns the
peer identity (bearer token), the
`FlightSqlChannel`, the prepared queries, the endpoints of the last
query, `returnResultFromLocal`
and the deferred executors of #62259 / #67503, together with their idle
bound. `ConnectContext`
keeps `checkTimeout` and the idle reaper unchanged; only the list moved.
It also serializes the commands of a session. gRPC runs each call of a
session on whatever thread it
likes and nothing in the Flight transport orders them, while
`ConnectContext` is not thread-safe
(the existing `DorisFlightSqlProducerTest` spells that out).
`runCommand` / `callCommand` take a
per-session lock, make the session the thread's current `ConnectContext`
for the duration, restore
the previous one afterwards, and give up with `UNAVAILABLE` after the
session's query timeout if
another command is still running. `DorisFlightSqlProducer` runs
statement execution, prepared
statement creation and close, DoGet of a frontend-side result and the
catalog / schema / table
metadata requests through it. DoGet of a frontend-side result streams
under the lock on purpose: the
next statement of the session resets the channel, whose removal listener
closes the
`VectorSchemaRoot` being streamed. Session teardown (token expiry,
`CloseSession`, `KILL`) does not
take the lock; that path is reworked when the token becomes the session
credential.
**`ConnectContext`** -- gets `protocolAdapter` and three factories:
`forMysql(StreamConnection)`,
`forMysqlProxy(sessionId)` (replaces the `new ConnectContext(null, true,
sessionId)` call in
`FrontendServiceImpl`) and `forFlight(peerIdentity)` (replaces the
subclass in
`FlightSessionsManager`). The existing constructors stay as thin
wrappers, so the ~115 test files that
call `new ConnectContext()` are untouched. Every protocol-specific
getter keeps its signature and
delegates: `getMysqlChannel()`, `getCapability()`,
`getFlightSqlChannel()`, `isReturnResultFromLocal()`
and so on. A getter that only makes sense on the other protocol throws
`IllegalStateException`
naming the actual protocol (the subclass used to throw a
`RuntimeException` for `getMysqlChannel()`;
the base class used to return `null` / an empty list for the Flight
ones, which no caller relied on).
`FlightSqlConnectContext` is deleted: its `getClientIP` /
`getRemoteHostPortString` /
`closeChannel` / `setQueryId` overrides are the adapter's
`remoteHostPortString` / `closeConnection`
/ `connectPool`, and its `kill` override only differed in log text.
Removed as dead code while touching the class: `isSend` / `setIsSend`
(nothing read them; the real
flag lives on `MysqlChannel`), `cloneContext()` (no caller, and it would
have to share a channel
between two adapters), and the two lines of `resetConnection()` that
cleared Flight-only fields
(`COM_RESET_CONNECTION` is only sent by MySQL clients).
Not in this PR, deliberately: the execution layer still branches on
`ConnectType`, and an internal
context is still a MySQL context over a `DummyMysqlChannel`, exactly as
before. Both go away in the
follow-up PRs that introduce the result sender and the capability bits.
### Verification
- **Golden baseline of #67789**: `MysqlPacketGoldenTest` (27 cases, byte
for byte) and
`FlightResultGoldenTest` pass unchanged. Not a byte of the recorded
traffic moved.
- New unit tests: `FlightProtocolAdapterTest` (commands of one session
run one at a time, a waiting
command fails with `UNAVAILABLE` after the query timeout, the thread's
current context is set and
restored, a failing command releases the session, `KILL` unregisters the
session from the Flight
pool, the trace id lands in the Flight pool) and
`MysqlProtocolAdapterTest` (internal and proxy
contexts, the cursor-terminator decision and its per-statement reset,
the accept-query loop and
close going through the channel).
- Existing tests adjusted to the factories: the ones that built a
`FlightSqlConnectContext`, poked
`mysqlChannel` / `connectType` through reflection, or used a plain `new
ConnectContext()` as a
Flight session (`ShortCircuitPointQueryTest`,
`AuditLogWorkloadGroupTest`, `StmtExecutorTest`,
`ConnectContextTest`, `MysqlProtoTest`, `ConnectionExceedTest`). 26 test
classes around the
session, the MySQL channel and the Flight producer: 166 tests, 0
failures.
- Regression on a local cluster built from this branch:
`arrow_flight_sql_p0` (8 suites, including
the forward-to-master, query-release, point-query, SQL cache and
`DatabaseMetaData.getColumns`
paths) and `prepared_stmt_p0` (cursor fetch and server-side prepare over
MySQL).
- `checkstyle:check` on fe-core (main and test sources): 0 violations.1 parent d1cefa9 commit 83ebddd
29 files changed
Lines changed: 1175 additions & 453 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris
- mysql
- protocol
- qe
- protocol
- service
- arrowflight
- protocol
- sessions
- test/java/org/apache/doris
- mysql
- protocol
- nereids/rules/rewrite
- qe
- protocol
- service/arrowflight
- protocol
- sessions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
25 | 26 | | |
26 | 27 | | |
27 | 28 | | |
| |||
63 | 64 | | |
64 | 65 | | |
65 | 66 | | |
66 | | - | |
| 67 | + | |
67 | 68 | | |
68 | 69 | | |
69 | 70 | | |
| |||
132 | 133 | | |
133 | 134 | | |
134 | 135 | | |
135 | | - | |
| 136 | + | |
136 | 137 | | |
137 | 138 | | |
138 | 139 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
| |||
34 | 35 | | |
35 | 36 | | |
36 | 37 | | |
| 38 | + | |
37 | 39 | | |
38 | 40 | | |
39 | 41 | | |
40 | 42 | | |
| 43 | + | |
41 | 44 | | |
42 | 45 | | |
43 | 46 | | |
| |||
46 | 49 | | |
47 | 50 | | |
48 | 51 | | |
49 | | - | |
| 52 | + | |
50 | 53 | | |
51 | 54 | | |
52 | 55 | | |
53 | 56 | | |
54 | 57 | | |
55 | 58 | | |
56 | 59 | | |
57 | | - | |
| 60 | + | |
58 | 61 | | |
59 | | - | |
| 62 | + | |
60 | 63 | | |
61 | 64 | | |
62 | 65 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
0 commit comments