Repository navigation
Commit 62b26c2
authored
[refactor](session) Replace the execution layer's ConnectType branches with protocol capabilities (#67900)
### What problem does this PR solve?
Issue Number: #67577
Related PR: #67883 (`ResultSender`), #67835 (`ProtocolAdapter`), #67789
(protocol goldens)
Problem Summary:
Third and last refactor step of stage 1 of #67577. After #67835 and
#67883 the connection half and the result-encoding half of the two wire
protocols live behind `ProtocolAdapter` / `ResultSender`, but the
execution layer still asked `ctx.getConnectType()` in a dozen places:
five times in `StmtExecutor`, in `ConnectContext.supportHandleByFe`, in
nine insert / transaction `Command`s (each resetting the MySQL channel),
in both coordinators, in the short-circuit rewrite rule and in
`FEOpExecutor`; and the Flight-only `returnResultFromLocal` flag was
flipped from four places outside the adapter. This PR replaces every one
of them with a capability or a lifecycle hook on the adapter, one method
per use:
```
+---------------------------+ +-----------------------------+
| MySQL client | | Arrow Flight SQL client |
+-------------+-------------+ +--------------+--------------+
| |
v v
+---------------------------+ +-----------------------------+
| MysqlServer | | DorisFlightSqlProducer |
| AcceptListener | | every call goes through |
| ReadListener | | adapter.runCommand() |
+-------------+-------------+ | executeQueryStatement: |
| | adapter.beginRequest() NEW|
v +--------------+--------------+
+---------------------------+ |
| MysqlConnectProcessor | v
| COM_FIELD_LIST | +-----------------------------+
| handleExecute: | | FlightSqlConnectProcessor |
| adapter.beforeStatement NEW | |
| finalizeCommand() = | | |
| adapter.finishCommand() | | |
+-------------+-------------+ +--------------+--------------+
| |
+----------------------------+-----------------------------+
|
v
+-------------------------------------------------------------------------------------------+
| ConnectProcessor.executeQuery -- parse, one StmtExecutor per statement, audit |
| for each statement: adapter.beforeStatement(ctx) <-- NEW |
| executor.execute() |
| adapter.finishStatement(ctx, executor, i, n) |
| proxyExecute (master side): MysqlProtocolAdapter.restoreFromForwardRequest <-- NEW |
+---------------------------------------------+---------------------------------------------+
|
v
+-------------------------------------------------------------------------------------------+
| StmtExecutor -- plans and runs one statement, protocol-agnostic |
| |
| forwarding a query to the master: adapter.canReplayForwardedQueryResult() <-- NEW |
| retry after a failed attempt: adapter.canRetryQuery(ctx) <-- NEW |
| execute(queryId), every attempt: adapter.beforeAttempt(ctx) <-- NEW |
| executeAndSendResult: adapter.beforeQuery(ctx), then the coordinator; |
| relay rows unless !ctx.isReturnResultFromLocal() |
| FEOpExecutor.buildStmtForwardParams: adapter.fillForwardRequest(ctx, request) <-- NEW |
| |
| Commands (insert / txn): no channel reset any more, beforeStatement did it |
| Coordinator / NereidsCoordinator / QueryProcessor: ctx.isReturnResultFromLocal() |
| decides receivers vs. Flight endpoints, no ConnectType assertion |
| LogicalResultSinkToShortCircuitPointQuery: adapter.supportsShortCircuitPointQuery() NEW|
| no ConnectType branch and no getMysqlChannel() left in qe/** and nereids/** |
+---------------------------------------------+---------------------------------------------+
|
v
+-------------------------------------------------------------------------------------------+
| ConnectContext -- the session, one per connection |
| protocolAdapter : ProtocolAdapter getResultSender() = adapter.resultSender(this)|
| supportHandleByFe() = adapter.supportsFeSideResult() && command != COM_STMT_EXECUTE |
| isReturnResultFromLocal() = adapter.returnsResultFromLocal(this) (setter is gone) |
+---------------------------------------------+---------------------------------------------+
|
v
+-----------------------------------------------+
| <<interface>> qe.protocol.ProtocolAdapter |
| type() remoteHostPortString(ctx) |
| resultSinkType() resultSender(ctx) |
| connectPool(scheduler) closeConnection(ctx) |
| |
| capabilities: |
| supportsSqlCacheReplay() |
| canReplayForwardedQueryResult() <-- NEW |
| supportsFeSideResult() <-- NEW |
| supportsShortCircuitPointQuery() <-- NEW |
| canRetryQuery(ctx) <-- NEW |
| |
| statement lifecycle: |
| beforeStatement(ctx) <-- NEW |
| beforeAttempt(ctx) <-- NEW |
| beforeQuery(ctx) <-- NEW |
| returnsResultFromLocal(ctx) <-- NEW |
| finishStatement(ctx, executor, i, n) |
| afterStatement(ctx) |
| |
| forwarding: |
| fillForwardRequest(ctx, request) <-- NEW |
+-----------------------+-----------------------+
|
+--------------------+----------------------------+
| |
+-----------------------+----------------------+ +-----------------------+----------------------+
| mysql.protocol.MysqlProtocolAdapter | | arrowflight.protocol.FlightProtocolAdapter |
| capabilities: all true; | | capabilities: all false (the reasons are |
| canRetryQuery = nothing flushed yet | | documented on each method) |
| beforeStatement: channel.reset() -- drops | | beforeStatement: the statement's result is |
| what the previous statement of the | | on this frontend until beforeQuery says |
| request left unsent | | a backend produces it; the coordinator |
| beforeAttempt: nothing | | beforeAttempt: result on this frontend, |
| beforeQuery: nothing, rows are relayed | | endpoints of the failed attempt dropped |
| | | then registers the endpoints |
| returnsResultFromLocal: true | | returnsResultFromLocal: that flag |
| fillForwardRequest: capability flags, | | fillForwardRequest: nothing |
| deprecate-EOF, execute packet + cursor | | beginRequest(): deferred executors, result |
| restoreFromForwardRequest (master side) | | cache, endpoints, flag (Flight-private) |
| proxyResultPackets() (master side) | | |
+-----------------------+----------------------+ +-----------------------+----------------------+
| |
v v
+----------------------------------------------+ +----------------------------------------------+
| <<interface>> qe.protocol.ResultSender | | |
| sendResultSet(rs, fieldInfos, binaryRows) | | |
| sendFields(names, fieldInfos, types) | | |
| sendRow(wireRow) | | |
| reset() -- now only for a retried query | | |
+----------------------------------------------+ +----------------------------------------------+
| mysql.protocol.MysqlResultSender (unchanged) | | arrowflight.protocol.FlightResultSender |
| | | sendResultSet no longer touches the |
| | | result-location flag (EXPLAIN never marks |
| | | the result as coming from a backend now) |
+----------------------------------------------+ +----------------------------------------------+
```
**Capabilities.** `canReplayForwardedQueryResult()` guards the refusal
to forward a query to the master on a Flight session (#67569; the
message is unchanged). `supportsFeSideResult()` is the protocol half of
`ConnectContext.supportHandleByFe()`; `supportsShortCircuitPointQuery()`
is the protocol half of
`LogicalResultSinkToShortCircuitPointQuery.scanMatchShortCircuitCondition`
(#67368); `canRetryQuery(ctx)` is the retry condition of
`handleQueryWithRetry`: for MySQL "nothing was flushed to the socket
yet", for Flight false (the client is not told which attempt its
endpoints belong to). The reason each Flight answer is what it is moves
onto the Flight implementation, out of the call sites.
**Statement lifecycle.** `beforeStatement(ctx)` is called by
`ConnectProcessor.executeQuery` before each statement (and by
`MysqlConnectProcessor.handleExecute`): the MySQL adapter resets the
channel there, which is what the query path and the nine `Command`s did
each on their own; the Flight adapter puts the statement's result on
this frontend. `beforeQuery(ctx)` is called at the top of
`executeAndSendResult`, before a coordinator is built: the Flight
adapter marks the result as staying on the backends, and `Coordinator` /
`NereidsCoordinator` / `QueryProcessor` keep reading
`ctx.isReturnResultFromLocal()` to register endpoints instead of
receivers (the `checkState(ARROW_FLIGHT_SQL)` assertions go, the
decision itself does not move). Because `beforeQuery` runs only where a
coordinator follows, an `EXPLAIN` -- handled earlier in
`handleQueryStmt` -- never marks its result as coming from a backend,
and the `setReturnResultFromLocal(true)` that
`FlightResultSender.sendResultSet` had to do in #67883 is gone with the
setter. The four flips in `StmtExecutor` / `FlightSqlConnectProcessor`
are gone; what a Flight request drops from its predecessor (deferred
executors, result cache, endpoints, the flag) is
`FlightProtocolAdapter.beginRequest()`.
`beforeAttempt(ctx)` is called at the top of
`StmtExecutor.execute(TUniqueId)`, i.e. for the first attempt and for
each replan retry of `queryRetry` (a cloud `NEED_REPLAN` error); it
replaces the `setReturnResultFromLocal(true)` that stood there before
this PR, which the first version of this PR had folded into
`beforeStatement` -- once per statement, outside the replan loop. A
Flight attempt that had reached `beforeQuery` and then failed with a
replan error left the next attempt in the "result on the backends"
state; if that attempt failed before its own `beforeQuery`,
`StatementContext.close` skipped the connector statement scope and the
`finalizeQuery` in `handleQueryWithRetry` was skipped, with no deferred
executor or DoGet ever releasing them. The Flight `beforeAttempt` puts
the result back on this frontend and withdraws exactly the endpoints the
failed attempt registered (the adapter records how many the request held
when the statement started and trims back to that mark, so an earlier
statement of the request keeps its own), which `queryRetry` -- unlike
`handleQueryWithRetry`, which asks `canRetryQuery` -- did not withdraw
before, so a replanned Flight query no longer hands the client the
endpoints of a cancelled attempt next to its own. Covered by
`FlightProtocolAdapterTest` (the attempt lifecycle) and
`ProtocolCapabilityWiringTest.testEveryAttemptOfAReplannedFlightQueryStartsOnTheFrontend`,
which runs a real Flight query whose first attempt fails on the mocked
backend with an `E-230` status after registering its endpoint and whose
second attempt fails while planned: the session ends with its result on
the frontend and no endpoint (the test fails on the flag without the
hook).
**Forwarding.** `fillForwardRequest(ctx, request)` adds to a
`TMasterOpRequest` what the master needs to know about the client: the
MySQL adapter writes the negotiated capability flags,
`CLIENT_DEPRECATE_EOF` and, for a `COM_STMT_EXECUTE`, the execute packet
and the cursor flag (formerly two blocks in
`FEOpExecutor.buildStmtForwardParams`, one of them behind `if (MYSQL)`);
the Flight adapter writes nothing, its session consumes the master's
status and rows rather than its packets (`carryForwardedOutcome`). The
master's side, `ConnectProcessor.restoreForwardedMysqlContext`, becomes
`MysqlProtocolAdapter.restoreFromForwardRequest`, and `proxyExecute`
reads the proxy channel's packets through
`MysqlProtocolAdapter.proxyResultPackets()` instead of
`StmtExecutor.getProxyQueryResultBufList()` casting the channel.
After this PR `grep -rn 'ConnectType\.\|getMysqlChannel()'
fe-core/src/main/java/org/apache/doris/{qe,nereids}` outside
`*/protocol/` finds only the `ConnectContext.getMysqlChannel()` delegate
itself and two lines of `MysqlConnectProcessor` (reading the client's
packet, the auth-switch handshake), which is MySQL protocol code by
definition.
**One behavior change, on the MySQL side, recorded in the golden.** The
channel used to be reset at the start of a *query* and inside the insert
/ transaction commands, and nowhere else. A client that did not
negotiate `CLIENT_MULTI_STATEMENTS` gets no intermediate response
between the statements of a request, so whatever a query wrote stayed in
the send buffer until the next query or insert reset it. When the next
statement was neither -- `select 1; set @A = 1` -- the buffered result
set of the `SELECT` went out together with the `OK` of the `SET`: a
result set terminated by a `0x00` OK packet, which no MySQL client
parses (it reads the OK as a row and waits for more). With the reset at
the start of every statement, such a request delivers only its last
statement's outcome, which is what
`MysqlProtocolAdapter.finishStatement` has documented as the intent all
along. The first commit makes `RecordingMysqlChannel` model the send
buffer (a reset drops what was written after the last flush, the way
`MysqlChannel.reset()` clears it) and records `select 1; set @A = 1`
with and without the capability as it is today; the second commit's
golden diff is exactly that: the three packets of `select 1` disappear
from `multi-statement-without-capability-query-then-set`, the `OK` keeps
its sequence id 4. The existing `multi-statement-without-capability`
case (`select 1; select 2`) loses the three packets of `select 1` in the
first commit only, because the recording channel now shows what reaches
the client -- its sequence ids, 4 to 7, already were the ones on the
wire. That the delivered response did not start at sequence id 1 was
pre-existing -- `MysqlChannel.reset()` cleared the buffer but did not
rewind the sequence id, so a client that checks sequence ids (pymysql,
libmysqlclient; Connector/J does not by default) already failed `select
1; select 2` without the capability with "Packet sequence number wrong -
got 5 expected 1" -- but the reset before every statement widened it: a
request without the capability whose later statement failed (`select 1;
select * from no_such_table`) used to deliver the buffered result set of
`select 1` followed by the ERR, and now delivered a lone ERR with that
hole, which pymysql rejects and go-sql-driver answers by closing the
connection. So it is fixed here: `MysqlChannel` tracks the sequence id
as of the last packet that reached the wire (the command packet
received, the last flush) and `reset()` rewinds to it,
`RecordingMysqlChannel` models the same, and the golden shows `select 1;
select 2` and `select 1; set @A = 1` numbered from 1 and gains `select
1; select * from no_such_table` with and without the capability (ERR at
5 after the intermediate response, ERR at 1 without one). The same reset
also fixes the retry of a forwarded query on the master:
`ProxyMysqlChannel` never flushes, so `canRetryQuery` held, but its
collected packets survived the reset and both attempts' packets
travelled to the client; it now drops them on reset.
Not in this PR: an internal adapter for the no-client context (it is
still a MySQL context over a `DummyMysqlChannel`, now with nothing in
the execution layer keyed on that), and the stage 1 performance
baseline.1 parent 341a312 commit 62b26c2
41 files changed
Lines changed: 1823 additions & 198 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris
- arrowflight
- protocol
- mysql
- protocol
- nereids
- rules/rewrite
- trees/plans/commands
- insert
- qe
- protocol
- test
- java/org/apache/doris
- arrowflight
- protocol
- mysql/protocol
- qe
- protocol
- utframe
- resources/protocol-golden
- regression-test/suites
- arrow_flight_sql_p0
- query_p0
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
197 | 197 | | |
198 | 198 | | |
199 | 199 | | |
200 | | - | |
201 | | - | |
202 | | - | |
203 | | - | |
204 | | - | |
205 | | - | |
206 | | - | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
207 | 205 | | |
208 | 206 | | |
209 | 207 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
67 | | - | |
68 | 67 | | |
69 | 68 | | |
70 | 69 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
| |||
66 | 67 | | |
67 | 68 | | |
68 | 69 | | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
69 | 78 | | |
70 | 79 | | |
71 | 80 | | |
| |||
129 | 138 | | |
130 | 139 | | |
131 | 140 | | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
132 | 235 | | |
133 | 236 | | |
134 | 237 | | |
| |||
232 | 335 | | |
233 | 336 | | |
234 | 337 | | |
235 | | - | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
236 | 348 | | |
237 | | - | |
238 | | - | |
239 | | - | |
240 | | - | |
241 | | - | |
242 | | - | |
243 | | - | |
244 | | - | |
| 349 | + | |
| 350 | + | |
245 | 351 | | |
246 | 352 | | |
247 | 353 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | 52 | | |
57 | 53 | | |
58 | 54 | | |
| |||
70 | 66 | | |
71 | 67 | | |
72 | 68 | | |
73 | | - | |
| 69 | + | |
74 | 70 | | |
75 | 71 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | 31 | | |
36 | 32 | | |
37 | 33 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
60 | 65 | | |
61 | 66 | | |
62 | 67 | | |
| |||
145 | 150 | | |
146 | 151 | | |
147 | 152 | | |
| 153 | + | |
148 | 154 | | |
149 | 155 | | |
150 | 156 | | |
| |||
429 | 435 | | |
430 | 436 | | |
431 | 437 | | |
| 438 | + | |
432 | 439 | | |
433 | 440 | | |
434 | 441 | | |
| |||
504 | 511 | | |
505 | 512 | | |
506 | 513 | | |
| 514 | + | |
507 | 515 | | |
508 | 516 | | |
509 | 517 | | |
| |||
556 | 564 | | |
557 | 565 | | |
558 | 566 | | |
| 567 | + | |
559 | 568 | | |
560 | 569 | | |
561 | 570 | | |
| |||
566 | 575 | | |
567 | 576 | | |
568 | 577 | | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
569 | 587 | | |
570 | 588 | | |
571 | 589 | | |
| |||
574 | 592 | | |
575 | 593 | | |
576 | 594 | | |
577 | | - | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
578 | 598 | | |
579 | 599 | | |
580 | 600 | | |
581 | 601 | | |
582 | 602 | | |
| 603 | + | |
583 | 604 | | |
584 | 605 | | |
585 | 606 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
37 | 49 | | |
38 | 50 | | |
39 | 51 | | |
| |||
0 commit comments