Repository navigation
Commit 6004261
[test](protocol) Record a golden baseline of MySQL packets and Arrow Flight results (#67789)
### What problem does this PR solve?
Issue Number: #67577 -- the tracking issue for the protocol-agnostic
session and execution
layer. This is the first PR of its Stage 1 and does not close it.
**In plain terms.** Nothing in this repository checks the bytes the
server sends to a client. Tests
stop at "the statement succeeded and the result had two rows"; how many
packets carried it, in what
order, with which sequence ids, whether the result set ended in an EOF
or an OK, and whether it was
flushed, is checked by nobody. The next PRs move the MySQL and the Arrow
Flight SQL front ends onto
one shared session layer, which must not change a single byte on the
wire -- but if one of them
drops a packet or reorders two, every existing test still passes and the
only symptom is a client
that hangs, weeks later, with no way to tell which refactor did it. So
this PR records what the
server sends today into two checked-in files, before anything moves.
From then on, a change to
those bytes turns a test red and names the exact line and byte that
moved. It touches no production
code -- it photographs the rooms before the renovation starts.
Problem Summary:
Nothing in the tree asserts the bytes a MySQL client actually receives,
or the Arrow batches an
Arrow Flight SQL client actually receives. Every test around the
frontend's result path stops at the
`QueryState` or at a `ShowResultSet`; the packet framing below that --
how many packets, in what
order, with which sequence ids, terminated by an EOF or an OK, flushed
where -- is only exercised by
real clients in the regression suites, and only for the handful of
statements those suites run.
That gap is about to matter. The session and result layers are being
reorganized so that MySQL and
Arrow Flight SQL become two front ends over one session (the follow-up
PRs extract a protocol
adapter and a result sender out of `ConnectContext`, `ConnectProcessor`
and `StmtExecutor`). Those
are pure refactors, and a pure refactor that drops an EOF packet,
reorders a column definition, or
shifts a sequence id passes every existing unit test and shows up only
as a client that hangs.
This PR takes the baseline first, before anything moves. It adds no
production code.
### What is changed?
Four test classes under
`fe/fe-core/src/test/java/org/apache/doris/qe/protocol/` and two golden
files under `fe/fe-core/src/test/resources/protocol-golden/`:
- `RecordingMysqlChannel` -- a `DummyMysqlChannel` that plays canned
request packets and records
every response packet: its payload, the sequence id it was framed with,
and whether the response
was flushed at it. The sequence id is advanced the way the real channel
advances it (once per
packet read, once per packet written), so what is recorded is what would
reach the wire.
- `ProtocolGolden` -- renders the recorded traffic as an annotated
hexdump and compares it with the
checked-in file. On a mismatch it writes the current traffic to
`target/protocol-golden/<name>`
and names it in the failure message, the same convention
`AccessControlBehaviorBaselineTest` uses.
- `MysqlPacketGoldenTest` -- 27 cases: constant selects, a NULL literal,
an empty result set,
`SHOW VARIABLES`, `DESC` (the one case whose row stream is longer than a
single packet), `SET`,
`USE`, `EXPLAIN`, a syntax error, an unknown table, an error followed by
a healthy statement on the
same connection, multi-statement requests with and without
`CLIENT_MULTI_STATEMENTS`, and the
connection commands (`COM_FIELD_LIST`, `COM_STMT_PREPARE`,
`COM_STMT_CLOSE`, `COM_SET_OPTION`,
`COM_RESET_CONNECTION`, `COM_PING`, `COM_INIT_DB`, `COM_STATISTICS`, an
unknown command,
`COM_QUIT`). Two of them run with `CLIENT_DEPRECATE_EOF` negotiated off,
the capability that
decides whether a result set ends in an EOF or an OK and whether the
column definitions get their
own terminator.
- `FlightResultGoldenTest` -- the same statements an Arrow Flight SQL
session can be answered
frontend-side, recorded as the schema and rows of the `VectorSchemaRoot`
the session caches, or as
the error it failed with.
Capability negotiation is replayed rather than mocked: the test derives
the effective capability the
way `MysqlProto.negotiate()` does (`server & client`) and sets
`clientDeprecatedEOF`,
`clientMultiStatements` and the serializer capability from it, so the
recorded packets match what a
real connection with those flags would produce.
Two responses are recorded as shape rather than bytes, because their
payload legitimately moves with
changes that have nothing to do with the protocol: a parser error
carries the entire keyword list of
the grammar (4881 bytes today), and an `EXPLAIN` carries the current
plan text. For those the golden
keeps the packet kinds, plus the error code, SQL state and the start of
the message.
The statement set is limited to what a frontend answers on its own.
`COM_STMT_EXECUTE` is
deliberately absent: `ConnectContext.supportHandleByFe()` is false for
it, so its result always comes
from a backend and belongs in `prepared_stmt_p0`, not here. No case in
either test reaches a backend
(verified: the mocked backend logs no `exec_plan_fragment` request
during the run).
The two golden files are added to `.licenserc.yaml`'s `paths-ignore`:
both tests regenerate the file
in full and compare it byte for byte, so a license header would be read
back as unexpected content.
### Release note
None.
### Check List (For Author)
- Test
- [x] Unit Test
- [ ] Regression test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>1 parent 5985916 commit 6004261
7 files changed
Lines changed: 1266 additions & 0 deletions
File tree
- fe/fe-core/src/test
- java/org/apache/doris/qe/protocol
- resources/protocol-golden
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
67 | 72 | | |
68 | 73 | | |
69 | 74 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
0 commit comments