Skip to content

[fix](fe) Fix prepared statement streaming, forwarding and cursor framing - #69314

Merged
CalvinKirs merged 4 commits into
apache:masterfrom
CalvinKirs:mysql-protocol
Oct 20, 2026
Merged

CalvinKirs merged 4 commits into
apache:masterfrom
CalvinKirs:mysql-protocol

Conversation

@CalvinKirs

@CalvinKirs CalvinKirs commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

What problem does this PR solve?

Problem Summary:

  • COM_STMT_SEND_LONG_DATA incorrectly returns an error packet, leaving streaming clients out of sync. Accept parameter chunks without replying, defer upload errors until EXECUTE, preserve binary bytes, and clear upload state on EXECUTE/RESET/CLOSE.
  • Reusing server-side prepared statements through a non-master FE can fail with NoSuchElementException: No value present: the driver omits unchanged parameter types, while the master prepares each forwarded execution independently. Include cached types, the NULL bitmap and complete values in forwarded EXECUTE payloads.
  • Forwarded PREPARE can use a stale statement context when planning placeholders. Use the parser's current prepare-stage context.
  • Empty results with useCursorFetch=true can hang or leave unread packets when client connection attributes are disabled or rewritten. Return a valid empty cursor, complete FETCH with LAST_ROW_SENT, retain cursor state on the client-facing FE, and preserve existing framing for older FEs. Nonempty results continue to use buffered delivery; nonempty cursor pagination is not added.
  • Normalized streamed values can remain attached to idle pooled connections after statement close/reset. Release the command-only execution buffer after response finalization and forwarding complete.

Release note

  • Fix streamed parameters, repeated prepared-statement forwarding and empty cursor results in the MySQL protocol, and reduce retained parameter-buffer memory on idle connections.

Check List (For Author)

  • Test:
    • Unit Test: 32 focused long-data, protocol adapter and forwarding tests passed on the final source, including the new buffer-lifecycle test.
    • Regression test: prepared_stmt_p0/send_long_data and prepared_stmt_p0/cursor_fetch_empty_result passed during implementation.
    • Manual test: raw-protocol upload/cursor lifecycle and connection reuse; rewritten client-attribute proxy reproduction passed after the fix.
    • Checkstyle: zero violations in the latest modified Java files.
  • Behavior changed:
    • Yes. Long-data uploads no longer produce responses; forwarded parameters are self-contained; empty cursors finish through FETCH; completed commands release execution buffers.
  • Does this need documentation?
    • No.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

### What problem does this PR solve?

Issue Number: N/A

Problem Summary: Streaming parameters with server-side prepared statements send
COM_STMT_SEND_LONG_DATA before COM_STMT_EXECUTE. The command previously fell
through to the unsupported-command handler, emitting an unread ERR packet and
leaving subsequent responses out of sync. Uploaded parameter values were also
not retained or bound during execution.

Handle uploads without a response, accumulate chunks per statement and parameter,
and defer upload errors until execution. Bind uploaded values without consuming
inline parameter bytes, preserve BLOB bytes as VARBINARY, enforce the parameter
max_allowed_packet limit, and clear upload state after execute or reset. Inline
uploaded values and cached types in the existing forwarding buffer so the master
can bind them without connection-local upload state. Use the parser's current
prepare context when planning a forwarded PREPARE.

### Release note

Fix streaming parameters in server-side prepared statements and prevent extra
response packets from desynchronizing MySQL connections.

### Check List (For Author)

- Test: Unit Test / Regression test / Manual test
    - FE build and Checkstyle via build.sh --fe
    - run-fe-ut.sh --run: MysqlConnectProcessorLongDataTest, MysqlConnectProcessorCursorFetchTest, ConnectContextTest
    - run-regression-test.sh --run -d prepared_stmt_p0 -s send_long_data; generate expected output with -genOut
    - Raw protocol silence, deferred errors, binary chunks, empty values, reset, close, and connection reuse
    - MariaDB C API mysql_stmt_send_long_data and Connector/J 8.0.28 observer-to-master forwarding with BE 4.1.3
- Behavior changed: Yes, accept streamed parameters without replying to upload commands
- Does this need documentation: No
…atements

### What problem does this PR solve?

Issue Number: N/A

Related PR: apache#48689

Problem Summary: Connector/J can omit unchanged parameter types when reusing a
server-side prepared statement. A non-master FE retains those types locally but
forwards the original EXECUTE buffer. The master prepares the SQL again and has
no cached types, so repeated execution and rewritten cached INSERT batches fail
with NoSuchElementException: No value present. Direct master connections work.

Extend the existing long-data forwarding buffer normalization to executions that
omit parameter types. Include the cached type codes, NULL bitmap and original
value bytes so each forwarded execution can be decoded independently. Test fresh
untyped proxy placeholders, unsigned types, NULL transitions and type updates.

### Release note

Fix repeated server-side prepared statements and cached batch INSERTs on non-master FEs.

### Check List (For Author)

- Test: Unit Test / Manual test
    - build.sh --fe and Checkstyle passed
    - run-fe-ut.sh: 48 tests passed, including 12 long-data/type-forwarding tests
    - Original Connector/J 8.0.27 reproducer passed on master and observer
    - QA Connector/J 8.0.27/8.0.33/8.4.0/9.7.0: 76 passed, 12 existing TLS cases
      skipped because enable_ssl=false; cached INSERT batches, INSERT SELECT,
      stored values and long-data forwarding passed on every driver
    - BE 4.1.3-rc02 was used with enable_local_shuffle_planner=false during
      stored-data validation because it lacks the new LOCAL_EXCHANGE_NODE;
      the original setting was restored afterward
- Behavior changed: Yes; forwarded executions include cached parameter types
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: N/A

Related PR: apache#67520

Problem Summary: With useCursorFetch and a positive fetch size, older Connector/J
releases consume an empty result's sole end marker while probing for a cursor and
wait for another packet. Sending an extra marker instead causes newer drivers to
return an empty result prematurely and leaves unread packets on the connection,
particularly when connection attributes are disabled or rewritten by a proxy.

Return a valid empty cursor for empty results with CLIENT_DEPRECATE_EOF and
complete COM_STMT_FETCH with LAST_ROW_SENT. Track cursor state per prepared
statement, clear it on execution/reset/close, and preserve it on the client-facing
FE after forwarding. Nonempty results retain immediate result delivery. Keep the
previous framing for requests from older FEs that cannot complete FETCH, using
an optional request capability checked against maintained Thrift branches.

### Release note

Fix queries hanging or desynchronizing JDBC connections when useCursorFetch is
enabled, including empty results and clients with disabled connection attributes.
Nonempty result pagination remains unsupported.

### Check List (For Author)

- Test: Unit Test / Regression test / Manual test
    - build.sh --fe passed with zero Checkstyle violations
    - run-fe-ut.sh: 66 relevant tests passed; after the compatibility adjustment,
      30 adapter/sender/forwarding tests passed
    - run-regression-test.sh: cursor_fetch_empty_result passed
    - QA Connector/J 8.0.27/8.0.33/8.4.0/9.7.0: 92 passed, zero failed;
      12 TLS cases skipped because enable_ssl=false
    - Proxy-rewritten Connector/J 8.0.27 reproduced the hang before the fix
      and passed afterward; anonymous Connector/J 9.7.0 and raw cursor lifecycle passed
    - BE 4.1.3-rc02 used be_exec_version=10 and
      enable_local_shuffle_planner=false; an undersized 1 GB BE memory limit
      caused initial test failures, and the successful rerun used 2 GB
- Behavior changed: Yes; empty cursor requests return CURSOR_EXISTS and complete on COM_STMT_FETCH
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: N/A

Related PR: apache#67520

Problem Summary: Normalizing COM_STMT_SEND_LONG_DATA parameters creates a full
EXECUTE payload on the connection adapter. Clearing uploads or closing/resetting
the statement does not clear that independent copy, so an idle pooled connection
can retain the entire streamed value until another EXECUTE or disconnect.

Clear the command-only execution buffer in afterStatement, which runs after
response finalization and synchronous forwarding. Add a test through real upload
and EXECUTE dispatch that verifies complete normalized bytes remain available
during execution and are released after the response.

### Release note

Reduce FE heap retention on pooled connections after executing statements with
streamed parameters.

### Check List (For Author)

- Test: Unit Test
    - run-fe-ut.sh: 32 long-data, protocol adapter and forwarding tests passed
    - Checkstyle: both modified Java files checked, zero violations
- Behavior changed: Yes; command-only parameter buffers are released after response completion
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@CalvinKirs

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 92.37% (121/131) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 28986 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3fedd6185eefae5bf968733b8bd20c7954838073, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17592	5149	5062	5062
q2	2151	322	274	274
q3	10184	1395	819	819
q4	4678	465	334	334
q5	7461	806	537	537
q6	182	190	156	156
q7	764	793	614	614
q8	9259	1374	1125	1125
q9	5597	4242	4227	4227
q10	6830	1303	1054	1054
q11	432	247	231	231
q12	631	396	283	283
q13	18110	3685	3058	3058
q14	280	265	247	247
q15	q16	742	719	696	696
q17	1228	756	592	592
q18	7135	6203	6119	6119
q19	1123	962	627	627
q20	393	336	226	226
q21	5542	2433	2460	2433
q22	403	344	272	272
Total cold run time: 100717 ms
Total hot run time: 28986 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5961	5909	6114	5909
q2	736	593	584	584
q3	5674	5724	5256	5256
q4	2060	2222	1575	1575
q5	5838	5626	5836	5626
q6	243	188	146	146
q7	2234	2029	1741	1741
q8	3087	2699	2649	2649
q9	7868	7916	7835	7835
q10	3657	3594	3140	3140
q11	584	414	385	385
q12	682	715	527	527
q13	3292	3551	3090	3090
q14	303	283	265	265
q15	q16	672	691	624	624
q17	7707	7054	6850	6850
q18	12920	12169	12786	12169
q19	935	811	804	804
q20	2155	2147	1914	1914
q21	5794	4758	4863	4758
q22	487	431	401	401
Total cold run time: 72889 ms
Total hot run time: 66248 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 150143 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3fedd6185eefae5bf968733b8bd20c7954838073, data reload: false

query5	4314	573	438	438
query6	425	207	224	207
query7	4822	455	222	222
query8	314	166	149	149
query9	8759	3085	3146	3085
query10	489	317	258	258
query11	6067	3881	3562	3562
query12	172	94	87	87
query13	1263	432	330	330
query14	6150	3914	3613	3613
query14_1	3397	3419	3377	3377
query15	209	201	176	176
query16	966	443	364	364
query17	876	682	539	539
query18	2446	440	311	311
query19	209	161	138	138
query20	88	86	93	86
query21	213	126	106	106
query22	12925	13029	12841	12841
query23	13530	12789	12367	12367
query23_1	12494	12434	12447	12434
query24	7369	825	452	452
query24_1	459	466	453	453
query25	508	398	333	333
query26	1266	252	131	131
query27	2787	436	268	268
query28	4620	1946	1952	1946
query29	1546	578	431	431
query30	294	218	183	183
query31	997	924	818	818
query32	147	85	88	85
query33	516	298	233	233
query34	905	890	491	491
query35	728	743	634	634
query36	827	845	731	731
query37	124	95	82	82
query38	1785	1726	1667	1667
query39	793	797	741	741
query39_1	733	729	745	729
query40	210	111	99	99
query41	65	63	65	63
query42	94	84	83	83
query43	335	363	313	313
query44	1315	681	677	677
query45	173	170	159	159
query46	825	953	547	547
query47	2956	2615	2660	2615
query48	286	308	216	216
query49	578	402	288	288
query50	675	268	205	205
query51	10331	10318	10385	10318
query52	77	79	71	71
query53	187	209	159	159
query54	234	216	181	181
query55	76	70	65	65
query56	242	200	207	200
query57	1685	1627	1560	1560
query58	293	276	260	260
query59	2118	2126	1906	1906
query60	381	253	237	237
query61	165	169	162	162
query62	391	333	293	293
query63	193	161	168	161
query64	2875	1048	891	891
query65	3319	3301	3252	3252
query66	1792	440	329	329
query67	20086	20394	19991	19991
query68	3327	999	598	598
query69	446	276	249	249
query70	838	764	751	751
query71	307	231	208	208
query72	2799	2591	2328	2328
query73	518	521	287	287
query74	4645	4516	4337	4337
query75	2353	2288	1916	1916
query76	2426	998	588	588
query77	338	361	263	263
query78	9716	9775	9214	9214
query79	962	900	509	509
query80	1442	429	346	346
query81	609	314	270	270
query82	640	130	103	103
query83	276	193	176	176
query84	311	122	106	106
query85	1202	432	374	374
query86	400	237	230	230
query87	2024	1966	1804	1804
query88	3712	2639	2637	2637
query89	332	280	243	243
query90	1928	166	163	163
query91	164	150	121	121
query92	94	90	89	89
query93	1064	988	568	568
query94	625	314	279	279
query95	571	394	299	299
query96	666	477	229	229
query97	2445	2441	2323	2323
query98	163	152	150	150
query99	779	760	646	646
Total cold run time: 236339 ms
Total hot run time: 150143 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 24.35 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3fedd6185eefae5bf968733b8bd20c7954838073, data reload: false

query1	0.01	0.01	0.01
query2	0.10	0.05	0.05
query3	0.27	0.09	0.09
query4	1.60	0.10	0.10
query5	0.25	0.24	0.23
query6	1.16	0.69	0.66
query7	0.05	0.01	0.01
query8	0.06	0.04	0.04
query9	0.41	0.32	0.34
query10	0.59	0.56	0.59
query11	0.21	0.14	0.15
query12	0.18	0.15	0.14
query13	0.49	0.48	0.49
query14	1.02	1.04	1.02
query15	0.63	0.61	0.61
query16	0.34	0.35	0.36
query17	1.12	1.17	1.14
query18	0.20	0.19	0.19
query19	2.11	1.98	1.95
query20	0.02	0.01	0.01
query21	15.49	0.19	0.13
query22	4.99	0.06	0.06
query23	16.21	0.32	0.13
query24	2.82	0.47	0.60
query25	0.14	0.10	0.06
query26	1.10	0.21	0.15
query27	0.05	0.03	0.03
query28	3.92	0.80	0.39
query29	12.54	4.11	3.23
query30	0.29	0.13	0.11
query31	2.78	0.55	0.32
query32	3.23	0.60	0.50
query33	3.18	3.23	3.20
query34	15.61	4.01	3.37
query35	3.32	3.35	3.33
query36	0.54	0.43	0.47
query37	0.10	0.07	0.06
query38	0.06	0.04	0.03
query39	0.04	0.03	0.03
query40	0.18	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 97.57 s
Total hot run time: 24.35 s

@CalvinKirs

Copy link
Copy Markdown
Member Author

Local pipeline review — ✅ PASS

schema: doris-repo-review/v1
status: PASS
pr: apache/doris#69314
commit: 3fedd6185eefae5bf968733b8bd20c7954838073
base: 01e735b29c8b8803b9e2df10b46b9b02db1e6251
reviewed_at: 2026-10-20T14:22+08:00
reviewer: CalvinKirs
model: gpt-6.1-sol
effort: xhigh
findings: {blocker: 0, major: 0, minor: 0, nit: 0}
rounds: 1
converged: true

Notes for maintainers

  • Source-only review covered all 15 changed files. No builds or tests were run during the review; the existing post-fix 32-test FE run and Checkstyle logs were inspected as supporting evidence.

Reviewed locally with the doris-repo-review pipeline (a local port of .github/workflows/code-review-runner.yml). This is not a CI status check.

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 66.40% (31761/47834)
Line Coverage 51.15% (334445/653813)
Region Coverage 46.51% (270346/581233)
Branch Coverage 48.24% (127385/264049)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 77.21% (35786/46351)
Line Coverage 62.45% (403926/646821)
Region Coverage 59.03% (341858/579139)
Branch Coverage 59.86% (157685/263408)

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 58.13% (93/160) 🎉
Increment coverage report
Complete coverage report

@CalvinKirs
CalvinKirs merged commit 6fafa31 into apache:master Oct 20, 2026
46 checks passed
@CalvinKirs
CalvinKirs deleted the mysql-protocol branch October 20, 2026 07:32
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Oct 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

yiguolei pushed a commit that referenced this pull request Oct 24, 2026
…ranch-4.2 (#69463)

### What problem does this PR solve?

Related PR: #69314

Problem Summary:

- `COM_STMT_SEND_LONG_DATA` returns an unexpected response and does not
bind uploaded chunks, leaving streaming clients out of sync. Accept
chunks without replying, defer upload errors until EXECUTE, and clear
uploaded data after execution or RESET.
- Reusing cached server-prepared statements through a non-master FE can
fail because the master prepares each forwarded execution independently.
Forward complete parameter types and values, including streamed
parameters, and use the active prepare-stage context during proxy
planning.
- Empty results with `useCursorFetch=true` can hang or leave unread
packets, including when connection attributes are absent or rewritten.
Return an empty cursor, complete FETCH with `LAST_ROW_SENT`, and
preserve framing for older FEs through an optional peer capability.
- Completed EXECUTE payloads can remain attached to idle connections.
Release command payload buffers after response finalization and
forwarding finish.

Backport adaptations for branch-4.2:

- Keep protocol state in the existing
`ConnectContext`/`ConnectProcessor` architecture and preserve
branch-4.2's expansion of cached parameter types only when forwarding.
- Regenerate regression expectations with the standard test script to
retain branch-4.2's existing VARBINARY function semantics.

### Release note

Fix streamed prepared parameters, repeated prepared execution through
non-master FEs, and empty cursor fetch hangs.

### Check List (For Author)

- Test
- [x] Regression test: `prepared_stmt_p0/send_long_data` and
`prepared_stmt_p0/cursor_fetch_empty_result` passed with
`run-regression-test.sh`.
- [x] Unit Test: 91 FE tests passed with `run-fe-ut.sh`, with zero
failures, errors or skips.
- [x] Manual test: Connector/J 8.0.27, 8.0.33, 8.4.0 and 9.7.0 each
passed 428 checks against the built branch-4.2 FE and BE 4.1.3. Connect
explicitly to master and follower endpoints; verify streamed bytes,
empty streams and statement reuse; run 50 batches of three rows per
version and compare all stored values; alternate empty/nonempty cursor
results with and without connection attributes, including forwarded
execution and connection reuse.
- [x] Full FE build passed via `build.sh --fe`; Checkstyle passed with
zero violations.

- Behavior changed:
- [x] Yes. Long-data uploads have no response; forwarded parameters are
self-contained; empty cursors complete through FETCH; completed command
buffers are released.

- Does this need documentation?
    - [x] No.

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
CalvinKirs added a commit that referenced this pull request Nov 3, 2026
### What problem does this PR solve?

Related PR: #69314

JDBC `setBytes()` and Connector/ODBC binary bindings can send
`MYSQL_TYPE_VAR_STRING` / `MYSQL_TYPE_STRING` rather than BLOB. FE
decoded those bytes with Java's replacement UTF-8 decoder, silently
changing arbitrary binary data. For example, `0080FF4100C328` became
`00EFBFBDEFBFBD4100EFBFBD28` in `SELECT HEX(?)`. The BLOB handling added
by #69314 did not cover these wire types.

Decode string parameters strictly and retain non-UTF-8 payloads as
binary literals. Bind those payloads through a BE-side cast to the
original string category, preserving string function and comparison
typing without replacing bytes. Valid UTF-8 retains the existing text
literal types, and BLOB types retain their existing binary behavior. Use
value-based binary literal child hashes so equivalent casts remain equal
after rebinding and forwarding.

### Release note

Preserve non-UTF-8 bytes in MySQL server-prepared string parameters,
including JDBC `setBytes()` and Connector/ODBC binary bindings.

### Check List (For Author)

- Test:
- [x] Regression test: generated and replayed
`prepared_stmt_p0/binary_parameters`; replayed
`prepared_stmt_p0/send_long_data`. Covers all byte values, malformed
UTF-8, parameter reuse, stored HEX/MD5, string expression predicates,
NULL/empty values and ordinary Unicode text.
- [x] Unit Test: 17 tests passed across `MysqlParameterLiteralTest` and
`MysqlConnectProcessorLongDataTest`, including string wire aliases,
cached types, uploads and forwarding.
- [x] Manual test: Connector/J 8.2.0 server-prepared `setBytes`,
`setBinaryStream` and `setBlob` preserve `0080FF4100C328`.
- [x] Manual test: Linux Connector/ODBC 5.3.14 Unicode, 17 cases passed,
including 1-MiB binary/Unicode inputs. An additional 17-case
proxy-rewritten VARCHAR wire-type probe also passed.
    - [x] FE build and Checkstyle passed.
- Behavior changed:
- [x] Yes. Non-UTF-8 string-coded parameters preserve the input bytes
instead of substituting U+FFFD. Valid UTF-8 text behavior is unchanged.
- Does this need documentation?
    - [x] No.

Runtime regression checks used the newly built master FE with an
existing 4.1.4 BE and condition cache disabled. The predicate regression
uses `CONCAT(value, '') = ?`; direct constant predicate pushdown on a
current master BE was not validated. No BE, storage-format or
protocol-field changes are included.
morningman added a commit that referenced this pull request Nov 5, 2026
…meter types (#70150)

### What problem does this PR solve?

Issue Number: None

Related PR: #68162, #69314

Problem Summary:

**In short.** This ports `MysqlPreparedStatementForwardingTest` from
branch-4.1 #68162 to master. Master already has the fix the test guards,
through #69314. The port adds what master's own tests do not cover: a
NULL bitmap of two bytes, and the forward request that `FEOpExecutor`
builds and Thrift carries. The PR changes tests only.

**Background.** A client that runs a server-side prepared statement
again may leave out the parameter types (`new_params_bind_flag` = 0) and
have the server reuse the ones it cached. A follower forwards every
COM_STMT_EXECUTE to the master. The master prepares the statement again
for each forwarded execution, so it has no cached types and cannot bind
the parameters. #68162 fixed this on branch-4.1, together with the Lance
prepared vector search it was written for. Master fixed it independently
in #69314: when the client reuses cached types,
`MysqlConnectProcessor.handleExecute` writes the forwarded packet again,
with the NULL bitmap, `new_params_bind_flag` = 1, every type code
including the unsigned flag, and the values. #69314 also has the prepare
stage plan with the context of the re-parsed statement, which is the
other generic part of #68162. Its tests in
`MysqlConnectProcessorLongDataTest` cover cached types, unsigned flags,
changing NULLs, changing types, all-NULL executions and the prepare
context.

**What those tests leave out.** They use at most four parameters, so the
NULL bitmap always fits in one byte. They also give the master the
buffer kept on the follower's context, rather than the request
`FEOpExecutor.buildStmtForwardParams` builds.

**What this PR adds.** The test runs a ten-parameter statement five
times on a follower connection:

- The parameter types are sent with the first and the fourth execution;
the other executions reuse the cached ones.
- One execution has a NULL parameter in each byte of the NULL bitmap.
- An unsigned INT parameter becomes a BIGINT in the fourth execution.

After each execution, the test builds the forward request with
`FEOpExecutor`, passes it through a Thrift round trip and executes it on
a new proxy context whose placeholders have no types. It then checks
three things: every parameter binds to the same value as on the
follower, the unsigned flag carries over, and the client's packet is
read to its end and left unchanged. The code is the 4.1 test adapted to
master's structure: master's `PrepareCommand`, query id and audit setup,
as in `MysqlConnectProcessorLongDataTest`, and expressions compared
directly. The 4.1 assertion that the follower keeps the client's buffer
was dropped, because master writes a separate buffer for cached types by
design.

**Results.**

| `handleExecute` on master | `MysqlConnectProcessorLongDataTest` (14) |
`MysqlPreparedStatementForwardingTest` (1) |
|---|---|---|
| as is | pass | pass |
| no rewrite of the forwarded packet for cached types (the bug #68162
fixed) | 3 fail | fails: the master binds untyped placeholders (`No
value present`) |
| the rewrite keeps only the first byte of the NULL bitmap | pass |
fails: the master reads the packet one byte off |

Co-authored-by: Gabriel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/4.1.x dev/4.2.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants