Repository navigation
Conversation
### What problem does this PR solve? Problem Summary: TQueryOptions used field 237 for paimon_blob_max_size, while master uses the same i64 field for spill_analytic_sink_mem_limit_bytes. A branch-4.1 sender with a 32 MiB blob limit can therefore replace the intended 64 MiB spill threshold without a decoding error. Move the optional blob size option to field 1024 and extend the reserved-id comment to cover 237. Keep the 10 MiB default and the option type unchanged. Generated-type checks confirm that the new payload preserves the 32 MiB blob limit and leaves master's 64 MiB spill threshold unchanged. This does not retrofit old binaries: update FE and BE together, since mixed field-237/field-1024 builds do not preserve custom blob limits. ### Release note Paimon blob size query options use Thrift field 1024. Deploy matching FE and BE builds when replacing builds that used field 237. ### Check List (For Author) - Test: Manual binary/compact protocol validation with real generated branch-4.1 and master types; baseline collision reproduction; fixed non-default 32 MiB round trip and unchanged 64 MiB master spill limit; Java/C++/Python Thrift code generation; live branch ID checks and local history search; git diff --check. Full FE/BE builds and SQL regression were not run for this IDL-only change. - Behavior changed: Yes. The wire field id changes from 237 to 1024; the option default and single-value limit semantics are unchanged. - Does this need documentation: No. The rollout boundary is documented above.
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
Closing this approach: keep the existing branch-4.1 paimon_blob_max_size field at 237. A follow-up PR will align master with that field and move its conflicting spill option to a separate unused id. |
HappenLee
added a commit
to HappenLee/incubator-doris
that referenced
this pull request
Nov 3, 2026
### What problem does this PR solve? Related PR: apache#69658 Problem Summary: branch-4.1 uses TQueryOptions field 237 for paimon_blob_max_size, while master uses the same i64 field for spill_analytic_sink_mem_limit_bytes. A branch-4.1 FE sending a 32 MiB blob limit silently replaces the intended 64 MiB analytic spill threshold. The branch-4.1 BLOB scan metadata also lacks matching definitions on master. Align the BLOB Thrift contract with branch-4.1: - Keep optional i64 paimon_blob_max_size at field 237 with its 10 MiB default; move the analytic spill option to unused field 238 with its 64 MiB default. Existing analytic accessors and FE setters keep their names. - Add TPaimonBlobFileDesc with optional path, file_size, first_row_id, row_count and field_id at the same field IDs 1 through 5. - Append optional TPaimonFileDesc fields 21 blob_files, 22 anchor_first_row_id and 23 anchor_row_count. Preserve master's existing field 20 original_file_path. These changes align and reserve the protocol definitions. Paimon BLOB materialization and FILE execution support remain separate changes. The anchor fields describe the complete Parquet anchor's logical row range; byte splits share the same origin, rather than using their first returned row. The field assignments were checked against current branch-3.1, branch-4.0, branch-4.1, branch-4.2 and master definitions. The branch-4.1 contract stays unchanged. This replaces the field-1024 approach in closed PR apache#70027. ### Release note Align BLOB query and scan metadata with branch-4.1. Master builds using the previous analytic spill field 237 are not wire compatible with this correction: deploy matching FE/BE builds when replacing those master builds. This is not a guarantee of general cross-version query-plan compatibility. ### Check List (For Author) - Test: Java/C++/Python Thrift code generation; Binary/Compact protocol checks using actual generated branch-4.1 and modified-master types: reproduce the query-option collision, preserve the 32 MiB blob limit in both directions, retain the 64 MiB spill default, round-trip the separate 128 MiB spill limit, and verify the old-master boundary; round-trip all BLOB descriptor members and anchor metadata in both directions, preserve master's field 20, and check missing optional fields. git diff --check; SelectDB revert tree equality. Full FE/BE builds and SQL regression were not run for this IDL-only change. - Behavior changed: Yes. Master field 237 represents the Paimon blob limit; the analytic spill limit uses 238, with unchanged defaults. Optional BLOB scan metadata uses the same IDs and types as branch-4.1. - Does this need documentation: No. The rollout boundary is documented above.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Problem Summary: TQueryOptions used field 237 for paimon_blob_max_size,
while master uses the same i64 field for spill_analytic_sink_mem_limit_bytes.
A branch-4.1 sender with a 32 MiB blob limit can therefore replace the
intended 64 MiB spill threshold without a decoding error.
Move the optional blob size option to field 1024 and extend the reserved-id
comment to cover 237. Keep the 10 MiB default and the option type unchanged.
Generated-type checks confirm that the new payload preserves the 32 MiB
blob limit and leaves master's 64 MiB spill threshold unchanged.
This does not retrofit old binaries: update FE and BE together, since
mixed field-237/field-1024 builds do not preserve custom blob limits.
Release note
Paimon blob size query options use Thrift field 1024. Deploy matching FE
and BE builds when replacing builds that used field 237.
Check List (For Author)
branch-4.1 and master types; baseline collision reproduction; fixed
non-default 32 MiB round trip and unchanged 64 MiB master spill limit;
Java/C++/Python Thrift code generation; live branch ID checks and local
history search; git diff --check.
Full FE/BE builds and SQL regression were not run for this IDL-only change.
the option default and single-value limit semantics are unchanged.