Skip to content

branch-4.2: [fix](variant) Read exact sparse cells during path promotion - #70356

Merged
yiguolei merged 8 commits into
apache:branch-4.2from
eldenmoon:branch-pick-70212-4.2-20261106
Nov 10, 2026
Merged

yiguolei merged 8 commits into
apache:branch-4.2from
eldenmoon:branch-pick-70212-4.2-20261106

Conversation

@eldenmoon

Copy link
Copy Markdown
Member

cherry-pick #70212

Conflicts resolved: kept branch-4.2 exact-match _fill_variant_v2_path (already equivalent; dual V1/V2 API) and added intent comment. Adapted new BE UTs to branch APIs (ColumnVariant::create_binary_column_fn / StorageReadOptions.stats / use_variant_v2=true). BE UT left to CI (branch-4.2 full UT build known lance_reader issue); same adaptations validated via branch-4.1 UT run.

@eldenmoon
eldenmoon requested a review from yiguolei as a code owner November 6, 2026 12:08
@eldenmoon

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@eldenmoon

Copy link
Copy Markdown
Member Author

run buildall

@eldenmoon

Copy link
Copy Markdown
Member Author

run cloud_p0

…tests

On branch-4.x, casting Variant(max_subcolumns=0) from parse_to_variant
into a column typed Variant(max_subcolumns=1) is rejected. Insert JSON
strings instead so the cast uses the target column type, keeping the
compaction sparse-ancestor promotion coverage from apache#70212.
@eldenmoon

Copy link
Copy Markdown
Member Author

run buildall

Same branch-4.x Variant→Variant cast rejection as the literal inserts.
@eldenmoon

Copy link
Copy Markdown
Member Author

run buildall

…anch-4.x

branch-4.x CAST(variant_string AS STRING) returns JSON-quoted "text",
while the cherry-picked master .out expected unquoted text. NonConcurrent
sparse_ancestor_promotion_extended failed qt_before on that cell.
@eldenmoon

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 64.98% (30500/46937)
Line Coverage 49.93% (323736/648318)
Region Coverage 45.70% (260925/570944)
Branch Coverage 47.42% (123628/260693)

… bool/[]

On branch-4.x, CAST(bool AS STRING) yields true/false while CAST(variant AS
STRING) serializes bool as 1/0; empty array sparse values are omitted.
@eldenmoon

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 64.99% (30504/46937)
Line Coverage 49.96% (323909/648318)
Region Coverage 45.70% (260916/570944)
Branch Coverage 47.43% (123641/260693)

@eldenmoon

Copy link
Copy Markdown
Member Author

run cloud_p0

…on on branch-4.x

Before compaction, CAST(v AS STRING) still shows "a":1; after vertical
compaction promotes the typed path, branch-4.x serializes the bool as true.
Align promoted/repromoted .out accordingly.
@eldenmoon

Copy link
Copy Markdown
Member Author

run buildall

@eldenmoon

Copy link
Copy Markdown
Member Author

run buildall

@eldenmoon

Copy link
Copy Markdown
Member Author

run cloud_p0

@eldenmoon

Copy link
Copy Markdown
Member Author

run nonConcurrent

@eldenmoon

Copy link
Copy Markdown
Member Author

run cloud_p0

@eldenmoon

Copy link
Copy Markdown
Member Author

run external

@eldenmoon

Copy link
Copy Markdown
Member Author

run cloud_p0

@eldenmoon

Copy link
Copy Markdown
Member Author

run external

@eldenmoon

Copy link
Copy Markdown
Member Author

run cloud_p0

@eldenmoon

Copy link
Copy Markdown
Member Author

run external

@yiguolei

Copy link
Copy Markdown
Contributor

skip buildall

@yiguolei
yiguolei merged commit b6b9ea8 into apache:branch-4.2 Nov 10, 2026
30 of 31 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Nov 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants