### What problem does this PR solve?

Issue Number: None

Related PR: apache#66475

Problem Summary: This change separately submits the Meta Service portion of apache#66475. During a partition-level inverted-index format rollout, a rowset can use a format that differs from its shared schema metadata. The recycler and checker previously selected index paths from shared schema metadata or its cache, which could target V1 paths for a rowset whose files use V3.

Persist the actual inverted-index storage format in rowset and tablet metadata. Make the recycler and checker prefer the rowset format, while retaining schema and schema-KV fallback for metadata written before the field existed. Add a Cloud unit test where the shared schema is V1 and the rowset is V3, verifying that recycler deletes the V3 index file.

### Release note

None

### Check List (For Author)

- Test: Unit Test
    - `./run-cloud-ut.sh --run --filter=recycler_test:RecyclerTest.delete_tmp_rowset_data_with_rowset_idx_v3`
- Behavior changed: Yes. Meta Service recycler and checker use each rowset\047s persisted inverted-index format when selecting index files.
- Does this need documentation: No

Related PR: #66475

Problem Summary:

This PR separately submits the Meta Service changes from #66475. When
the shared schema format differs from the physical format used by an
existing rowset, recycler and checker must use the rowset format to
select the correct inverted-index files.

The change persists the physical inverted-index format in metadata and
makes recycler/checker prefer the rowset value, with the existing schema
and schema-KV fallback retained for old metadata.

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [x] Unit Test
- `./run-cloud-ut.sh --run
--filter=recycler_test:RecyclerTest.delete_tmp_rowset_data_with_rowset_idx_v3`
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
- [x] Yes. Recycler and checker select inverted-index files from the
persisted rowset format.

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

…68449) (#68618)

### What problem does this PR solve?

Issue Number: None

Related PR: #68449

Problem Summary:

Backport #68449 (`50a39819e185f54fbb3a82e67182b1ec8f78433b`) to
`branch-4.1`. The changes stabilize regression clients and orchestration
for TLS/mTLS, Docker isolation, multi-cluster state cleanup, cloud
cache/compaction, asynchronous metadata visibility, and delete-bitmap
restart/concurrency tests. Doris Compose uses exact container-name
matching and retries concurrent removal races.

Resolved 27 conflict paths with these branch adaptations:

- Keep the `nereids_p0` suite paths, generated database names, planner
settings, existing SHOW columns and 4.1-only test logic.
- Preserve configured-master JDBC/HTTP endpoints in the
insert-visible-timeout suite, including mapped ports from #65340.
- Keep the routine-load WHERE expectation `(k00 = 8)`: 4.1's
`PlanUtils.ExpressionToExpr` stores unquoted slot labels, unlike master.
Retain the source topic fix, diagnostic fields and independent polling
counters.
- Keep `nereids_p0/hbo` in the non-concurrent exclusions while excluding
all `cloud_p0` suites as intended by the source fix.
- Combine the shared TLS/compaction helpers with 4.1's existing
framework behavior; do not import unrelated scenarios present only in
the source commit's parent.
- Omit `test_partition_cloud_add_build_index_validation.groovy`: the
partition index-format rollout feature from #66475 and its required
properties/configuration are absent from 4.1.
- Reuse the original PR's result files without manually creating
expected output; retain the deterministic variant ordering fix.

The resulting backport changes 350 files; no FE implementation,
protocol, or storage-format change is added.

### Release note

None

### Check List (For Author)

- Test:
- [x] Regression test on an existing 4.1.4 cloud cluster (1 FE / 3 BE):
127 passed, 3 failed, 186 skipped across all 316 changed suites.
- Retained failures: `test_create_view_nereids` (SHOW VIEW column-name
quoting mismatch), `test_streaming_insert_job_priv` (one successful task
and one pending task after the 300-second wait), and
`test_load_channel_profile` (S3 TVF query reports unknown column `c1`).
Their failed assertions/queries are unchanged by this backport. Root
causes are not established by this run.
- Python support is disabled: 19 attempts hit that prerequisite and 23
more suites were skipped. One JDBC fixture was unavailable, and the
single-BE Java static-load case returned early on this three-BE cluster;
these are counted as skipped, not passes.
- Other skipped suites require dedicated Docker/multi-compute-group
fixtures, restarts, TLS/external services, global configuration/fault
injection, or stress capacity. Four packed-file injection suites were
excluded before launch.
- Arrow Flight authentication passed after correcting only the
task-local endpoint configuration. Three small-table TTL/row-store
compaction follow-ups also passed.
- The failed streaming job and its test user were cleaned up. All FE/BE
nodes remained alive and HTTP health returned 200 after execution.
- [x] Unit Test: `run-regression-test.sh --compile` completed on the
remote validation host; all 35 regression-framework tests passed (0
failures, 0 errors, 0 skips).
    - [x] Manual test:
        - Diff whitespace and conflict-marker checks passed.
- All 340 changed existing Groovy files passed conversion-phase parsing
with Groovy 4.0.19 / JDK 17; both changed Python files passed
`py_compile`.
- `build-support/check-format.sh` passed with clang-format 16.0.6 (4,400
C/C++ files).
- Compared conflict resolutions with the source patch and branch-4.1
implementations, including RPC/debug-point dependencies and routine-load
expression serialization.
- BE/FE module builds, deployment and clang-tidy were not run, per the
requested existing-cluster validation scope. The live enterprise cluster
is not a build of this backport, so BE debug-point additions and
excluded scenarios still require CI.
- Behavior changed: Yes. Regression framework/test behavior and Doris
Compose discovery match the applicable source fixes; BE additions are
gated debug points.
- Does this need documentation: No.

<details>
<summary>Backport self-review checkpoints</summary>

| Checkpoint | Conclusion |
| --- | --- |
| Goal and tests | Applicable source fixes preserved; syntax/static
checks and 35 framework unit tests pass. Existing-cluster suite outcomes
and remaining CI scope are listed above. |
| Scope | Source patch plus documented branch adaptations; unsupported
feature suite omitted. |
| Concurrency and locking | Existing production lock order retained;
debug-only post-calculation barrier releases the rowset lock before
waiting. Suite worker errors propagate to their owners. |
| Lifecycle | Source connection/process cleanup and shared-state
restoration retained. |
| Configuration | Existing TLS and debug configuration used; referenced
cloud sync settings exist in 4.1. |
| Compatibility | Target RPC/HTTP/debug APIs verified; branch endpoint
mappings and result schemas retained. No wire/storage format changes. |
| Parallel paths | Relevant plaintext/TLS and local/cloud helper paths
retained. |
| Conditional checks | Source bounded polling and negative assertions
retained; branch adaptations checked against implementation. |
| Test coverage | Source coverage retained; 35 framework tests and 127
existing-cluster suites passed, with 3 retained failures and 186
documented skips. |
| Expected results | Source result files reused; no hand-written golden
output. |
| Observability | Source timeout/last-state diagnostics and
token/tablet/transaction debug logs retained. |
| Persistence and failover | No production persistence or replay change.
|
| Writes and atomicity | Production write behavior unchanged outside
enabled test debug points. |
| FE/BE propagation | Existing target sync-load RPC and metadata
endpoints support new helpers. |
| Performance | Production additions are gated debug checks; regression
polling is bounded. |
| Other findings | Focused framework/BE, TLS-suite and cloud-suite
reviews found no unresolved backport-specific blocker. |

</details>

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases and CI results
- [ ] Confirm document
- [ ] Add branch pick label

…68449) (#68618)

### What problem does this PR solve?

Issue Number: None

Related PR: #68449

Problem Summary:

Backport #68449 (`50a39819e185f54fbb3a82e67182b1ec8f78433b`) to
`branch-4.1`. The changes stabilize regression clients and orchestration
for TLS/mTLS, Docker isolation, multi-cluster state cleanup, cloud
cache/compaction, asynchronous metadata visibility, and delete-bitmap
restart/concurrency tests. Doris Compose uses exact container-name
matching and retries concurrent removal races.

Resolved 27 conflict paths with these branch adaptations:

- Keep the `nereids_p0` suite paths, generated database names, planner
settings, existing SHOW columns and 4.1-only test logic.
- Preserve configured-master JDBC/HTTP endpoints in the
insert-visible-timeout suite, including mapped ports from #65340.
- Keep the routine-load WHERE expectation `(k00 = 8)`: 4.1's
`PlanUtils.ExpressionToExpr` stores unquoted slot labels, unlike master.
Retain the source topic fix, diagnostic fields and independent polling
counters.
- Keep `nereids_p0/hbo` in the non-concurrent exclusions while excluding
all `cloud_p0` suites as intended by the source fix.
- Combine the shared TLS/compaction helpers with 4.1's existing
framework behavior; do not import unrelated scenarios present only in
the source commit's parent.
- Omit `test_partition_cloud_add_build_index_validation.groovy`: the
partition index-format rollout feature from #66475 and its required
properties/configuration are absent from 4.1.
- Reuse the original PR's result files without manually creating
expected output; retain the deterministic variant ordering fix.

The resulting backport changes 350 files; no FE implementation,
protocol, or storage-format change is added.

### Release note

None

### Check List (For Author)

- Test:
- [x] Regression test on an existing 4.1.4 cloud cluster (1 FE / 3 BE):
127 passed, 3 failed, 186 skipped across all 316 changed suites.
- Retained failures: `test_create_view_nereids` (SHOW VIEW column-name
quoting mismatch), `test_streaming_insert_job_priv` (one successful task
and one pending task after the 300-second wait), and
`test_load_channel_profile` (S3 TVF query reports unknown column `c1`).
Their failed assertions/queries are unchanged by this backport. Root
causes are not established by this run.
- Python support is disabled: 19 attempts hit that prerequisite and 23
more suites were skipped. One JDBC fixture was unavailable, and the
single-BE Java static-load case returned early on this three-BE cluster;
these are counted as skipped, not passes.
- Other skipped suites require dedicated Docker/multi-compute-group
fixtures, restarts, TLS/external services, global configuration/fault
injection, or stress capacity. Four packed-file injection suites were
excluded before launch.
- Arrow Flight authentication passed after correcting only the
task-local endpoint configuration. Three small-table TTL/row-store
compaction follow-ups also passed.
- The failed streaming job and its test user were cleaned up. All FE/BE
nodes remained alive and HTTP health returned 200 after execution.
- [x] Unit Test: `run-regression-test.sh --compile` completed on the
remote validation host; all 35 regression-framework tests passed (0
failures, 0 errors, 0 skips).
    - [x] Manual test:
        - Diff whitespace and conflict-marker checks passed.
- All 340 changed existing Groovy files passed conversion-phase parsing
with Groovy 4.0.19 / JDK 17; both changed Python files passed
`py_compile`.
- `build-support/check-format.sh` passed with clang-format 16.0.6 (4,400
C/C++ files).
- Compared conflict resolutions with the source patch and branch-4.1
implementations, including RPC/debug-point dependencies and routine-load
expression serialization.
- BE/FE module builds, deployment and clang-tidy were not run, per the
requested existing-cluster validation scope. The live enterprise cluster
is not a build of this backport, so BE debug-point additions and
excluded scenarios still require CI.
- Behavior changed: Yes. Regression framework/test behavior and Doris
Compose discovery match the applicable source fixes; BE additions are
gated debug points.
- Does this need documentation: No.

<details>
<summary>Backport self-review checkpoints</summary>

| Checkpoint | Conclusion |
| --- | --- |
| Goal and tests | Applicable source fixes preserved; syntax/static
checks and 35 framework unit tests pass. Existing-cluster suite outcomes
and remaining CI scope are listed above. |
| Scope | Source patch plus documented branch adaptations; unsupported
feature suite omitted. |
| Concurrency and locking | Existing production lock order retained;
debug-only post-calculation barrier releases the rowset lock before
waiting. Suite worker errors propagate to their owners. |
| Lifecycle | Source connection/process cleanup and shared-state
restoration retained. |
| Configuration | Existing TLS and debug configuration used; referenced
cloud sync settings exist in 4.1. |
| Compatibility | Target RPC/HTTP/debug APIs verified; branch endpoint
mappings and result schemas retained. No wire/storage format changes. |
| Parallel paths | Relevant plaintext/TLS and local/cloud helper paths
retained. |
| Conditional checks | Source bounded polling and negative assertions
retained; branch adaptations checked against implementation. |
| Test coverage | Source coverage retained; 35 framework tests and 127
existing-cluster suites passed, with 3 retained failures and 186
documented skips. |
| Expected results | Source result files reused; no hand-written golden
output. |
| Observability | Source timeout/last-state diagnostics and
token/tablet/transaction debug logs retained. |
| Persistence and failover | No production persistence or replay change.
|
| Writes and atomicity | Production write behavior unchanged outside
enabled test debug points. |
| FE/BE propagation | Existing target sync-load RPC and metadata
endpoints support new helpers. |
| Performance | Production additions are gated debug checks; regression
polling is bounded. |
| Other findings | Focused framework/BE, TLS-suite and cloud-suite
reviews found no unresolved backport-specific blocker. |

</details>

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases and CI results
- [ ] Confirm document
- [ ] Add branch pick label

…llout (#69205)

### What problem does this PR solve?

Issue Number: None

Related PR: [#66793](#66793),
[#66475](#66475),
[#67583](#67583),
[#68449](#68449), [selectdb-core
#11528](selectdb/selectdb-core#11528)

Problem Summary:

Backport Cloud partition inverted-index storage format rollout to
`branch-4.2`. The change records the physical format in tablet and
rowset metadata, preserves it through recycling and schema changes, and
exposes the per-partition format in metadata views. New partitions can
use a new format while existing partitions retain theirs.

The commits carry the Meta Service prerequisite
(`a303308d7ed77027420c470dc314474463d38959`), the feature
(`7219c67265bc700c1457b3339c6a9d0143d36c1f`), the validation repair
(`8d14eba8b41f9ddad018903861a5d0d3c4a8a668`), and the four
feature-specific V1 test deletions from
`50a39819e185f54fbb3a82e67182b1ec8f78433b`.
[#68618](#68618) already backported
the other #68449 changes but skipped this suite because the feature was
absent then. The legacy ALTER TABLE analyzer still exists on this
branch, so the port also applies its integration from selectdb-core
#11528 (`ba8404d9db6f8cfea889ce5b67d852e16b039648`) and adds a focused
test.

V2, V3, and SNII are available on this branch.

Protobuf compatibility was checked against Doris master, both target
branches, and selectdb-core 4.1. All four repositories use exactly these
tags:

| Message | Field number |
| --- | ---: |
| `RowsetMetaPB.inverted_index_storage_format` | 1018 |
| `RowsetMetaCloudPB.inverted_index_storage_format` | 117 |
| `TabletMetaPB.inverted_index_storage_format` | 1006 |
| `TabletMetaCloudPB.inverted_index_storage_format` | 104 |

### Source and hunk audit

The port preserves existing branch APIs and JUnit style. It omits
unrelated row-binlog context absent from this branch while retaining the
exact protobuf field tags.

The range diff for #66793 shows only the protobuf context shift from
missing row-binlog fields. The #66475 range diff shows target API and
test adaptations, including the existing `CloudInternalCatalogTest`
class and the absence of `BinlogSize` in `PartitionsProcDir`. The #67583
range diff preserves its source code before the branch-specific format
adaptation. The following table records every source hunk; "Adapted"
rows explain target differences.

| PR | File | Source hunk | Status | Added lines for review |
| --- | --- | --- | --- | --- |
| 66793 | be/src/cloud/pb_convert.cpp | @@ -104,0 +105,3 @@ void
doris_rowset_meta_to_cloud(RowsetMetaCloudPB* out, const RowsetMetaPB&
in) | Ported | if (in.has_inverted_index_storage_format()) {;
out->set_inverted_index_storage_format(in.inverted_index_storage_format());
|
| 66793 | be/src/cloud/pb_convert.cpp | @@ -200,0 +204,3 @@ void
doris_rowset_meta_to_cloud(RowsetMetaCloudPB* out, RowsetMetaPB&& in) {
| Ported | if (in.has_inverted_index_storage_format()) {;
out->set_inverted_index_storage_format(in.inverted_index_storage_format());
|
| 66793 | be/src/cloud/pb_convert.cpp | @@ -312,0 +319,3 @@ void
cloud_rowset_meta_to_doris(RowsetMetaPB* out, const RowsetMetaCloudPB&
in) | Ported | if (in.has_inverted_index_storage_format()) {;
out->set_inverted_index_storage_format(in.inverted_index_storage_format());
|
| 66793 | be/src/cloud/pb_convert.cpp | @@ -407,0 +417,3 @@ void
cloud_rowset_meta_to_doris(RowsetMetaPB* out, RowsetMetaCloudPB&& in) {
| Ported | if (in.has_inverted_index_storage_format()) {;
out->set_inverted_index_storage_format(in.inverted_index_storage_format());
|
| 66793 | be/src/cloud/pb_convert.cpp | @@ -792,0 +805,3 @@ void
doris_tablet_meta_to_cloud(TabletMetaCloudPB* out, const TabletMetaPB&
in) | Ported | if (in.has_inverted_index_storage_format()) {;
out->set_inverted_index_storage_format(in.inverted_index_storage_format());
|
| 66793 | be/src/cloud/pb_convert.cpp | @@ -877,0 +893,3 @@ void
doris_tablet_meta_to_cloud(TabletMetaCloudPB* out, TabletMetaPB&& in) {
| Ported | if (in.has_inverted_index_storage_format()) {;
out->set_inverted_index_storage_format(in.inverted_index_storage_format());
|
| 66793 | be/src/cloud/pb_convert.cpp | @@ -969,0 +988,3 @@ void
cloud_tablet_meta_to_doris(TabletMetaPB* out, const TabletMetaCloudPB&
in) | Ported | if (in.has_inverted_index_storage_format()) {;
out->set_inverted_index_storage_format(in.inverted_index_storage_format());
|
| 66793 | be/src/cloud/pb_convert.cpp | @@ -1054,0 +1076,3 @@ void
cloud_tablet_meta_to_doris(TabletMetaPB* out, TabletMetaCloudPB&& in) {
| Ported | if (in.has_inverted_index_storage_format()) {;
out->set_inverted_index_storage_format(in.inverted_index_storage_format());
|
| 66793 | cloud/src/recycler/checker.cpp | @@ -774,0 +775,4 @@ int
InstanceChecker::do_check() { | Ported | const auto index_format =;
rs_meta.has_inverted_index_storage_format() |
| 66793 | cloud/src/recycler/checker.cpp | @@ -777,2 +781 @@ int
InstanceChecker::do_check() { | Ported | if (index_format ==
InvertedIndexStorageFormatPB::V1) { |
| 66793 | cloud/src/recycler/recycler.cpp | @@ -3474,2 +3474,3 @@ int
InstanceRecycler::delete_rowset_data(const RowsetMetaCloudPB&
rs_meta_pb) { | Ported | InvertedIndexStorageFormatPB index_format =
rs_meta_pb.has_inverted_index_storage_format(); |
| 66793 | cloud/src/recycler/recycler.cpp | @@ -3487 +3488,2 @@ int
InstanceRecycler::delete_rowset_data(const RowsetMetaCloudPB&
rs_meta_pb) { | Ported | if
(!rs_meta_pb.has_inverted_index_storage_format() &&;
rs_meta_pb.tablet_schema().has_inverted_index_storage_format()) { |
| 66793 | cloud/src/recycler/recycler.cpp | @@ -3501 +3503,3 @@ int
InstanceRecycler::delete_rowset_data(const RowsetMetaCloudPB&
rs_meta_pb) { | Ported | if
(!rs_meta_pb.has_inverted_index_storage_format()) {; index_format =
index_info.first; |
| 66793 | cloud/src/recycler/recycler.cpp | @@ -4239 +4243,3 @@ int
InstanceRecycler::delete_rowset_data( | Ported |
InvertedIndexStorageFormatPB index_format =
rs.has_inverted_index_storage_format(); |
| 66793 | cloud/src/recycler/recycler.cpp | @@ -4247 +4253,2 @@ int
InstanceRecycler::delete_rowset_data( | Ported | if
(!rs.has_inverted_index_storage_format() &&;
rs.tablet_schema().has_inverted_index_storage_format()) { |
| 66793 | cloud/src/recycler/recycler.cpp | @@ -4265 +4272,3 @@ int
InstanceRecycler::delete_rowset_data( | Ported | if
(!rs.has_inverted_index_storage_format()) {; index_format =
index_info.first; |
| 66793 | cloud/test/recycler_test.cpp | @@ -8068,0 +8069,65 @@
TEST(RecyclerTest, delete_tmp_rowset_data_with_idx_v2) { | Ported |
TEST(RecyclerTest, delete_tmp_rowset_data_with_rowset_idx_v3) {; auto
txn_kv = std::make_shared<MemTxnKv>(); |
| 66793 | gensrc/proto/olap_file.proto | @@ -185,0 +186,3 @@ message
RowsetMetaPB { | Adapted | // Actual inverted-index format used by this
rowset's files. It is independent; // from the shared (index_id,
schema_version) schema key.; Unrelated row-binlog fields are absent on
target; format tags retain source numbers. |
| 66793 | gensrc/proto/olap_file.proto | @@ -303,0 +307,3 @@ message
RowsetMetaCloudPB { | Adapted | // Actual inverted-index format used by
this rowset's files. It is independent; // from the shared (index_id,
schema_version) schema key.; Unrelated row-binlog fields are absent on
target; format tags retain source numbers. |
| 66793 | gensrc/proto/olap_file.proto | @@ -735,0 +742,2 @@ message
TabletMetaPB { | Ported | // Immutable format of this tablet's
inverted-index files.; optional InvertedIndexStorageFormatPB
inverted_index_storage_format = 1006; |
| 66793 | gensrc/proto/olap_file.proto | @@ -794,0 +803,2 @@ message
TabletMetaCloudPB { | Ported | // Immutable format of this tablet's
inverted-index files.; optional InvertedIndexStorageFormatPB
inverted_index_storage_format = 104; |
| 66475 | be/src/cloud/cloud_rowset_writer.cpp | @@ -80,0 +81,4 @@
Status CloudRowsetWriter::init(const RowsetWriterContext&
rowset_writer_context) | Ported | if
(_context.persist_inverted_index_storage_format &&;
_context.inverted_index_storage_format.has_value()) { |
| 66475 | be/src/cloud/cloud_tablet.cpp | @@ -851,0 +852,3 @@
Result<std::unique_ptr<RowsetWriter>> CloudTablet::create_rowset_writer(
| Ported | context.inverted_index_storage_format =
tablet_meta()->inverted_index_storage_format();;
context.persist_inverted_index_storage_format = |
| 66475 | be/src/cloud/cloud_tablet.cpp | @@ -900,0 +904,3 @@
Result<std::unique_ptr<RowsetWriter>>
CloudTablet::create_transient_rowset_write | Ported |
context.inverted_index_storage_format =
tablet_meta()->inverted_index_storage_format();;
context.persist_inverted_index_storage_format = |
| 66475 | be/src/service/http/action/show_nested_index_file_action.cpp |
@@ -24,0 +25 @@ | Ported | #include "cloud/cloud_tablet.h" |
| 66475 | be/src/service/http/action/show_nested_index_file_action.cpp |
@@ -55,0 +57,4 @@ Status
ShowNestedIndexFileAction::_handle_show_nested_index_file(HttpRequest*
re | Ported | if (auto cloud_tablet =
std::dynamic_pointer_cast<CloudTablet>(tablet)) {; // The debug endpoint
must inspect all visible cloud rowsets, n |
| 66475 | be/src/storage/rowset/beta_rowset_writer.cpp | @@ -397,0
+398,4 @@ Status BaseBetaRowsetWriter::init(const RowsetWriterContext&
rowset_writer_conte | Ported | if
(_context.persist_inverted_index_storage_format &&;
_context.inverted_index_storage_format.has_value()) { |
| 66475 | be/src/storage/rowset/rowset_meta.cpp | @@ -74,0 +75,6 @@ bool
RowsetMeta::init_from_pb(const RowsetMetaPB& rowset_meta_pb) { | Ported
| if (rowset_meta_pb.has_inverted_index_storage_format()) {;
_rowset_meta_pb.set_inverted_index_storage_format( |
| 66475 | be/src/storage/rowset/rowset_meta.cpp | @@ -76 +82,6 @@ bool
RowsetMeta::init_from_pb(const RowsetMetaPB& rowset_meta_pb) { | Ported
| TabletSchemaPB schema_pb = rowset_meta_pb.tablet_schema();; if
(rowset_meta_pb.has_inverted_index_storage_format()) { |
| 66475 | be/src/storage/rowset/rowset_meta.cpp | @@ -213,0 +225,4 @@
void RowsetMeta::to_rowset_pb(RowsetMetaPB* rs_meta_pb, bool
skip_schema) const | Ported | if
(rs_meta_pb->has_inverted_index_storage_format()) {;
rs_meta_pb->mutable_tablet_schema()->set_inverted_index_storage_fo |
| 66475 | be/src/storage/rowset/rowset_meta.cpp | @@ -225,0 +241,6 @@
void RowsetMeta::set_tablet_schema(const TabletSchemaSPtr&
tablet_schema) { | Ported | if
(_rowset_meta_pb.has_inverted_index_storage_format()) {; TabletSchemaPB
schema_pb; |
| 66475 | be/src/storage/rowset/rowset_meta.cpp | @@ -234,0 +256,5 @@
void RowsetMeta::set_tablet_schema(const TabletSchemaPB& tablet_schema)
{ | Ported | TabletSchemaPB resolved_schema = tablet_schema;; if
(_rowset_meta_pb.has_inverted_index_storage_format()) { |
| 66475 | be/src/storage/rowset/rowset_meta.cpp | @@ -239 +265 @@ void
RowsetMeta::set_tablet_schema(const TabletSchemaPB& tablet_schema) { |
Ported | TabletSchema::deterministic_string_serialize(resolved_schema));
|
| 66475 | be/src/storage/rowset/rowset_meta.cpp | @@ -243,0 +270,10 @@
void RowsetMeta::set_tablet_schema(const TabletSchemaPB& tablet_schema)
{ | Ported | void
RowsetMeta::set_inverted_index_storage_format(InvertedIndexStorageFormatPB
format) {; _rowset_meta_pb.set_inverted_index_storage_format(forma |
| 66475 | be/src/storage/rowset/rowset_meta.cpp | @@ -250 +286,6 @@ bool
RowsetMeta::_deserialize_from_pb(std::string_view value) { | Ported |
TabletSchemaPB schema_pb = _rowset_meta_pb.tablet_schema();; if
(_rowset_meta_pb.has_inverted_index_storage_format()) { |
| 66475 | be/src/storage/rowset/rowset_meta.cpp | @@ -262,0 +304,4 @@
bool RowsetMeta::_serialize_to_pb(std::string* value) { | Ported | if
(rowset_meta_pb.has_inverted_index_storage_format()) {;
rowset_meta_pb.mutable_tablet_schema()->set_inverted_index_storage_form
|
| 66475 | be/src/storage/rowset/rowset_meta.h | @@ -115,0 +116,10 @@
public: | Ported | bool has_inverted_index_storage_format() const {;
return _rowset_meta_pb.has_inverted_index_storage_format(); |
| 66475 | be/src/storage/rowset/rowset_writer_context.h | @@ -76 +76,5
@@ struct RowsetWriterContext { | Ported | // Immutable inverted-index
file format inherited from the owner tablet.;
std::optional<InvertedIndexStorageFormatPB> inverted_index_storage_f |
| 66475 | be/src/storage/tablet/tablet.cpp | @@ -2060,0 +2061 @@ void
Tablet::_init_context_common_fields(RowsetWriterContext& context) { |
Ported | context.inverted_index_storage_format =
tablet_meta()->inverted_index_storage_format(); |
| 66475 | be/src/storage/tablet/tablet_meta.cpp | @@ -281,2 +281,2 @@
TabletMeta::TabletMeta(const TabletMeta& b) | Ported |
_vertical_compaction_num_columns_per_group(b._vertical_compaction_num_columns_per_group),;
_inverted_index_storage_format(b._inver |
| 66475 | be/src/storage/tablet/tablet_meta.cpp | @@ -801,0 +802,4 @@
void TabletMeta::init_from_pb(const TabletMetaPB& tablet_meta_pb) { |
Ported | _inverted_index_storage_format.reset();; if
(tablet_meta_pb.has_inverted_index_storage_format()) { |
| 66475 | be/src/storage/tablet/tablet_meta.cpp | @@ -832 +836,5 @@ void
TabletMeta::init_from_pb(const TabletMetaPB& tablet_meta_pb) { | Ported
| TabletSchemaPB schema_pb = tablet_meta_pb.schema();; if
(_inverted_index_storage_format.has_value()) { |
| 66475 | be/src/storage/tablet/tablet_meta.cpp | @@ -957,0 +966,5 @@
void TabletMeta::to_meta_pb(TabletMetaPB* tablet_meta_pb, bool
cloud_get_rowset_ | Ported | if
(_inverted_index_storage_format.has_value()) {;
tablet_meta_pb->set_inverted_index_storage_format(*_inverted_index_storage_format);
|
| 66475 | be/src/storage/tablet/tablet_meta.h | @@ -336,0 +337,9 @@
public: | Ported | bool has_inverted_index_storage_format() const {;
return _inverted_index_storage_format.has_value(); |
| 66475 | be/src/storage/tablet/tablet_meta.h | @@ -405,0 +415,3 @@
private: | Ported | // The schema KV is shared by (index_id,
schema_version). Keep the tablet's; // immutable file format outside
that shared identity. |
| 66475 | be/test/cloud/cloud_meta_mgr_test.cpp | @@ -28,0 +29 @@ |
Ported | #include <utility> |
| 66475 | be/test/cloud/cloud_meta_mgr_test.cpp | @@ -31,0 +33 @@ |
Ported | #include "cloud/pb_convert.h" |
| 66475 | be/test/cloud/cloud_meta_mgr_test.cpp | @@ -287,0 +290,51 @@
TEST_F(CloudMetaMgrTest, abort_txn_uses_label_when_txn_id_is_missing) {
| Ported | TEST_F(CloudMetaMgrTest,
inverted_index_format_survives_meta_pb_conversion) {; RowsetMetaPB
local_rowset; |
| 66475 | be/test/storage/rowset/beta_rowset_test.cpp | @@ -296,0 +297,5
@@ class S3ClientMockGetErrorData : public S3ClientMock { | Ported |
class TestBetaRowsetWriter final : public BaseBetaRowsetWriter {;
public: |
| 66475 | be/test/storage/rowset/beta_rowset_test.cpp | @@ -376,0
+382,26 @@ TEST_F(BetaRowsetTest, AddToBinlogTest) { | Ported |
TEST_F(BetaRowsetTest, PersistInvertedIndexStorageFormatInRowsetMeta) {;
auto tablet_schema = std::make_shared<TabletSchema>(); |
| 66475 | be/test/storage/rowset/rowset_meta_test.cpp | @@ -35,0 +36 @@
| Ported | #include "storage/tablet/tablet_schema.h" |
| 66475 | be/test/storage/rowset/rowset_meta_test.cpp | @@ -111,0
+113,80 @@ TEST_F(RowsetMetaTest, TestInit) { | Ported |
TEST_F(RowsetMetaTest,
TopLevelInvertedIndexFormatOverridesEmbeddedSchemaFormat) {;
RowsetMetaPB rowset_meta_pb; |
| 66475 | be/test/storage/tablet/tablet_meta_test.cpp | @@ -83,0 +84,50
@@ TEST(TabletMetaTest, SaveAsBufferAndParse) { | Ported |
TEST(TabletMetaTest,
TopLevelInvertedIndexFormatOverridesSharedSchemaFormat) {; TabletMetaPB
tablet_meta_pb; |
| 66475 | fe/fe-common/src/main/java/org/apache/doris/common/Config.java
| @@ -2895,0 +2896,6 @@ public class Config extends ConfigBase { |
Ported | @ConfField(mutable = true, masterOnly = true,; callback =
PartitionInvertedIndexStorageFormatRolloutConfHandler.class, description
= " |
| 66475 |
fe/fe-common/src/main/java/org/apache/doris/common/ConfigBase.java | @@
-98,0 +99,15 @@ public class ConfigBase { | Ported | static class
PartitionInvertedIndexStorageFormatRolloutConfHandler implements
ConfHandler {; @OverRide |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/alter/CloudSchemaChangeJobV2.java
| @@ -42,0 +43 @@ import org.apache.doris.task.AgentTaskQueue; | Ported
| import org.apache.doris.thrift.TInvertedIndexFileStorageFormat; |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/alter/CloudSchemaChangeJobV2.java
| @@ -230 +230,0 @@ public class CloudSchemaChangeJobV2 extends
SchemaChangeJobV2 { | Adapted | ; The blank-line deletion is present in
the target diff. |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/alter/CloudSchemaChangeJobV2.java
| @@ -239,0 +240,3 @@ public class CloudSchemaChangeJobV2 extends
SchemaChangeJobV2 { | Ported | TInvertedIndexFileStorageFormat
invertedIndexFileStorageFormat =; originIndexId == tbl.getBaseIndexId()
|
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/alter/CloudSchemaChangeJobV2.java
| @@ -266 +269 @@ public class CloudSchemaChangeJobV2 extends
SchemaChangeJobV2 { | Ported | invertedIndexFileStorageFormat, |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/analysis/InvertedIndexUtil.java
| @@ -320,5 +320 @@ public class InvertedIndexUtil { | Ported | //
dict_compression now silently ignores by V2/V3 inverted index |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
| @@ -34,0 +35 @@ import org.apache.doris.qe.GlobalVariable; | Ported |
import org.apache.doris.thrift.TInvertedIndexFileStorageFormat; |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
| @@ -239,0 +241,8 @@ public class CatalogRecycleBin extends
MasterDaemon implements Writable { | Ported | return
recyclePartition(dbId, tableId, tableName, partition, range,
listPartitionItem, dataProperty,; replicaAlloc, isInMemory |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
| @@ -249 +258,2 @@ public class CatalogRecycleBin extends MasterDaemon
implements Writable { | Ported | range, listPartitionItem, dataProperty,
replicaAlloc, isInMemory, isMutable,; invertedIndexFileStorageFormat); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
| @@ -1077,0 +1088,4 @@ public class CatalogRecycleBin extends
MasterDaemon implements Writable { | Ported | if
(recoverPartitionInfo.getInvertedIndexFileStorageFormat() != null) {;
partitionInfo.setInvertedIndexFileStorageFormat(p |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
| @@ -1143,0 +1158,4 @@ public class CatalogRecycleBin extends
MasterDaemon implements Writable { | Ported | if
(recyclePartitionInfo.getInvertedIndexFileStorageFormat() != null) {;
partitionInfo.setInvertedIndexFileStorage |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
| @@ -1794,0 +1813,2 @@ public class CatalogRecycleBin extends
MasterDaemon implements Writable { | Ported | @SerializedName("iifsf");
private TInvertedIndexFileStorageFormat invertedIndexFileStorageFormat;
|
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
| @@ -1803 +1823,2 @@ public class CatalogRecycleBin extends
MasterDaemon implements Writable { | Ported | boolean isInMemory,
boolean isMutable,; TInvertedIndexFileStorageFormat invert |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
| @@ -1812,0 +1834 @@ public class CatalogRecycleBin extends
MasterDaemon implements Writable { | Ported |
this.invertedIndexFileStorageFormat = invertedIndexFileStorageFormat; |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
| @@ -1850,0 +1873,4 @@ public class CatalogRecycleBin extends
MasterDaemon implements Writable { | Ported | public
TInvertedIndexFileStorageFormat getInvertedIndexFileStorageFormat() {;
return invertedIndexFileStorageFormat; |
| 66475 | fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java |
@@ -4024,0 +4025,7 @@ public class Env { | Ported | if
(olapTable.getTableProperty() != null; &&
olapTable.getTableProperty().getPartitionInvertedIndexFileStorageFormat()
!= null |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java | @@
-1392 +1392,2 @@ public class OlapTable extends Table implements
MTMVRelatedTableIf, GsonPostProc | Ported |
recyclePartitionParam.isMutable,;
recyclePartitionParam.invertedIndexFileStorageFormat); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java | @@
-1412 +1413,2 @@ public class OlapTable extends Table implements
MTMVRelatedTableIf, GsonPostProc | Ported |
recyclePartitionParam.isMutable,;
recyclePartitionParam.invertedIndexFileStorageFormat); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java | @@
-1431 +1433,2 @@ public class OlapTable extends Table implements
MTMVRelatedTableIf, GsonPostProc | Ported |
recyclePartitionParam.isMutable,;
recyclePartitionParam.invertedIndexFileStorageFormat); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java | @@
-2242,0 +2246,2 @@ public class OlapTable extends Table implements
MTMVRelatedTableIf, GsonPostProc | Ported |
recyclePartitionParam.invertedIndexFileStorageFormat =;
partitionInfo.getInvertedIndexFileStorageFormat(partition.getId()); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java | @@
-2283,0 +2289,2 @@ public class OlapTable extends Table implements
MTMVRelatedTableIf, GsonPostProc | Ported |
partitionInfo.setInvertedIndexFileStorageFormat(newPartition.getId(),;
getPartitionInvertedIndexFileStorageFormat()); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java | @@
-3217,0 +3225,7 @@ public class OlapTable extends Table implements
MTMVRelatedTableIf, GsonPostProc | Ported | public void
setPartitionInvertedIndexFileStorageFormat(;
TInvertedIndexFileStorageFormat invertedIndexFileStorageFormat) { |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java | @@
-3231,0 +3246,17 @@ public class OlapTable extends Table implements
MTMVRelatedTableIf, GsonPostProc | Ported | public
TInvertedIndexFileStorageFormat
getPartitionInvertedIndexFileStorageFormat() {; if
(!Config.enable_partition_inverted_index_storage |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionInfo.java |
@@ -29,0 +30 @@ import org.apache.doris.common.DdlException; | Ported |
import org.apache.doris.thrift.TInvertedIndexFileStorageFormat; |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionInfo.java |
@@ -71,0 +73,3 @@ public class PartitionInfo { | Ported | // partition
id -> resolved base inverted-index storage format;
@SerializedName("IdToInvertedIndexFileStorageFormat") |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionInfo.java |
@@ -100,0 +105 @@ public class PartitionInfo { | Ported |
this.idToInvertedIndexFileStorageFormat = new HashMap<>(); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionInfo.java |
@@ -110,0 +116 @@ public class PartitionInfo { | Ported |
this.idToInvertedIndexFileStorageFormat = new HashMap<>(); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionInfo.java |
@@ -314,0 +321,13 @@ public class PartitionInfo { | Ported | public
TInvertedIndexFileStorageFormat getInvertedIndexFileStorageFormat(long
partitionId) {; return idToInvertedIndexFileStorageFormat == |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionInfo.java |
@@ -365,0 +385,3 @@ public class PartitionInfo { | Ported | if
(idToInvertedIndexFileStorageFormat != null) {;
idToInvertedIndexFileStorageFormat.remove(partitionId); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionInfo.java |
@@ -425,0 +448,2 @@ public class PartitionInfo { | Ported | Map<Long,
TInvertedIndexFileStorageFormat> origIdToInvertedIndexFileStorageFormat
=; idToInvertedIndexFileStorageFormat; |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionInfo.java |
@@ -430,0 +455 @@ public class PartitionInfo { | Ported |
idToInvertedIndexFileStorageFormat = Maps.newHashMap(); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionInfo.java |
@@ -459,0 +485,7 @@ public class PartitionInfo { | Ported | if
(origIdToInvertedIndexFileStorageFormat != null) {;
TInvertedIndexFileStorageFormat invertedIndexFileStorageFormat = |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/RecyclePartitionParam.java
| whole file | Ported(verbatim) | |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/catalog/TableProperty.java |
@@ -764,0 +765,5 @@ public class TableProperty implements
GsonPostProcessable { | Ported | public TInvertedIndexFileStorageFormat
getPartitionInvertedIndexFileStorageFormat() {; String format =
properties.get(PropertyAnalyzer.PRO |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandler.java
| @@ -43,0 +44 @@ import org.apache.doris.system.Backend; | Ported |
import org.apache.doris.thrift.TInvertedIndexFileStorageFormat; |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandler.java
| @@ -55,0 +57 @@ import java.util.ArrayList; | Ported | import
java.util.HashMap; |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandler.java
| @@ -132,0 +135 @@ public class CloudSchemaChangeHandler extends
SchemaChangeHandler { | Ported |
add(PropertyAnalyzer.PROPERTIES_PARTITION_INVERTED_INDEX_STORAGE_FORMAT);
|
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandler.java
| @@ -147,0 +151,19 @@ public class CloudSchemaChangeHandler extends
SchemaChangeHandler { | Ported | if
(properties.containsKey(PropertyAnalyzer.PROPERTIES_PARTITION_INVERTED_INDEX_STORAGE_FORMAT))
{; TInvertedIndexFileStorageForma |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandler.java
| @@ -383,0 +406,2 @@ public class CloudSchemaChangeHandler extends
SchemaChangeHandler { | Ported | } else if
(properties.containsKey(PropertyAnalyzer.PROPERTIES_PARTITION_INVERTED_INDEX_STORAGE_FORMAT))
{; // Existing tablet meta |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/cloud/datasource/CloudInternalCatalog.java
| @@ -326,0 +327,2 @@ public class CloudInternalCatalog extends
InternalCatalog { | Ported | TInvertedIndexFileStorageFormat
partitionInvertedIndexFileStorageFormat =;
tbl.getPartitionInvertedIndexFileStorageFormat(); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/cloud/datasource/CloudInternalCatalog.java
| @@ -406 +408 @@ public class CloudInternalCatalog extends
InternalCatalog { | Ported | indexId == tbl.getBaseIndexId() ?
partitionInvertedIndexFileStorageFormat : null, |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/cloud/datasource/CloudInternalCatalog.java
| @@ -649,0 +652,6 @@ public class CloudInternalCatalog extends
InternalCatalog { | Ported | // schema KV is shared by (index_id,
schema_version). Persist the physical tablet format; // separately only
after partition-level for |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/cloud/datasource/CloudInternalCatalog.java
| @@ -685,0 +694,4 @@ public class CloudInternalCatalog extends
InternalCatalog { | Ported | if
(Config.enable_partition_inverted_index_storage_format_rollout; &&
schema.hasInvertedIndexStorageFormat()) { |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/common/proc/PartitionsProcDir.java
| @@ -117 +117 @@ public class PartitionsProcDir implements
ProcDirInterface { | Adapted |
.add("RowCount").add("BinlogSize").add("InvertedIndexStorageFormat");
The target has no BinlogSize column; the new format column follows
RowCount. |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/common/proc/PartitionsProcDir.java
| @@ -640,0 +641,5 @@ public class PartitionsProcDir implements
ProcDirInterface { | Ported | String invertedIndexStorageFormat =;
olapTable.getInvertedIndexFileStorageFormatForPartition(partitionId).name
|
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
| @@ -129,0 +130,3 @@ public class PropertyAnalyzer { | Ported | public
static final String PROPERTIES_PARTITION_INVERTED_INDEX_STORAGE_FORMAT
=; "partition.inverted_index_storage_format"; |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
| @@ -1262,0 +1266,25 @@ public class PropertyAnalyzer { | Adapted |
public static TInvertedIndexFileStorageFormat
analyzePartitionInvertedIndexFileStorageFormat(; Map<String, String>
properties) throws ; The later #67583 validation fix modifies this
source hunk. |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -1500,0 +1501 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported | TInvertedIndexFileStorageFormat
partitionInvertedIndexFileStorageFormat; |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -1682,0 +1684 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported | partitionInvertedIndexFileStorageFormat
= olapTable.getPartitionInvertedIndexFileStorageFormat(); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -1768,0 +1771,3 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported | } else if
(olapTable.getPartitionInvertedIndexFileStorageFormat(); !=
partitionInvertedIndexFileStorageFormat) |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -1815,0 +1821,2 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported |
partitionInfo.setInvertedIndexFileStorageFormat(partitionId,;
partitionInvertedIndexFileStorageFormat); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -1823 +1830,2 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported | isTempPartition,
partitionInfo.getIsMutable(partitionId),;
partitionInvertedIndexFileStorageFormat); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -1828 +1836,2 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported | isTempPartition,
partitionInfo.getIsMutable(partitionId),;
partitionInvertedIndexFileStorageFormat); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -1833 +1842,2 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported | isTempPartition,
partitionInfo.getIsMutable(partitionId),;
partitionInvertedIndexFileStorageFormat); |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -1981,0 +1992,4 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported | if
(info.getInvertedIndexFileStorageFormat() != null) {;
partitionInfo.setInvertedIndexFileStorageFormat(partition.getId() |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -2672,0 +2687,11 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported | TInvertedIndexFileStorageFormat
partitionInvertedIndexFileStorageFormat;; try { |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -3192,0 +3218,2 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported |
partitionInfo.setInvertedIndexFileStorageFormat(partitionId,;
olapTable.getPartitionInvertedIndexFileStorageFo |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -3284,0 +3312,2 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported |
partitionInfo.setInvertedIndexFileStorageFormat(partition.getId(),;
olapTable.getPartitionInvertedInde |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
| @@ -3762,0 +3792,3 @@ public class InternalCatalog implements
CatalogIf<Database> { | Ported | } else if
(olapTable.getPartitionInvertedIndexFileStorageFormat(); !=
copiedTbl.getPartitionInvertedIndexFileStorageFo |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ShowPartitionIdCommand.java
| whole file | Ported(verbatim) | |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java
| @@ -808,0 +809,7 @@ public class CreateTableInfo { | Ported | try {;
invertedIndexFileStorageFormat = |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java
| @@ -812,5 +819,7 @@ public class CreateTableInfo { | Ported | if
(invertedIndexFileStorageFormat == null) {; try { |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/ModifyTablePropertiesOp.java
| @@ -37,0 +38 @@ import com.google.common.base.Strings; | Ported |
import java.util.HashMap; |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/ModifyTablePropertiesOp.java
| @@ -283,0 +285,3 @@ public class ModifyTablePropertiesOp extends
AlterTableOp { | Ported | } else if
(properties.containsKey(PropertyAnalyzer.PROPERTIES_PARTITION_INVERTED_INDEX_STORAGE_FORMAT))
{; PropertyAnalyzer.analyz |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/persist/PartitionPersistInfo.java
| whole file | Ported(verbatim) | |
| 66475 |
fe/fe-core/src/main/java/org/apache/doris/tablefunction/PartitionsTableValuedFunction.java
| @@ -88 +88,4 @@ public class PartitionsTableValuedFunction extends
MetadataTableValuedFunction { | Ported | new Column("UnsyncTables",
ScalarType.createStringType()),; new Column("CommittedVersion",
ScalarType.createType(PrimitiveType |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/alter/CloudSchemaChangeJobV2Test.java
| whole file | Ported(verbatim) | |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/catalog/CatalogRecycleBinTest.java
| @@ -29,0 +30 @@ import
org.apache.doris.nereids.trees.expressions.functions.FunctionBuilder; |
Ported | import org.apache.doris.thrift.TInvertedIndexFileStorageFormat;
|
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/catalog/CatalogRecycleBinTest.java
| @@ -480 +481,2 @@ public class CatalogRecycleBinTest extends
TestWithFeService { | Ported | false,;
TInvertedIndexFileStorageFormat.SNII |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/catalog/CatalogRecycleBinTest.java
| @@ -485,0 +488,2 @@ public class CatalogRecycleBinTest extends
TestWithFeService { | Adapted |
Assertions.assertEquals(TInvertedIndexFileStorageFormat.SNII,
olapTable.getPartitionInfo(); .getInvertedIndexFileStorageFormat; The
target test uses JUnit Assert instead of Assertions. |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/catalog/CreateTableTest.java |
@@ -28,0 +29 @@ import org.apache.doris.resource.Tag; | Ported | import
org.apache.doris.thrift.TInvertedIndexFileStorageFormat; |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/catalog/CreateTableTest.java |
@@ -273,0 +275,17 @@ public class CreateTableTest extends
TestWithFeService { | Ported | @test; public void
testPartitionsStoreTableInvertedIndexStorageFormat() throws Exception {
|
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java |
@@ -25,0 +26,2 @@ import org.apache.doris.common.Config; | Ported |
import org.apache.doris.common.ConfigBase;; import
org.apache.doris.common.ConfigException; |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java |
@@ -31,0 +34 @@ import org.apache.doris.qe.SessionVariable; | Ported |
import org.apache.doris.thrift.TInvertedIndexFileStorageFormat; |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java |
@@ -44,0 +48,2 @@ import java.io.IOException; | Ported | import
java.nio.file.Files;; import java.nio.file.Path; |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java |
@@ -54,0 +60,86 @@ public class OlapTableTest { | Ported | @test; public
void testPartitionFormatChangeDoesNotChangeLogicalSchemaVersion() { |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/catalog/TablePropertyTest.java
| @@ -22,0 +23 @@ import org.apache.doris.resource.Tag; | Ported |
import org.apache.doris.thrift.TInvertedIndexFileStorageFormat; |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/catalog/TablePropertyTest.java
| @@ -43,0 +45,10 @@ public class TablePropertyTest { | Ported | @test;
public void testPartitionInvertedIndexStorageFormat() { |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandlerTest.java
| @@ -31,0 +32,2 @@ import org.apache.doris.common.util.DebugPointUtil;
| Ported | import org.apache.doris.common.util.DynamicPartitionUtil;;
import org.apache.doris.common.util.PropertyAnalyzer; |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandlerTest.java
| @@ -35,0 +38 @@ import org.apache.doris.system.SystemInfoService; |
Ported | import org.apache.doris.thrift.TInvertedIndexFileStorageFormat;
|
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandlerTest.java
| @@ -439,0 +443,88 @@ public class CloudSchemaChangeHandlerTest { |
Ported | @test; public void
testUpdatePartitionInvertedIndexStorageFormatDoesNotScanPartitions()
throws Exception { |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandlerTest.java
| @@ -477,0 +569 @@ public class CloudSchemaChangeHandlerTest { | Ported
| |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/cloud/datasource/CloudInternalCatalogTest.java
| @@ -0,0 +1,84 @@ | Adapted | // Licensed to the Apache Software
Foundation (ASF) under one; // or more contributor license agreements.
See the NOTICE file; The target already had this class; the source test
uses the target builder signature. |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/common/PropertyAnalyzerTest.java
| @@ -390,0 +391,42 @@ public class PropertyAnalyzerTest { | Adapted | ;
@test; The later #67583 validation fix modifies this test. |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/CreateTableCommandTest.java
| @@ -34,0 +35 @@ import org.apache.doris.catalog.Type; | Ported |
import org.apache.doris.common.Config; |
| 66475 |
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/CreateTableCommandTest.java
| @@ -244,0 +246,21 @@ public class CreateTableCommandTest extends
TestWithFeService { | Ported | @test; public void
testPartitionInvertedIndexStorageFormatRejectedInLocalMode() { |
| 66475 |
regression-test/data/external_table_p0/tvf/test_partitions_tvf.out |
whole file | Ported(verbatim) | |
| 66475 |
regression-test/suites/cloud_p0/test_partition_cloud_add_build_index_validation.groovy
| whole file | Adapted | ; The later #68449 repair removes unsupported
V1 cases. |
| 66475 |
regression-test/suites/cloud_p0/test_partition_cloud_inverted_index_format.groovy
| whole file | Ported(verbatim) | |
| 66475 |
regression-test/suites/cloud_p0/test_partition_cloud_inverted_index_storage_format_meta_write_switch.groovy
| whole file | Ported(verbatim) | |
| 66475 |
regression-test/suites/cloud_p0/test_partition_cloud_inverted_index_storage_format_recycle_truncate.groovy
| whole file | Ported(verbatim) | |
| 66475 |
regression-test/suites/cloud_p0/test_partition_cloud_inverted_index_storage_format_rollout.groovy
| @@ -0,0 +1,507 @@ | Adapted | // Licensed to the Apache Software
Foundation (ASF) under one; // or more contributor license agreements.
See the NOTICE file; The later #67583 validation fix modifies this
suite. |
| 66475 |
regression-test/suites/cloud_p0/test_partition_cloud_inverted_index_storage_format_rowset_schema.groovy
| whole file | Ported(verbatim) | |
| 66475 |
regression-test/suites/cloud_p0/test_partition_cloud_inverted_index_storage_format_stream_load.groovy
| whole file | Ported(verbatim) | |
| 67583 |
fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
| @@ -1298,5 +1298,9 @@ public class PropertyAnalyzer { | Ported |
String formatValue =
properties.remove(PROPERTIES_PARTITION_INVERTED_INDEX_STORAGE_FORMAT);;
// "default" resolves to the FE config in |
| 67583 |
fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
| @@ -1305 +1309,4 @@ public class PropertyAnalyzer { | Ported | ;
Map<String, String> formatProperties = new HashMap<>(); |
| 67583 |
fe/fe-core/src/test/java/org/apache/doris/common/PropertyAnalyzerTest.java
| @@ -400,2 +400,2 @@ public class PropertyAnalyzerTest { | Ported |
properties.put(PropertyAnalyzer.PROPERTIES_PARTITION_INVERTED_INDEX_STORAGE_FORMAT,
"V3");; Assertions.assertEquals(TInvertedI |
| 67583 |
fe/fe-core/src/test/java/org/apache/doris/common/PropertyAnalyzerTest.java
| @@ -403,0 +404,5 @@ public class PropertyAnalyzerTest { | Ported |
properties.put(PropertyAnalyzer.PROPERTIES_PARTITION_INVERTED_INDEX_STORAGE_FORMAT,
"default");; AnalysisException defaultExce |
| 67583 |
regression-test/suites/cloud_p0/test_partition_cloud_inverted_index_storage_format_rollout.groovy
| whole file | Adapted | Release branches limit table names to 64
characters; the validation case uses a shorter name. |
| 68449 |
regression-test/suites/cloud_p0/test_partition_cloud_add_build_index_validation.groovy
| @@ -42 +41,0 @@
suite("test_partition_cloud_add_build_index_validation", "p0, docker") {
| Ported | Removed unsupported V1 case omitted from #68618 backport. |
| 68449 |
regression-test/suites/cloud_p0/test_partition_cloud_add_build_index_validation.groovy
| @@ -46 +44,0 @@
suite("test_partition_cloud_add_build_index_validation", "p0, docker") {
| Ported | Removed unsupported V1 case omitted from #68618 backport. |
| 68449 |
regression-test/suites/cloud_p0/test_partition_cloud_add_build_index_validation.groovy
| @@ -75,28 +72,0 @@
suite("test_partition_cloud_add_build_index_validation", "p0, docker") {
| Ported | Removed unsupported V1 case omitted from #68618 backport. |
| 68449 |
regression-test/suites/cloud_p0/test_partition_cloud_add_build_index_validation.groovy
| @@ -173,35 +142,0 @@
suite("test_partition_cloud_add_build_index_validation", "p0, docker") {
| Ported | Removed unsupported V1 case omitted from #68618 backport. |
| 11528 |
fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyTablePropertiesClause.java
| @@ -35,0 +36 @@ import com.google.common.base.Strings; | Ported | Keep
the legacy analyzer available on both release branches. |
| 11528 |
fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyTablePropertiesClause.java
| @@ -273,0 +275,4 @@ public class ModifyTablePropertiesClause extends
AlterTableClause { | Ported | Validate the property without consuming it
and select synchronous alteration. |
| Local |
fe/fe-core/src/test/java/org/apache/doris/analysis/ModifyTablePropertiesClauseTest.java
| whole file | Adapted | New focused test covers legacy acceptance and
rejection. |
| Local |
fe/fe-core/src/test/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandlerTest.java
| follow-up test hunk | Adapted | Mock cloud mode while rollout is
disabled so validation reaches the intended branch. |
| Local |
regression-test/suites/cloud_p0/test_partition_cloud_inverted_index_storage_format_rollout.groovy
| follow-up test hunk | Adapted | Shorten the rejected-default table
name to 50 characters on release branches. |

| Local | be/test/storage/rowset/beta_rowset_test.cpp | follow-up
include | Adapted | Include the writer definition explicitly because
release headers do not provide it transitively. |
| Local | cloud/test/recycler_test.cpp | follow-up declaration | Adapted
| Use explicit pointer and bool literals in the added Recycler test. |

### Release note

Cloud OLAP tables can roll out inverted-index storage formats by
partition.

The merge-tree clang-format 16 and FE Checkstyle preflight passed.
Direct LLVM 20 clang-tidy reports an unmatched `NOLINTEND` in unchanged
`be/src/core/types.h:578` and existing function-complexity findings; the
repository marks its clang-tidy review job as nonrequired.

### Check List (For Author)

- Test
- [x] Regression test: Seven cloud_p0 Docker suites passed against the
branch-built FE, BE, and Meta Service image.
- [x] Unit Test: 15 FE, 7 BE, and 1 Cloud Recycler targeted tests
passed.
- [x] Manual test: Protobuf tags match Doris master, both release
branches, and selectdb-core 4.1; protoc and the full FE and BE cloud
builds passed.
- Behavior changed:
- [x] Yes. Cloud partitions can retain separate physical inverted-index
formats.
- Does this need documentation?
    - [x] No.

### Check List (For Reviewer who merge this PR)

- [ ] Confirm release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label

---------

Co-authored-by: hoshinojyunn <[email protected]>
Co-authored-by: Chenyang Sun <[email protected]>
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
Co-authored-by: chunping <[email protected]>