Skip to content

Add a setting to control what to do with indices during mutations#89335

Merged
Algunenano merged 24 commits intoClickHouse:masterfrom
Algunenano:rebuild_secondary_index
Nov 19, 2025
Merged

Add a setting to control what to do with indices during mutations#89335
Algunenano merged 24 commits intoClickHouse:masterfrom
Algunenano:rebuild_secondary_index

Conversation

@Algunenano
Copy link
Copy Markdown
Member

@Algunenano Algunenano commented Oct 31, 2025

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Add a MergeTree setting alter_column_secondary_index_mode to control what to do with indices during mutations. Possible values: throw, drop, rebuild, and compatibility. Closes #77797

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

Details

The setting alter_column_secondary_index_mode controls behavior of secondary indices during mutations:

  • The value throw will prevent doing ALTER COLUMN MODIFY, UPDATE and DELETE of columns covered by secondary indices or if these mutations will require rewriting the index (DELETES always do).
  • The value drop will drop the dependent secondary indices that would otherwise need to be rewritten.
  • The value rebuild rebuilds any indices affected by the mutation commands.
  • The value compatibility keeps the old behaviour: It throws for ALTER COLUMN MODIFY, and it rebuilds for UPDATE or DELETE.

Closes #77797

@Algunenano Algunenano self-assigned this Oct 31, 2025
@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Oct 31, 2025

Workflow [PR], commit [cd4a414]

Summary:

job_name test_name status info comment
Stateless tests (arm_asan, targeted) failure
02241_filesystem_cache_on_write_operations FAIL cidb
Upgrade check (amd_tsan) failure
Killed by signal (in clickhouse-server.log) FAIL cidb
Fatal message in clickhouse-server.log (see fatal_messages.txt) FAIL cidb
Killed by signal (output files) FAIL cidb
Found signal in gdb.log FAIL cidb
BuzzHouse (amd_debug) failure
Logical error: 'Inconsistent AST formatting: the query: FAIL cidb
BuzzHouse (arm_asan) failure
Logical error: 'Part all_1_319_5_1727 intersects next part all_1_3958_23. It is a bug or a result of manual intervention'. FAIL cidb
Integration tests (amd_asan, flaky) error

@clickhouse-gh clickhouse-gh bot added the pr-improvement Pull request with some product improvements label Oct 31, 2025
@Algunenano Algunenano changed the title Rebuild secondary index Add a setting to control what to do with indices during mutations Oct 31, 2025
@Algunenano Algunenano marked this pull request as ready for review November 3, 2025 17:50
@Algunenano Algunenano force-pushed the rebuild_secondary_index branch from dc27321 to 82a8e7e Compare November 3, 2025 19:44
@Algunenano Algunenano removed their assignment Nov 4, 2025
@ahmadov ahmadov self-assigned this Nov 4, 2025
@Algunenano Algunenano requested a review from ahmadov November 10, 2025 17:57
Copy link
Copy Markdown
Member

@ahmadov ahmadov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM.

extern const MergeTreeSettingsBool allow_suspicious_indices;
extern const MergeTreeSettingsBool allow_summing_columns_in_partition_or_order_key;
extern const MergeTreeSettingsBool allow_coalescing_columns_in_partition_or_order_key;
extern const MergeTreeSettingsAlterColumnSecondaryIndexMode alter_column_secondary_index_mode;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[very minor] maybe them after the MergeTreeSettingsBool properties? It makes it easier to read them

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's shorted by alphabetical order of the setting name (last column) but we can change it

{
ASTPtr expr_list = index.expression_list_ast->clone();
for (const auto & expr : expr_list->children)
indices_recalc_expr_list->children.push_back(expr->clone());
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] do we need to clone the expr from the cloned list? can we avoid allocation and just move?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved it from the existing code (in getIndicesToRecalculate), I'll have a look to see if we need to clone of we can simply move it (if deep cloned).

@EmeraldShift
Copy link
Copy Markdown
Contributor

Is the plan to change the default to rebuild?

@Algunenano
Copy link
Copy Markdown
Member Author

Yes, this PR changes the default to rebuild

@Algunenano Algunenano enabled auto-merge November 19, 2025 13:37
@Algunenano Algunenano added this pull request to the merge queue Nov 19, 2025
@Algunenano Algunenano removed this pull request from the merge queue due to a manual request Nov 19, 2025
@Algunenano
Copy link
Copy Markdown
Member Author

25.11 has been released. Need to change the settings history

@Algunenano Algunenano enabled auto-merge November 19, 2025 17:22
@Algunenano Algunenano added this pull request to the merge queue Nov 19, 2025
Merged via the queue into ClickHouse:master with commit 2453a19 Nov 19, 2025
126 of 132 checks passed
@Algunenano Algunenano deleted the rebuild_secondary_index branch November 19, 2025 19:39
@robot-ch-test-poll4 robot-ch-test-poll4 added the pr-synced-to-cloud The PR is synced to the cloud repo label Nov 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow to ALTER and UPDATE columns related to secondary indices

4 participants