Skip to content

Remove empty parts after they were pruned by TTL [2]#16895

Merged
alesapin merged 3 commits intoClickHouse:masterfrom
CurtizJ:remove-empty-parts
Nov 13, 2020
Merged

Remove empty parts after they were pruned by TTL [2]#16895
alesapin merged 3 commits intoClickHouse:masterfrom
CurtizJ:remove-empty-parts

Conversation

@CurtizJ
Copy link
Copy Markdown
Member

@CurtizJ CurtizJ commented Nov 11, 2020

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Remove empty parts after they were pruned by TTL, mutation, or collapsing merge algorithm.

Detailed description / Documentation draft:
Based on #15511.

Fixes #16629. Fixes #15180. Fixes #5491.

@robot-clickhouse robot-clickhouse added the pr-improvement Pull request with some product improvements label Nov 11, 2020
@CurtizJ
Copy link
Copy Markdown
Member Author

CurtizJ commented Nov 11, 2020

This PR contains also #16875 and should be merged after it.

@CurtizJ CurtizJ changed the title Remove empty parts after they were pruned by TTL Remove empty parts after they were pruned by TTL [2] Nov 11, 2020
@alesapin alesapin self-assigned this Nov 12, 2020
{
if (part->rows_count == 0)
{
ASTPtr literal = std::make_shared<ASTLiteral>(part->name);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quite ugly, better to have separate method for AST, but ok.

clearOldLogs();
clearOldBlocks();
clearOldMutations();
storage.clearEmptyParts();
Copy link
Copy Markdown
Member

@alesapin alesapin Nov 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if multiple replicas will assign DROP_RANGE for the same parts almost at the same time. Will it work well? Maybe we will have some exceptions in logs? I hope everything will work well, just need to add comment.

@alesapin
Copy link
Copy Markdown
Member

Discussed in telegram -- need tiny fixes.

Copy link
Copy Markdown
Member

@alesapin alesapin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alesapin alesapin merged commit 8d3858f into ClickHouse:master Nov 13, 2020
@nikitamikhaylov nikitamikhaylov added pr-must-backport Pull request should be backported intentionally. Use this label with great care! and removed pr-must-backport Pull request should be backported intentionally. Use this label with great care! labels Mar 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements

Projects

None yet

4 participants