Skip to content

Provide hint for loading uncommitted blocks in merge predicate#43675

Merged
tavplubix merged 6 commits intomasterfrom
merge_predicate_uncommitted_blocks_hint
Nov 29, 2022
Merged

Provide hint for loading uncommitted blocks in merge predicate#43675
tavplubix merged 6 commits intomasterfrom
merge_predicate_uncommitted_blocks_hint

Conversation

@tavplubix
Copy link
Copy Markdown
Member

@tavplubix tavplubix commented Nov 25, 2022 •
edited
Loading

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in official stable or prestable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Optimized number of List requests to ZooKeeper when selecting a part to merge. Previously it could produce thousands of requests in some cases. Fixes #43647

@robot-ch-test-poll1 robot-ch-test-poll1 added the pr-bugfix Pull request with bugfix, not backported by default label Nov 25, 2022
@antonio2368 antonio2368 self-assigned this Nov 28, 2022
antonio2368
antonio2368 reviewed Nov 28, 2022
View reviewed changes
antonio2368
antonio2368 approved these changes Nov 29, 2022
View reviewed changes
@tavplubix
Copy link
Copy Markdown
Member Author

tavplubix commented Nov 29, 2022

Bugfix validate check - OK (it's not trivial to add a test)
Integration tests (release) [3/4] - fixed in master
Stateless tests (tsan) [4/5] - #43659
Stress test (asan) - #43800
Stress test (debug) - #42262
Stress test (msan) - #41369
Stress test (ubsan) - #43445

@tavplubix tavplubix added the skip mergeable check The label to omit checks for required statuses. Does not help with `pr-feature` new docs check label Nov 29, 2022
@tavplubix tavplubix merged commit 24940ad into master Nov 29, 2022
@tavplubix tavplubix deleted the merge_predicate_uncommitted_blocks_hint branch November 29, 2022 14:55
@robot-clickhouse robot-clickhouse mentioned this pull request Nov 29, 2022
Merged
@robot-ch-test-poll robot-ch-test-poll mentioned this pull request Nov 29, 2022
Merged
robot-clickhouse pushed a commit that referenced this pull request Nov 29, 2022
Backport #43675 to 22.11: Provide hint for loading uncommitted blocks…
812e082 
… in merge predicate
@robot-ch-test-poll robot-ch-test-poll added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label Nov 29, 2022
tavplubix added a commit that referenced this pull request Nov 30, 2022
@tavplubix
Merge pull request #43807 from ClickHouse/backport/22.11/43675
28f72d8 
Backport #43675 to 22.11: Provide hint for loading uncommitted blocks in merge predicate
@azat azat mentioned this pull request Jan 24, 2023
Merged
@NickStepanov
Copy link
Copy Markdown

NickStepanov commented Feb 26, 2023

Hi, was this fix included in 22.12 or just backported to 22.11? We upgraded to 22.12 from 22.8 and the amount of list requests has killed the Zookeeper performance for us. Which versions is the fix in?

@den-crane
Copy link
Copy Markdown
Contributor

den-crane commented Feb 26, 2023

@NickStepanov #42878 (comment)
This patch does not fix the issue completely. There is no solution.

@NickStepanov
Copy link
Copy Markdown

NickStepanov commented Feb 28, 2023 •
edited
Loading

@tavplubix are there any plans to fix the issue completely?

@tavplubix
Copy link
Copy Markdown
Member Author

tavplubix commented Feb 28, 2023

Hi, was this fix included in 22.12 or just backported to 22.11?

It was included in 22.12 and backported to 22.11.

@tavplubix are there any plans to fix the issue completely?

See #42878 (comment) (and let's continue in #42878). There are no plans currently, but I will prioritize it and will try to fix #31919

@NickStepanov
Copy link
Copy Markdown

NickStepanov commented May 4, 2023

@tavplubix are there any plans to fix the issue completely?
Maybe this can be given a priority now?

@raosong raosong mentioned this pull request Jun 11, 2023
Closed
@tavplubix
Copy link
Copy Markdown
Member Author

tavplubix commented Jun 12, 2023

@NickStepanov, see #42878 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@antonio2368 antonio2368 antonio2368 approved these changes

Assignees

@antonio2368 antonio2368

Labels

pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-bugfix Pull request with bugfix, not backported by default skip mergeable check The label to omit checks for required statuses. Does not help with `pr-feature` new docs check

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

High amount of ZooKeeper requests/CPU Load after upgrade to 22.11.1 from 22.3

6 participants

@tavplubix @NickStepanov @den-crane @antonio2368 @robot-ch-test-poll @robot-ch-test-poll1