Skip to content

Remove thread limiting.#12840

Merged
KochetovNicolai merged 4 commits intomasterfrom
remove-wrong-threads-limiting
Jul 28, 2020
Merged

Remove thread limiting.#12840
KochetovNicolai merged 4 commits intomasterfrom
remove-wrong-threads-limiting

Conversation

@KochetovNicolai
Copy link
Copy Markdown
Member

@KochetovNicolai KochetovNicolai commented Jul 27, 2020

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • Bug Fix

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fix unnecessary limiting for the number of threads for selects from local replica.

@robot-clickhouse robot-clickhouse added the pr-bugfix Pull request with bugfix, not backported by default label Jul 27, 2020
@KochetovNicolai
Copy link
Copy Markdown
Member Author

@azat I have already forgotten why do we limit threads here.
I suppose that this is not needed already, because the number of threads is limited by pipeline itself (it shouldn't be more than max streams for every step).

@alexey-milovidov
Copy link
Copy Markdown
Member

Perf test? (reading small amount of data from a table with large amount of parts)

@alexey-milovidov alexey-milovidov added pr-performance Pull request with some performance improvements pr-must-backport Pull request should be backported intentionally. Use this label with great care! and removed pr-bugfix Pull request with bugfix, not backported by default labels Jul 27, 2020
@robot-clickhouse robot-clickhouse added the pr-bugfix Pull request with bugfix, not backported by default label Jul 27, 2020
@KochetovNicolai
Copy link
Copy Markdown
Member Author

Sure

@azat
Copy link
Copy Markdown
Member

azat commented Jul 27, 2020

@azat I have already forgotten why do we limit threads here.

It had been added in #11114

Quote from the comment to the patch from that PR:

Before this patch pipeline.setMaxThreads() was called with 0 (since
streams.size() == 0, while pipes.size() == 1), which leads that
pipeline.getMaxThreads() returns max_streams.

I suppose that this is not needed already, because the number of threads is limited by pipeline itself (it shouldn't be more than max streams for every step).

Yes, since there is no more deprecated streams (non-processors) in this code.

Perf test? (reading small amount of data from a table with large amount of parts)

@alexey-milovidov FWIW there is a test for this tests/queries/0_stateless/01283_max_threads_simple_query_optimization.sql

@@ -0,0 +1,3 @@
<test>
<query>select sum(number) from remote('127.0.0.{{1|2}}', numbers_mt(1000000000)) group by bitAnd(number, 1)</query>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, not sure that this query covers that case, since this reads lots of rows and also it uses numbers_mt

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right in this case the number of threads was limited to 1.

@KochetovNicolai
Copy link
Copy Markdown
Member Author

KochetovNicolai commented Jul 27, 2020

@azat tests are ok.
What do you think, do we have other reasons to keep this code?

Waiting for perftests now.

@azat
Copy link
Copy Markdown
Member

azat commented Jul 27, 2020

@azat tests are ok.
What do you think, do we have other reasons to keep this code?

Nope, LGTM

@KochetovNicolai KochetovNicolai marked this pull request as ready for review July 28, 2020 09:02
@KochetovNicolai KochetovNicolai merged commit 1f2390b into master Jul 28, 2020
@KochetovNicolai KochetovNicolai deleted the remove-wrong-threads-limiting branch July 28, 2020 09:03
KochetovNicolai added a commit that referenced this pull request Jul 28, 2020
Remove thread limiting.

(cherry picked from commit 1f2390b)
KochetovNicolai added a commit that referenced this pull request Jul 28, 2020
Remove thread limiting.

(cherry picked from commit 1f2390b)
KochetovNicolai added a commit that referenced this pull request Jul 28, 2020
Remove thread limiting.

(cherry picked from commit 1f2390b)
KochetovNicolai added a commit that referenced this pull request Jul 28, 2020
Remove thread limiting.

(cherry picked from commit 1f2390b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix Pull request with bugfix, not backported by default pr-must-backport Pull request should be backported intentionally. Use this label with great care! pr-performance Pull request with some performance improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants