Skip to content

Expose all GlobalThreadPool config params#31285

Merged
nikitamikhaylov merged 2 commits intoClickHouse:masterfrom
BetterStackHQ:th/expose-all-global-thread-configs
Nov 12, 2021
Merged

Expose all GlobalThreadPool config params#31285
nikitamikhaylov merged 2 commits intoClickHouse:masterfrom
BetterStackHQ:th/expose-all-global-thread-configs

Conversation

@gyfis
Copy link
Copy Markdown
Contributor

@gyfis gyfis commented Nov 11, 2021

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Exposes all GlobalThreadPool configurations to the configuration files.

Detailed description / Documentation draft:

Adds three new configuration options

max_thread_pool_free_size: The max number of threads that are free in the Global Thread pool.
thread_pool_queue_size: The limit to the number of jobs that can be scheduled on the Global Thread pool.
thread_pool_shutdown_on_exception: The flag deciding if the thread pool should shutdown on exception.

Fixes this issue: #31256

@robot-clickhouse robot-clickhouse added doc-alert pr-feature Pull request with new product feature labels Nov 11, 2021
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Nov 11, 2021

CLA assistant check
All committers have signed the CLA.

@gyfis
Copy link
Copy Markdown
Contributor Author

gyfis commented Nov 11, 2021

@nikitamikhaylov Hi! What steps should I take to make the PR mergeable, please?


## max_thread_pool_free_size {#max-thread-pool-free-size}

The number of threads that are free in the Global Thread pool.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to say that this is a number of threads that global pool holds all the time even if there are no compute load. Other threads could be released to OS and then allocated from it once again.


The limit to the number of jobs that can be scheduled on the Global Thread pool.

Default value: 10000.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be noticed that increasing this number will lead to more memory consumption. Better to set it to the value equals to max_thread_pool_size if the latter is changed.

config().getUInt("max_thread_pool_size", 10000),
config().getUInt("max_thread_pool_free_size", 1000),
config().getUInt("thread_pool_queue_size", 10000),
config().getBool("thread_pool_shutdown_on_exception", false)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am convinced that this is bad idea (to allow the user to change this flag).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to be consistent, but I’ll remove it. Thanks!

@nikitamikhaylov
Copy link
Copy Markdown
Member

@gyfis Thanks for PR! But a couple of comments raised...

@gyfis
Copy link
Copy Markdown
Contributor Author

gyfis commented Nov 11, 2021

@gyfis Thanks for PR! But a couple of comments raised...

Thanks! I implemented the recommended changes (plus a ThreadPool.h change I missed)

@nikitamikhaylov nikitamikhaylov merged commit 445b0ba into ClickHouse:master Nov 12, 2021
@gyfis gyfis deleted the th/expose-all-global-thread-configs branch November 12, 2021 13:25
@ka1bi4
Copy link
Copy Markdown
Contributor

ka1bi4 commented Nov 17, 2021

Internal documentation ticket: DOCSUP-18492

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature Pull request with new product feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants