Skip to content

Add max_execution_speed & max_execution_bytes_speed & min_execution_bytes_speed settings#4430

Merged
alexey-milovidov merged 4 commits intoClickHouse:masterfrom
zhang2014:feature/limit_execution
Mar 4, 2019
Merged

Add max_execution_speed & max_execution_bytes_speed & min_execution_bytes_speed settings#4430
alexey-milovidov merged 4 commits intoClickHouse:masterfrom
zhang2014:feature/limit_execution

Conversation

@zhang2014
Copy link
Copy Markdown
Contributor

@zhang2014 zhang2014 commented Feb 18, 2019

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Category (leave one):

  • New Feature

Short description:

In some case, we might need to cap the query to limit its use of computing resource

@zhang2014 zhang2014 changed the title Add max_execution_speed & max_execution_bytes_speed & min_execution_bytes_speed Settings Add max_execution_speed & max_execution_bytes_speed & min_execution_bytes_speed settings Feb 18, 2019
@zhang2014
Copy link
Copy Markdown
Contributor Author

zhang2014 commented Feb 18, 2019 

ClickHouse :) SELECT COUNT() FROM test SETTINGS max_execution_bytes_speed = 30971520;

SELECT COUNT()
FROM test 
SETTINGS max_execution_bytes_speed = 30971520

┌──COUNT()─┐
│ 10000000 │
└──────────┘

1 rows in set. Elapsed: 0.329 sec. Processed 10.00 million rows, 10.00 MB (30.37 million rows/s., 30.37 MB/s.) 

ClickHouse :) SELECT COUNT() FROM test SETTINGS max_execution_bytes_speed = 20971520;

SELECT COUNT()
FROM test 
SETTINGS max_execution_bytes_speed = 20971520

┌──COUNT()─┐
│ 10000000 │
└──────────┘

1 rows in set. Elapsed: 0.477 sec. Processed 10.00 million rows, 10.00 MB (20.96 million rows/s., 20.96 MB/s.) 

ClickHouse :) 
ClickHouse :) 
ClickHouse :) SELECT COUNT() FROM test SETTINGS max_execution_bytes_speed = 10971520;

SELECT COUNT()
FROM test 
SETTINGS max_execution_bytes_speed = 10971520

┌──COUNT()─┐
│ 10000000 │
└──────────┘

1 rows in set. Elapsed: 0.911 sec. Processed 10.00 million rows, 10.00 MB (10.97 million rows/s., 10.97 MB/s.)

zhang2014
zhang2014 commented Feb 18, 2019
View reviewed changes
dbms/src/Interpreters/Settings.h Outdated
Copy link
Copy Markdown
Contributor Author

@zhang2014 zhang2014 Feb 18, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe rename min_execution_speed to min_execution_speed_per_thread is better ? but this may cause version incompatibility. I'm not sure if I should do that.

Copy link
Copy Markdown
Member

@alexey-milovidov alexey-milovidov Mar 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better not to change.

alexey-milovidov
alexey-milovidov reviewed Mar 1, 2019
View reviewed changes
dbms/src/DataStreams/IBlockInputStream.cpp Outdated
Copy link
Copy Markdown
Member

@alexey-milovidov alexey-milovidov Mar 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't use Throttler here? The logic looks the same.

Copy link
Copy Markdown
Contributor Author

@zhang2014 zhang2014 Mar 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because Throttler accepts different bytes value, this will cause the last processed bytes value to be stored in IBlockInputStream

alexey-milovidov
alexey-milovidov reviewed Mar 1, 2019
View reviewed changes
dbms/src/DataStreams/IBlockInputStream.h Outdated
Copy link
Copy Markdown
Member

@alexey-milovidov alexey-milovidov Mar 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

min/max_execution_bytes_speed
-> min/max_execution_speed_bytes

to keep it uniform with other settings.

@zhang2014 zhang2014 force-pushed the feature/limit_execution branch from 80183a7 to fb44b66 Compare March 4, 2019 02:10
@alexey-milovidov alexey-milovidov merged commit b172ad4 into ClickHouse:master Mar 4, 2019
@nvartolomei nvartolomei mentioned this pull request Jun 24, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alexey-milovidov alexey-milovidov alexey-milovidov left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@zhang2014 @alexey-milovidov @alesapin