Skip to content

SQL uniqState memory leak test#4277

Closed
Kozloff wants to merge 1 commit intoClickHouse:masterfrom
Kozloff:uniqstate-memory-leak
Closed

SQL uniqState memory leak test#4277
Kozloff wants to merge 1 commit intoClickHouse:masterfrom
Kozloff:uniqstate-memory-leak

Conversation

@Kozloff
Copy link
Copy Markdown

@Kozloff Kozloff commented Feb 5, 2019

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=ru

Category (leave one):

  • Build/Testing/Packaging Improvement

Short description (up to few sentences):

Memory leak when memory limit (for query) exceeded

Detailed description (optional):

ClickHouse: v19.1.6
How to reproduce: run query with low memory limit to invoke exception, then a memory leak occurs.
Query:

  • uniq() function
  • -State combinators
  • GROUP BY high cardinality key

Example:

  • create aggregation table (engine: SummingMergeTree) from main table with UniqUsers column

2

  • run 4 times query with low memory
  • run 1 more time with enough memory
query_start_time query_duration memory_usage exception
2019-02-05 16:19:24 242676 9.31 GiB Code: 241, e.displayText() = DB::Exception: Memory limit (for query) exceeded: would use 9.31 GiB
2019-02-05 16:53:57 254514 9.31 GiB Code: 241, e.displayText() = DB::Exception: Memory limit (for query) exceeded: would use 9.31 GiB
2019-02-05 17:19:10 233147 9.31 GiB Code: 241, e.displayText() = DB::Exception: Memory limit (for query) exceeded: would use 9.31 GiB
2019-02-05 17:42:44 237689 9.31 GiB Code: 241, e.displayText() = DB::Exception: Memory limit (for query) exceeded: would use 9.31 GiB
2019-02-05 18:30:09 518513 16.53 GiB Successful

@alexey-milovidov alexey-milovidov self-assigned this Feb 5, 2019
@alexey-milovidov
Copy link
Copy Markdown
Member

alexey-milovidov commented Feb 6, 2019

Confirmed.

It can be easily reproduced by the following command:

for i in {1..1000}; do clickhouse-client --max_memory_usage 100000000 <<< "SELECT uniqExactState(number) FROM system.numbers GROUP BY number % 1000"; done

@akuzm akuzm self-assigned this Jul 1, 2019
@stale stale bot added the not planned Known issue, no plans to fix it currenlty label Oct 20, 2019
@blinkov blinkov added pr-build Pull request with build/testing/packaging improvement and removed not planned Known issue, no plans to fix it currenlty labels Oct 20, 2019
@abyss7
Copy link
Copy Markdown
Contributor

abyss7 commented Jan 22, 2020

Do we have an issue on this? Have we fixed this already?

@akuzm
Copy link
Copy Markdown
Contributor

akuzm commented Jan 22, 2020

Do we have an issue on this? Have we fixed this already?

Not sure, personally I haven't done anything about this.

@qoega qoega mentioned this pull request Feb 4, 2020
Closed
@qoega
Copy link
Copy Markdown
Member

qoega commented Feb 4, 2020

I made an issue about this. Thank you for your your report.
#8995

@qoega qoega closed this Feb 4, 2020
@ClickHouse ClickHouse deleted a comment from stale bot May 18, 2020
@alexey-milovidov
Copy link
Copy Markdown
Member

alexey-milovidov commented May 18, 2020

@qoega Why closed? We have to fix the issue and integrate this test with author's commit.
I will reopen and merge when the issue will be fixed.

alexey-milovidov added a commit that referenced this pull request May 18, 2020
@alexey-milovidov
Moved test #4277 to bugs
738842b
@alexey-milovidov
Copy link
Copy Markdown
Member

alexey-milovidov commented May 18, 2020

I have merged it manually: efab707

@alexey-milovidov
Copy link
Copy Markdown
Member

alexey-milovidov commented May 18, 2020

And we appreciate adding more tests to bugs directory!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@alexey-milovidov alexey-milovidov

Labels

pr-build Pull request with build/testing/packaging improvement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Kozloff @alexey-milovidov @abyss7 @akuzm @qoega @blinkov