Skip to content

Merging #11022#11097

Merged
alexey-milovidov merged 7 commits intomasterfrom
azat-memory-tracking-2
May 21, 2020
Merged

Merging #11022#11097
alexey-milovidov merged 7 commits intomasterfrom
azat-memory-tracking-2

Conversation

@alexey-milovidov
Copy link
Copy Markdown
Member

#11022

Changelog category (leave one):

  • Non-significant (changelog entry is not required)

azat and others added 5 commits May 21, 2020 02:06
…m Aggregator (TCP)

Example of such cases:
- SELECT GROUP BY LIMIT
- SELECT GROUP BY with subsequent MEMORY_LIMIT_EXCEEDED error

And it should be two-level aggregation, since otherwise there will be
only one hashtable which will be cleared correctly, only if you have
two-level GROUP BY some of hashtables will not be cleared since nobody
consume rows.

Before this patch:
    09:39.015292 [ 3070801 ] {609a0610-e377-4132-9cf3-f49454cf3c96} <Information> executeQuery: Read 1000000 rows, 7.63 MiB in 0.707 sec., 1413826 rows/sec., 10.79 MiB/sec.
    09:39.015348 [ 3070801 ] {609a0610-e377-4132-9cf3-f49454cf3c96} <Debug> MemoryTracker: Peak memory usage (for query): 51.93 MiB.
    09:39.015942 [ 3070801 ] {} <Trace> Aggregator: Destroying aggregate states <-- **problem**
    09:39.017057 [ 3070801 ] {} <Trace> Aggregator: Destroying aggregate states <--
    09:39.017961 [ 3070801 ] {} <Debug> MemoryTracker: Peak memory usage (for query): 51.93 MiB.
    09:39.018029 [ 3070801 ] {} <Information> TCPHandler: Processed in 0.711 sec.

After this patch:
    16:24.544030 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Information> executeQuery: Read 1000000 rows, 7.63 MiB in 0.599 sec., 1670199 rows/sec., 12.74 MiB/sec.
    16:24.544084 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Debug> MemoryTracker: Peak memory usage (for query): 72.11 MiB.
    16:24.544398 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Trace> Aggregator: Destroying aggregate states
    16:24.545485 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Trace> Aggregator: Destroying aggregate states
    16:24.547053 [ 3087333 ] {} <Debug> MemoryTracker: Peak memory usage (for query): 72.11 MiB.
    16:24.547093 [ 3087333 ] {} <Information> TCPHandler: Processed in 0.603 sec.
- GROUP BY
- GROUP BY LIMIT
- GROUP BY MEMORY_LIMIT_EXCEEDED
@blinkov blinkov added the pr-not-for-changelog This PR should not be mentioned in the changelog label May 21, 2020
@alexey-milovidov alexey-milovidov merged commit 2d94332 into master May 21, 2020
@alexey-milovidov alexey-milovidov deleted the azat-memory-tracking-2 branch May 21, 2020 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-docs-needed pr-not-for-changelog This PR should not be mentioned in the changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants