Skip to content

Fix crash when chaining uniqStates#24523

Merged
nikitamikhaylov merged 4 commits intoClickHouse:masterfrom
Algunenano:i24461
Jun 4, 2021
Merged

Fix crash when chaining uniqStates#24523
nikitamikhaylov merged 4 commits intoClickHouse:masterfrom
Algunenano:i24461

Conversation

@Algunenano
Copy link
Copy Markdown
Member

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • Bug Fix

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Disallow building uniqXXXXStates of other aggregation states

Detailed description / Documentation draft:

Fixes #24461
Affects all releases so I it should be backported but I'm unsure how the process works.

@robot-clickhouse robot-clickhouse added the pr-bugfix Pull request with bugfix, not backported by default label May 26, 2021
@nikitamikhaylov nikitamikhaylov self-assigned this May 26, 2021
@Algunenano
Copy link
Copy Markdown
Member Author

Notes about CI: The unrelated clang failure is fixed in master already (a21e4b3) and the failed stateless test (00910_zookeeper_test_alter_compression_codecs_long) has nothing to do with the changes.

@nikitamikhaylov
Copy link
Copy Markdown
Member

@Mergifyio update

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented May 26, 2021

Command update: success

Branch has been successfully updated

@Algunenano
Copy link
Copy Markdown
Member Author

Hi, does this need any change or improvement? The CI errors seem to be completely unrelated with the changes.

@alexey-milovidov
Copy link
Copy Markdown
Member

@Algunenano CI checks looks almost alright. It is waiting for review.

@Algunenano
Copy link
Copy Markdown
Member Author

Ah ok, thanks!

@alexey-milovidov
Copy link
Copy Markdown
Member

@Algunenano @nikitamikhaylov What was the main reason for the bug?
Why it's not possible to calculate the number of distinct aggregate function states?

@Algunenano
Copy link
Copy Markdown
Member Author

@Algunenano @nikitamikhaylov What was the main reason for the bug?

I couldn't find where the bug is. I expect the bug (or bugs) to be found in a completely different place of where the crashes are happening and after a couple of days of looking into it I couldn't find it and forbit it instead.

I don't know how many people need to actually calculate the distinct states of distintct states and it's more likely done by mistake rather than an actual need (at least it was in our case). But that's a different topic, if CH wanted to support it then the issues would need to be found and addressed.

@alexey-milovidov
Copy link
Copy Markdown
Member

how many people need to actually calculate the distinct states of distintct states and it's more likely done by mistake rather than an actual need

Yes, I also expect that :)

I just in doubt that the fix is just covering the culprit instead of making it clear.
I will try to investigate.

The fix is valuable nevertheless, thank you!
The possibility of crash on user provided query is just unbearable and is the №0 priority.

@Algunenano
Copy link
Copy Markdown
Member Author

I will try to investigate.

Please let me know if I can help in any way. I'm trying to (slowly) learn the internals of ClickHouse.

@alexey-milovidov
Copy link
Copy Markdown
Member

The real reason is that the type is wrong:

SELECT toTypeName(uniqExactState(x))
FROM
(
    SELECT quantileState(number) AS x
    FROM numbers(1000)
)

Query id: 3a8dd528-9b14-4506-a5a8-1ab2241ea2b2

┌─toTypeName(uniqExactState(x))───────┐
│ AggregateFunction(quantile, UInt64) │
└─────────────────────────────────────┘

Has to be something like

AggregateFunction(uniqExact, AggregateFunction(quantile, UInt64))

I will check how it is possible.

@alexey-milovidov
Copy link
Copy Markdown
Member

Another example:

SELECT toTypeName(initializeAggregation('anyState', initializeAggregation('uniqState', 0)))

@alexey-milovidov
Copy link
Copy Markdown
Member

The bug was introduced in #7109.

@alexey-milovidov
Copy link
Copy Markdown
Member

This PR does not fix the issue, here is another reproducer:

SELECT anyState(x) FROM (SELECT uniqState(number) AS x FROM numbers(1000))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix Pull request with bugfix, not backported by default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Crash when chaining different uniq*State

4 participants