fix DataTypeAggregateFunction deserialization#6773
fix DataTypeAggregateFunction deserialization#6773alexey-milovidov merged 3 commits intoClickHouse:masterfrom infinivision:fix_issue_6575
Conversation
|
|
||
|
|
||
| #define DEFAULT_MAX_STRING_SIZE 0x00FFFFFFULL | ||
| #define DEFAULT_MAX_STRING_SIZE 0x3FFFFFFFULL |
There was a problem hiding this comment.
Missing comment here. Why 3F, how it is motivated?
There was a problem hiding this comment.
The intermediate state of bitmap is also contained within the query to be transferred among nodes, it's very easy to exceed the original 16MB size for a bitmap state and it would be cut leading to parsing exception.
There was a problem hiding this comment.
bitmap serialization writes each UInt32 as 4 bytes, and become at most 8 bytes when escaped. 0x3FFFFFFFULL can cover at least (2^30)/8=125M UInt32 integers.
We could reduce bitmap serialization size by using the official CRoaring serialization. This also could help to improve bitmap serialization/deserialization performance.
There was a problem hiding this comment.
Ok.
This also could help to improve bitmap serialization/deserialization performance.
It will be appreciated if you can implement it with backwards compatibility.
|
Ok. But test is missing. |
|
@alexey-milovidov I don't understand how |
We run ClickHouse that listen to |
KochetovNicolai
added
pr-bugfix
fix DataTypeAggregateFunction deserialization (cherry picked from commit 845a612)
fix DataTypeAggregateFunction deserialization (cherry picked from commit 845a612)
4 participants
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
Category (leave one):
Short description (up to few sentences):
Try to fix 6575.
I have to enlarge DEFAULT_MAX_STRING_SIZE in dbms/src/IO/ReadHelpers.h, and max_query_size in users.xml.