-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Arrays of LowCardinality are not optimized for function calculations #6005
Copy link
Copy link
Closed
Labels
comp-datatype-wrapperType modifiers/wrappers (Nullable, LowCardinality, etc.).Type modifiers/wrappers (Nullable, LowCardinality, etc.).comp-regular-functionRegular scalar functions: string processing, data conversion, arithmetic, math, comparison, condi...Regular scalar functions: string processing, data conversion, arithmetic, math, comparison, condi...performance
Description
Arrays of LowCardinality Strings do not use LowCardinality optimisations for string operations.
create table test_lc_array (
s LowCardinality(String),
a Array(LowCardinality(String)) default array(s)
) Engine = MergeTree
PARTITION BY tuple()
ORDER BY tuple()
;
insert into test_lc_array (s)
select concat('qwertyuiopasdfghjkl', toString(number%10))
from numbers(100000000)
;
select count() from test_lc_array where s = 'qwertyuiopasdfghjkl0';
1 rows in set. Elapsed: 0.200 sec. Processed 100.00 million rows, 100.46 MB (501.07 million rows/s., 503.36 MB/s.)
<Debug> MemoryTracker: Peak memory usage (for query): 2.04 MiB.
select count() from test_lc_array where a[1] = 'qwertyuiopasdfghjkl0';
1 rows in set. Elapsed: 2.033 sec. Processed 100.00 million rows, 900.46 MB (49.19 million rows/s., 442.93 MB/s.)
<Debug> MemoryTracker: Peak memory usage (for query): 16.74 MiB.
select count() from test_lc_array where has(a, 'qwertyuiopasdfghjkl0');
1 rows in set. Elapsed: 1.136 sec. Processed 100.00 million rows, 900.46 MB (88.06 million rows/s., 792.97 MB/s.)
<Debug> MemoryTracker: Peak memory usage (for query): 10.99 MiB.
select count() from test_lc_array where indexOf(a, 'qwertyuiopasdfghjkl0')>0;
1 rows in set. Elapsed: 1.211 sec. Processed 100.00 million rows, 900.46 MB (82.60 million rows/s., 743.81 MB/s.)
<Debug> MemoryTracker: Peak memory usage (for query): 12.74 MiB.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
comp-datatype-wrapperType modifiers/wrappers (Nullable, LowCardinality, etc.).Type modifiers/wrappers (Nullable, LowCardinality, etc.).comp-regular-functionRegular scalar functions: string processing, data conversion, arithmetic, math, comparison, condi...Regular scalar functions: string processing, data conversion, arithmetic, math, comparison, condi...performance