Skip to content

Arrays of LowCardinality are not optimized for function calculations #6005

@alex-zaitsev

Description

@alex-zaitsev

Arrays of LowCardinality Strings do not use LowCardinality optimisations for string operations.

create table test_lc_array (
 s LowCardinality(String),
 a Array(LowCardinality(String)) default array(s)
) Engine = MergeTree 
PARTITION BY tuple()
ORDER BY tuple()
;

insert into test_lc_array (s)
select concat('qwertyuiopasdfghjkl', toString(number%10))
from numbers(100000000)
;  

select count() from test_lc_array where s = 'qwertyuiopasdfghjkl0';

1 rows in set. Elapsed: 0.200 sec. Processed 100.00 million rows, 100.46 MB (501.07 million rows/s., 503.36 MB/s.) 
<Debug> MemoryTracker: Peak memory usage (for query): 2.04 MiB.

select count() from test_lc_array where a[1] = 'qwertyuiopasdfghjkl0';

1 rows in set. Elapsed: 2.033 sec. Processed 100.00 million rows, 900.46 MB (49.19 million rows/s., 442.93 MB/s.) 
<Debug> MemoryTracker: Peak memory usage (for query): 16.74 MiB.

select count() from test_lc_array where has(a, 'qwertyuiopasdfghjkl0');

1 rows in set. Elapsed: 1.136 sec. Processed 100.00 million rows, 900.46 MB (88.06 million rows/s., 792.97 MB/s.) 
 <Debug> MemoryTracker: Peak memory usage (for query): 10.99 MiB.

select count() from test_lc_array where indexOf(a, 'qwertyuiopasdfghjkl0')>0;

1 rows in set. Elapsed: 1.211 sec. Processed 100.00 million rows, 900.46 MB (82.60 million rows/s., 743.81 MB/s.) 
<Debug> MemoryTracker: Peak memory usage (for query): 12.74 MiB.

Metadata

Metadata

Assignees

Labels

comp-datatype-wrapperType modifiers/wrappers (Nullable, LowCardinality, etc.).comp-regular-functionRegular scalar functions: string processing, data conversion, arithmetic, math, comparison, condi...performance

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions