Skip to content

Secondary index is slower than full scan #55653

@alexey-milovidov

Description

@alexey-milovidov
CREATE TABLE index_test
(
    z UInt32,
    x UInt16 ALIAS mortonDecode(2, z).1,
    y UInt16 ALIAS mortonDecode(2, z).2,
    INDEX i_x (mortonDecode(2, z).1) TYPE minmax,
    INDEX i_y (mortonDecode(2, z).2) TYPE minmax
) ENGINE = MergeTree ORDER BY z;

INSERT INTO index_test SELECT number FROM numbers(0x100000000) WHERE rand() % 3 = 1;

-- primary key, 5 ms.
SELECT count() FROM index_test WHERE z >= 1000000000 AND z <= 1000100000;

-- secondary index, 1341 ms.
SELECT count() FROM index_test WHERE x >= 20000 AND x <= 20100 AND y >= 10000 AND y <= 10100;

-- full scan, 417 ms.
ALTER TABLE index_test DROP INDEX i_x, DROP INDEX i_y;
SELECT count() FROM index_test WHERE x >= 20000 AND x <= 20100 AND y >= 10000 AND y <= 10100;

Logs of the query with secondary index:

[milovidov-desktop] 2023.10.16 06:49:58.303367 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Debug> executeQuery: (from [::ffff:127.0.0.1]:59990) SELECT count() FROM index_test WHERE x >= 20000 AND x <= 20100 AND y >= 10000 AND y <= 10100 (stage: Complete)
[milovidov-desktop] 2023.10.16 06:49:58.304468 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0
[milovidov-desktop] 2023.10.16 06:49:58.304856 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Trace> ContextAccess (default): Access granted: SELECT(x, y) ON default.index_test
[milovidov-desktop] 2023.10.16 06:49:58.304928 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
[milovidov-desktop] 2023.10.16 06:49:58.306240 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Debug> default.index_test (f8283f41-c765-4a2e-83fa-1688847d0c70) (SelectExecutor): Key condition: unknown, unknown, and, unknown, and, unknown, and
[milovidov-desktop] 2023.10.16 06:49:59.640236 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Debug> default.index_test (f8283f41-c765-4a2e-83fa-1688847d0c70) (SelectExecutor): Index `i_x` has dropped 173760/174761 granules.
[milovidov-desktop] 2023.10.16 06:49:59.640322 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Debug> default.index_test (f8283f41-c765-4a2e-83fa-1688847d0c70) (SelectExecutor): Index `i_y` has dropped 997/1001 granules.
[milovidov-desktop] 2023.10.16 06:49:59.640392 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Debug> default.index_test (f8283f41-c765-4a2e-83fa-1688847d0c70) (SelectExecutor): Selected 2/2 parts by partition key, 1 parts by primary key, 174761/174761 marks by primary key, 4 marks to read from 2 ranges
[milovidov-desktop] 2023.10.16 06:49:59.640431 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Trace> default.index_test (f8283f41-c765-4a2e-83fa-1688847d0c70) (SelectExecutor): Spreading mark ranges among streams (default reading)
[milovidov-desktop] 2023.10.16 06:49:59.640497 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Trace> default.index_test (f8283f41-c765-4a2e-83fa-1688847d0c70) (SelectExecutor): Reading 2 ranges in order from part all_1_1340_4_1343, approx. 32768 rows starting from 136921088
[milovidov-desktop] 2023.10.16 06:49:59.643213 [ 383074 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Trace> AggregatingTransform: Aggregating
[milovidov-desktop] 2023.10.16 06:49:59.643265 [ 383074 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Trace> Aggregator: An entry for key=4744199629802693815 found in cache: sum_of_sizes=1, median_size=1
[milovidov-desktop] 2023.10.16 06:49:59.643282 [ 383074 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Trace> Aggregator: Aggregation method: without_key
[milovidov-desktop] 2023.10.16 06:49:59.643337 [ 383074 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Trace> AggregatingTransform: Aggregated. 3359 to 1 rows (from 0.00 B) in 0.002588214 sec. (1297806.132 rows/sec., 0.00 B/sec.)
[milovidov-desktop] 2023.10.16 06:49:59.643358 [ 383074 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Trace> Aggregator: Merging aggregated data
┌─count()─┐
│    3359 │
└─────────┘
[milovidov-desktop] 2023.10.16 06:49:59.644367 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Debug> executeQuery: Read 32768 rows, 128.00 KiB in 1.341094 sec., 24433.783165087607 rows/sec., 95.44 KiB/sec.
[milovidov-desktop] 2023.10.16 06:49:59.644590 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Debug> MemoryTracker: Peak memory usage (for query): 4.48 MiB.
[milovidov-desktop] 2023.10.16 06:49:59.644607 [ 376016 ] {ca0840b5-ade6-4149-897b-67c8fd17b77c} <Debug> TCPHandler: Processed in 1.341629963 sec.

1 row in set. Elapsed: 1.341 sec. Processed 32.77 thousand rows, 131.07 KB (24.43 thousand rows/s., 97.72 KB/s.)

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions