-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Degraded performance of select queries after v22.8 #44500
Description
Describe the situation
There is still a performance drop between 22.7.7.24 -> 22.8.1.2097 (and up to the latest 22.12)
I'm running a trivial query "select count() from ... where counter_id = ... and ts > ... and ts < .. format Null settings local_filesystem_read_method='read'." (typical "hits" table).
The query pipeline is identical between the versions (22.7.7.24 and 22.8.1.2097). The slowness comes from "AggregatingTransform" step that's processing rows x3-4 times slower.
Trace logs:
22.7 - https://pastila.nl/?02aa875b/fd42b27162d53ca2a79b137e217b9ddd
22.8 - https://pastila.nl/?02aa875b/d0c917e0b0166c5f8dd3af2d1cc96035
The QPS drop is x2-3 times regardless of the concurrency level used in clickhouse-benchmark tool (same results for -c 1, 8, 16)
How to reproduce
I was not able to reproduce the issue when creating new tables and importing fake data. On the real machine I can upgrade/downgrade between the versions and clearly see the performance difference. I went to the point where one replica in the cluster is on 22.7.7.24 and another is 22.8.1.2097. Practically all queries are slower on 22.8 about x1.5-3 times.
Additional context
I checked all versions after 22.8 up to the latest 22.12, the issue persists.
The query in this case is x1.5 times slower.
Also, the code difference: v22.8.1.2097-lts...v22.7.7.24-stable

