You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
array_has is 3200x slower than it "should be" #12062
The performance of array_has seems to be pretty poor due to RowConverter.
I compared running array_has queries vs json_contain (not a particularly good comparison, but that's not the point here), I'd expect array_has to be somewhat faster but it's actually 3200x slower:
+----------+
| count(*) |
+----------+
| 4828 |
+----------+
mode: SELECT count(*) FROM test where json_contains(json, 'service.name'), query took 31.696875ms
+----------+
| count(*) |
+----------+
| 4828 |
+----------+
mode: SELECT count(*) FROM test where array_has(list, 'service.name'), query took 102.430949125s
Code for this example is here, and here is a flame graph from samply, you can see that 99% of time is in RowConverter:
Most of the problematic behaviour is in RowConverter, but I also think it should be much faster by making general_array_has_dispatch special cased or generic around ComparisonType rather than branching in the hot loop.
I don't like mixing logic for these array_has family, it not only makes the code less readability and I think it is also the main reason that slows down the code (it computes unnecessary things)
The issue here is similar to multi group by aggregate function where RowConvert on String Type is quite slow.
Activity
jayzhan211 commentedon Aug 21, 2024
I don't like mixing logic for these
array_hasfamily, it not only makes the code less readability and I think it is also the main reason that slows down the code (it computes unnecessary things)The issue here is similar to multi group by aggregate function where RowConvert on String Type is quite slow.
jayzhan211 commentedon Aug 26, 2024
Other issues tracked in #12163 , including
array_has_allandarray_has_any