Skip to content

array_has is 3200x slower than it "should be" #12062

Closed

Description

@samuelcolvin

Describe the bug

The performance of array_has seems to be pretty poor due to RowConverter.

I compared running array_has queries vs json_contain (not a particularly good comparison, but that's not the point here), I'd expect array_has to be somewhat faster but it's actually 3200x slower:

+----------+
| count(*) |
+----------+
| 4828     |
+----------+
mode: SELECT count(*) FROM test where json_contains(json, 'service.name'), query took 31.696875ms
+----------+
| count(*) |
+----------+
| 4828     |
+----------+
mode: SELECT count(*) FROM test where array_has(list, 'service.name'), query took 102.430949125s

Code for this example is here, and here is a flame graph from samply, you can see that 99% of time is in RowConverter:

image

To Reproduce

Clone https://github.com/samuelcolvin/array-has-slow and run cargo run --release.

Expected behavior

array_has should be much faster.

Most of the problematic behaviour is in RowConverter, but I also think it should be much faster by making general_array_has_dispatch special cased or generic around ComparisonType rather than branching in the hot loop.

Additional context

No response

Activity

  1. jayzhan211 commented on Aug 21, 2024

    @jayzhan211
    Contributor

    I don't like mixing logic for these array_has family, it not only makes the code less readability and I think it is also the main reason that slows down the code (it computes unnecessary things)

    The issue here is similar to multi group by aggregate function where RowConvert on String Type is quite slow.

  2. self-assigned this
    on Aug 21, 2024
  3. jayzhan211 commented on Aug 26, 2024

    @jayzhan211
    Contributor

    Other issues tracked in #12163 , including array_has_all and array_has_any

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions