Skip to content

bugs/friction when handling IN lists with multiple intervals #6668

@nwoolmer

Description

@nwoolmer

To reproduce

https://questdb.com/docs/query/operators/comparison/#in-value1-value2-

The docs say these are the same, but they definitely are not, and the second will have considerably worse performance. I'm not sure if this was supported syntax in the past. It would be very convenient syntax though.

For demo:

SELECT * FROM trades
WHERE timestamp IN ('2018-01-01', '2018-01-01T12:00', '2018-01-02');

Plan:

PageFrame
    Row forward scan
    Interval forward scan on: trades
      intervals: [("2018-01-01T00:00:00.000000Z","2018-01-01T00:00:00.000000Z"),("2018-01-01T12:00:00.000000Z","2018-01-01T12:00:00.000000Z"),("2018-01-02T00:00:00.000000Z","2018-01-02T00:00:00.000000Z")]
SELECT * FROM trades
WHERE timestamp IN '2018-01-01' OR timestamp IN '2018-01-01T12:00' OR timestamp IN '2018-01-02';

Plan:

Async JIT Filter workers: 46
  filter: ((timestamp in [1514764800000000,1514851199999999] or timestamp in [1514808000000000,1514808059999999]) or timestamp in [1514851200000000,1514937599999999])
    PageFrame
        Row forward scan
        Frame forward scan on: trades

QuestDB version:

9.3.1

OS, in case of Docker specify Docker and the Host OS:

N/A

File System, in case of Docker specify Host File System:

N/A

Full Name:

Nick Woolmer

Affiliation:

QuestDB

Have you followed Linux, MacOs kernel configuration steps to increase Maximum open files and Maximum virtual memory areas limit?

  • Yes, I have

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions