Skip to content

fix(sql): support order by desc with negative limit on Jit-filtered interval queries#5798

Merged
bluestreak01 merged 8 commits intomasterfrom
raphdal_intervalbwd_orderasc_bug
Jul 1, 2025
Merged

fix(sql): support order by desc with negative limit on Jit-filtered interval queries#5798
bluestreak01 merged 8 commits intomasterfrom
raphdal_intervalbwd_orderasc_bug

Conversation

@RaphDal
Copy link
Copy Markdown
Contributor

@RaphDal RaphDal commented Jun 30, 2025

This PR fixes an internal error that occurs when a user runs a SELECT query with:

  • A descending ORDER BY on the timestamp column
  • A negative LIMIT
  • Jit filtering enabled
  • And a timestamp interval filter

Here's an example of such a query:

CREATE TABLE x AS (
  SELECT
    rnd_double() price,
    timestamp_sequence('2024', 24*60*60*1000*1000) ts
  FROM
    long_sequence(2)
) timestamp(ts) PARTITION BY day;

SELECT price, ts
FROM x
WHERE price > 0 AND ts >= '2024-01-01' AND ts <= '2024-12-31'
ORDER BY ts DESC
LIMIT -1

Root cause

The issue arises under several specific conditions:

  • Jit is enabled, and the filter (price > 0) is compatible with it, so AsyncJitFilteredRecordCursorFactory is used.
  • The ORDER BY ts DESC causes the optimiser to set forceBackwardScan to true.
  • The timestamp filter (ts >= ... AND ts <= ...) leads SqlCodeGenerator to use IntervalBwdPartitionFrameCursorFactory.
  • A negative LIMIT causes the AsyncJitFilteredRecordCursorFactory to reverse the order from ORDER_DESC to ORDER_ASC.

However, unlike FullBwdPartitionFrameCursorFactory, the IntervalBwdPartitionFrameCursorFactory did not support ORDER_ASC and threw an UnsupportedOperationException.

Fix

This PR resolves the issue by applying the same strategy used in FullBwdPartitionFrameCursorFactory:
If ORDER_ASC is requested, the factory now creates an IntervalFwdPartitionFrameCursor on the fly.

Work In Progress

I'm keeping the PR as work in progress as I'd like to add fuzzy testing to this use case.

@RaphDal RaphDal marked this pull request as draft June 30, 2025 16:12
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Jun 30, 2025

CLA assistant check
All committers have signed the CLA.

@RaphDal RaphDal marked this pull request as ready for review June 30, 2025 16:17
@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 10 / 10 (100.00%)

file detail

path covered line new line coverage
🔵 io/questdb/cairo/IntervalBwdPartitionFrameCursorFactory.java 10 10 100.00%

@bluestreak01 bluestreak01 merged commit 91c92df into master Jul 1, 2025
37 checks passed
@bluestreak01 bluestreak01 deleted the raphdal_intervalbwd_orderasc_bug branch July 1, 2025 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants