Skip to content

perf(sql): improve performance of symbol index lookups#5953

Merged
bluestreak01 merged 4 commits intomasterfrom
vi_index_perf
Jul 18, 2025
Merged

perf(sql): improve performance of symbol index lookups#5953
bluestreak01 merged 4 commits intomasterfrom
vi_index_perf

Conversation

@bluestreak01
Copy link
Copy Markdown
Member

@bluestreak01 bluestreak01 commented Jul 16, 2025

  • significantly improves index lookup performance for partitions that do not change from query to query
  • replaces unnecessary unmap/remap of the index on every query to a more intelligent version check. Works fully on no-change to "append" partition update scenarios
  • improves performance of index lookups during O3 by reducing map/unmap calls @ideoma may be able to continue from here to improve O3 as well

Before:

image

After

image

thanks @RaphDal for the comprehensive benchmark!

ideoma
ideoma previously approved these changes Jul 17, 2025
@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 51 / 53 (96.23%)

file detail

path covered line new line coverage
🔵 io/questdb/cairo/BitmapIndexBwdNullReader.java 1 2 50.00%
🔵 io/questdb/cairo/BitmapIndexFwdNullReader.java 1 2 50.00%
🔵 io/questdb/griffin/engine/table/LatestByAllIndexedRecordCursor.java 1 1 100.00%
🔵 io/questdb/cairo/TableReader.java 6 6 100.00%
🔵 io/questdb/cairo/BitmapIndexFwdReader.java 2 2 100.00%
🔵 io/questdb/cairo/ConcurrentBitmapIndexFwdReader.java 2 2 100.00%
🔵 io/questdb/cairo/BitmapIndexBwdReader.java 2 2 100.00%
🔵 io/questdb/cairo/AbstractIndexReader.java 36 36 100.00%

@bluestreak01 bluestreak01 merged commit 129c7b0 into master Jul 18, 2025
34 checks passed
@bluestreak01 bluestreak01 deleted the vi_index_perf branch July 18, 2025 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants