fix(core): fix crash on LATEST BY ALL queries over large tables by ideoma · Pull Request #6832 · questdb/questdb

ideoma · 2026-02-27T15:39:00Z

Summary

Fix Numbers.MAX_SAFE_INT_POW_2 from 1L << 31 to 1L << 30. The old value (2^31) does not fit in a signed 32-bit int, so the rehash overflow guard newKeyCapacity > MAX_SAFE_INT_POW_2 let exactly 2^31 through. The subsequent (int) cast produced Integer.MIN_VALUE, and clear() fed ~18 EB to native memset, causing a SIGSEGV.
Deduplicate the constant from Unordered4Map, Unordered8Map, and UnorderedVarcharMap — each had a private copy with the same bug.
Add overflow guard tests in MapTest and OrderedMapTest.

Crash chain

A LATEST BY ALL query on a large table fills an OrderedMap until keyCapacity reaches 2^30.
rehash() doubles to newKeyCapacity = 1L << 31. The guard (> 1L << 31) is false at the boundary — allocation of 16 GB succeeds.
keyCapacity = (int)(1L << 31) truncates to Integer.MIN_VALUE (−2,147,483,648).
map.clear() computes (long)keyCapacity << 3 = 0xFFFFFFFC00000000 (~18.4 EB) and passes it to native memset → SIGSEGV.

Fix

MAX_SAFE_INT_POW_2 = 1L << 30 makes the guard reject newKeyCapacity = 2^31, throwing a clean CairoException("map capacity overflow") instead of crashing the JVM.

Numbers.MAX_SAFE_INT_POW_2 was 1L << 31 (2,147,483,648), which does not fit in a signed 32-bit int. The rehash overflow guard compared newKeyCapacity > MAX_SAFE_INT_POW_2, so when newKeyCapacity was exactly 2^31 the check passed. The subsequent (int) cast produced Integer.MIN_VALUE, and clear() fed a negative byte count to native memset, causing a SIGSEGV. Change the constant to 1L << 30 so the guard rejects 2^31 and throws a clean CairoException instead of crashing the JVM. Deduplicate the constant from Unordered4Map, Unordered8Map, and UnorderedVarcharMap — they each had a private copy with the same bug. Co-Authored-By: Claude Opus 4.6 <[email protected]>

coderabbitai · 2026-02-27T15:39:36Z

Walkthrough

Consolidates the MAX_SAFE_INT_POW_2 constant to a centralized location in Numbers.java by removing local definitions from three map classes and adding static imports. Updates the constant value from 1L << 31 to 1L << 30 and adjusts related tests to reflect the new boundary behavior.

Changes

Cohort / File(s)	Summary
Map Class Refactoring `core/src/main/java/io/questdb/cairo/map/Unordered4Map.java`, `core/src/main/java/io/questdb/cairo/map/Unordered8Map.java`, `core/src/main/java/io/questdb/cairo/map/UnorderedVarcharMap.java`	Removed local static constants and added static imports of `MAX_SAFE_INT_POW_2` from Numbers class. No behavioral changes to the referencing code.
Central Constant Definition `core/src/main/java/io/questdb/std/Numbers.java`	Updated public constant `MAX_SAFE_INT_POW_2` value from `1L << 31` to `1L << 30`, effectively halving the maximum safe integer power-of-two threshold.
Test Updates `core/src/test/java/io/questdb/test/cairo/map/MapTest.java`, `core/src/test/java/io/questdb/test/cairo/map/OrderedMapTest.java`	Updated testSetKeyCapacityOverflow boundary logic to account for the new constant value using a 0.75 load factor calculation. Added new testKeyCapacityOverflow in OrderedMapTest to verify capacity overflow exception handling.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Suggested reviewers

bluestreak01

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main fix: addressing a crash in LATEST BY ALL queries over large tables caused by map capacity overflow.
Description check	✅ Passed	The description clearly relates to the changeset, explaining the root cause of the crash, the fix applied, deduplication efforts, and test additions.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings (stacked PR)
📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch fix-map-capacity-overflow

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ast-grep (0.41.0)

core/src/main/java/io/questdb/std/Numbers.java

core/src/test/java/io/questdb/test/cairo/map/OrderedMapTest.java

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

ideoma · 2026-02-27T15:43:55Z

@coderabbitia review

coderabbitai · 2026-02-27T15:44:14Z

@ideoma, I'll review the changes in this PR.

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@core/src/test/java/io/questdb/test/cairo/map/OrderedMapTest.java`:
- Around line 1384-1387: The test constructs OrderedMap with a 0.5 load factor
but the comment and intended behavior match the 0.75 case; update the OrderedMap
instantiation in OrderedMapTest (the constructor call that currently passes 0.5)
to pass 0.75 so the subsequent map.setKeyCapacity(Integer.MAX_VALUE / 4 * 3 + 1)
check aligns with the comment and mirrors MapTest.testSetKeyCapacityOverflow
behavior.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3ea051b and d67f2b0.

📒 Files selected for processing (6)

core/src/main/java/io/questdb/cairo/map/Unordered4Map.java
core/src/main/java/io/questdb/cairo/map/Unordered8Map.java
core/src/main/java/io/questdb/cairo/map/UnorderedVarcharMap.java
core/src/main/java/io/questdb/std/Numbers.java
core/src/test/java/io/questdb/test/cairo/map/MapTest.java
core/src/test/java/io/questdb/test/cairo/map/OrderedMapTest.java

core/src/test/java/io/questdb/test/cairo/map/OrderedMapTest.java

glasstiger · 2026-02-27T16:30:19Z

[PR Coverage check]

😍 pass : 0 / 0 (0%)

…tdb#6832)

ideoma and others added 2 commits February 27, 2026 15:35

Fix int overflow in map key capacity guard

d67f2b0

ideoma added Bug Incorrect or unexpected behavior Core Related to storage, data type, etc. labels Feb 27, 2026

ideoma changed the title ~~fix(core): fix JVM crash on LATEST BY ALL queries over large tables~~ fix(core): fix crash on LATEST BY ALL queries over large tables Feb 27, 2026

puzpuzpuz approved these changes Feb 27, 2026

View reviewed changes

coderabbitai bot reviewed Feb 27, 2026

View reviewed changes

core/src/test/java/io/questdb/test/cairo/map/OrderedMapTest.java Show resolved Hide resolved

bluestreak01 merged commit 10a57ef into master Feb 27, 2026
46 checks passed

bluestreak01 deleted the fix-map-capacity-overflow branch February 27, 2026 17:19

maciulis pushed a commit to maciulis/questdb that referenced this pull request Mar 16, 2026

fix(core): fix crash on LATEST BY ALL queries over large tables (ques…

0a7b12a

…tdb#6832)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(core): fix crash on LATEST BY ALL queries over large tables#6832

fix(core): fix crash on LATEST BY ALL queries over large tables#6832
bluestreak01 merged 2 commits intomasterfrom
fix-map-capacity-overflow

ideoma commented Feb 27, 2026 •

edited

Loading

Uh oh!

coderabbitai bot commented Feb 27, 2026 •

edited

Loading

❌ Failed checks (1 warning)

Uh oh!

ideoma commented Feb 27, 2026

Uh oh!

coderabbitai bot commented Feb 27, 2026

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

glasstiger commented Feb 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

ideoma commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Crash chain

Fix

Uh oh!

coderabbitai bot commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Suggested reviewers

❌ Failed checks (1 warning)

Uh oh!

ideoma commented Feb 27, 2026

Uh oh!

coderabbitai bot commented Feb 27, 2026

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

glasstiger commented Feb 27, 2026

[PR Coverage check]

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ideoma commented Feb 27, 2026 •

edited

Loading

coderabbitai bot commented Feb 27, 2026 •

edited

Loading