Skip to content

fix(core): fix rare error on index drop that can suspend table#6147

Merged
bluestreak01 merged 12 commits intomasterfrom
fix-error-index-drop
Sep 24, 2025
Merged

fix(core): fix rare error on index drop that can suspend table#6147
bluestreak01 merged 12 commits intomasterfrom
fix-error-index-drop

Conversation

@ideoma
Copy link
Copy Markdown
Collaborator

@ideoma ideoma commented Sep 16, 2025

fixes #6146

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 16, 2025

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Replaced recursive QuickSort in IntGroupSort with an iterative version using IntList’s internal capacity as an explicit stack. Updated method signatures to accept IntList instead of int[]. IntList now passes itself to sorting and exposes a package-private resetCapacityInternal. Added a test constructing many reverse-ordered groups to validate non-recursive sorting.

Changes

Cohort / File(s) Summary of modifications
Iterative QuickSort refactor
core/src/main/java/io/questdb/std/IntGroupSort.java
Replaced recursive QuickSort with iterative loop using an explicit stack stored in IntList spare capacity; updated quickSort and quickSortImpl signatures to take IntList; added try/finally to restore IntList size; updated comments and internal usage from int[] to IntList.
IntList integration
core/src/main/java/io/questdb/std/IntList.java
sortGroups now calls IntGroupSort.quickSort with this (IntList) instead of backing array; added package-private int[] resetCapacityInternal(int) to ensure capacity and access backing array internally.
Test for deep recursion case
core/src/test/java/io/questdb/test/std/IntListTest.java
Added testSortGroupsStackOverflowScenario creating many reverse-ordered groups, invoking sortGroups, and verifying group-ordering without StackOverflowError.

Sequence Diagram(s)

sequenceDiagram
  actor Client
  participant IL as IntList
  participant IGS as IntGroupSort

  Client->>IL: sortGroups(groupSize)
  IL->>IGS: quickSort(n, this, groupLo, groupHi)
  activate IGS
  rect rgb(230,245,255)
    note over IGS: Allocate stack in IntList spare capacity
    loop while stack not empty
      IGS->>IGS: pop (low, high)
      IGS->>IGS: partition(IntList, low, high, n)
      alt push larger partition first
        IGS->>IGS: push ranges onto stack
      end
    end
    note over IGS: finally: restore original IntList size
  end
  deactivate IGS
  IL-->>Client: return
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested labels

Bug

Suggested reviewers

  • bluestreak01
  • RaphDal

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (4 passed)
Check name Status Explanation
Title Check ✅ Passed The title states the intent to fix a rare error on index drop that can suspend a table, which aligns with the PR objective to prevent the StackOverflowError observed during index removal; the changeset’s primary work is replacing the recursive IntGroupSort quicksort with an iterative implementation to avoid deep recursion. The title is concise, focused on the user-visible symptom, and sufficiently informative for a quick scan.
Linked Issues Check ✅ Passed Linked issue #6146 describes a StackOverflowError in IntGroupSort during index drop; the PR replaces the recursive quicksort with an iterative, explicit-stack implementation using IntList capacity, adds a package-private IntList helper to ensure capacity, updates callers, and introduces a regression test that reproduces the worst-case scenario. These coding changes address the reported root cause (deep recursion) and add test coverage to prevent regressions. Based on the provided summaries, the patch meets the linked issue’s coding-related objectives.
Out of Scope Changes Check ✅ Passed The changes are confined to IntGroupSort, IntList, and a unit test and directly implement the iterative quicksort and necessary IntList support; I do not see unrelated file edits or features added outside the scope of the linked issue. The new package-private helper is justified by the implementation and does not alter public APIs. No out-of-scope changes detected.
Description Check ✅ Passed The description "fixes #6146" directly references the linked issue that motivated the changes and is therefore on-topic for this changeset, even though it is minimal. It correctly ties the PR to the reported StackOverflowError during index drop.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ideoma
Copy link
Copy Markdown
Collaborator Author

ideoma commented Sep 16, 2025

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 16, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (6)
core/src/test/java/io/questdb/test/std/IntListTest.java (2)

285-314: Good regression for StackOverflow; add a couple of cheap correctness asserts.

This reliably exercises the worst-case partition pattern and validates ordering by the first key. Since groups are swapped as units, add quick checks that group members stayed intact after sorting (guards against accidental element-wise swaps or a wrong stride).

Apply this diff to extend the assertions:

         // Verify the result is correctly sorted
         for (int i = 0; i < numGroups - 1; i++) {
             int baseIndex = i * groupSize;
             int nextBaseIndex = (i + 1) * groupSize;

             // Compare first elements of consecutive groups - should be in ascending order
             Assert.assertTrue("Groups should be sorted in ascending order",
                     list.getQuick(baseIndex) <= list.getQuick(nextBaseIndex));
+
+            // Sanity-check group cohesion: members should remain tied to their group's first key
+            int key = list.getQuick(baseIndex);
+            Assert.assertEquals(key * 2, list.getQuick(baseIndex + 1));
+            Assert.assertEquals(key * 3, list.getQuick(baseIndex + 2));
         }

285-314: Consider marking as slow or trimming size to keep CI snappy.

50k groups (150k ints) is fine locally, but if CI gets tight, annotate as a slow test (repo convention permitting) or reduce to 20–30k while keeping the worst-case shape.

If the project uses categories, would you like me to wire this under a @category(Slow.class) (JUnit4) or a custom marker?

core/src/main/java/io/questdb/std/IntList.java (1)

397-400: Internal backdoor to the backing array: document and fence expectations.

Exposing the raw array is fine within the package for algorithms like IntGroupSort, but the invariants should be explicit.

Apply this diff to add an internal Javadoc:

-    int[] resetCapacityInternal(int intCapacity) {
+    /**
+     * INTERNAL: Ensures capacity ≥ {@code intCapacity} and returns the backing array reference.
+     * The caller may write to indices in [pos, capacity) as temporary scratch space,
+     * but must not rely on contents being preserved or visible to external callers.
+     * Do not mutate [0, pos) except via IntList APIs.
+     */
+    int[] resetCapacityInternal(int intCapacity) {
         checkCapacity(intCapacity);
         return data;
     }
core/src/main/java/io/questdb/std/IntGroupSort.java (3)

59-127: Add a tiny safety net to grow the stack if ever needed.

64 ints should suffice (≈32 ranges) under the “process smaller first” strategy, but a defensive check makes this future-proof and documents intent.

Apply this diff to ensure capacity before pushes:

-        // Push initial range onto stack
+        // Push initial range onto stack
+        if (stackPos + 2 > array.length) {
+            array = intList.resetCapacityInternal(stackPos + 64);
+        }
         array[stackPos++] = low;
         array[stackPos++] = high;

         while (stackPos > stackStart) {
             // Pop range from stack
             high = array[--stackPos];
             low = array[--stackPos];

             if (low + 1 < high) {
                 int pi = partition(array, low, high, n);

                 // Push the larger partition first, then smaller one
                 // This ensures the stack depth remains logarithmic
                 if (pi - low > high - pi - 1) {
                     // Left partition is larger
+                    if (stackPos + 4 > array.length) {
+                        array = intList.resetCapacityInternal(stackPos + 64);
+                    }
                     array[stackPos++] = low;
                     array[stackPos++] = pi;
                     array[stackPos++] = pi + 1;
                     array[stackPos++] = high;
                 } else {
                     // Right partition is larger or equal
+                    if (stackPos + 4 > array.length) {
+                        array = intList.resetCapacityInternal(stackPos + 64);
+                    }
                     array[stackPos++] = pi + 1;
                     array[stackPos++] = high;
                     array[stackPos++] = low;
                     array[stackPos++] = pi;
                 }
             }
         }

33-42: Comments mention “3 elements each” but n is variable.

Minor doc drift; update to avoid confusion.

Suggested edits:

  • Line 33: “Compare two groups” (drop “of 3 elements each”).
  • Line 130: “Swap two groups” (drop “of 3 elements each”).

Also applies to: 130-139


142-153: Clarify that groupHi is exclusive in the Javadoc.

The implementation treats [groupLo, groupHi); make it explicit.

Apply this diff to the Javadoc:

-     * @param groupLo start index of the group
-     * @param groupHi end index of the group
+     * @param groupLo start group index (inclusive)
+     * @param groupHi end group index (exclusive)
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7eb1a51 and e1a4cba.

📒 Files selected for processing (3)
  • core/src/main/java/io/questdb/std/IntGroupSort.java (2 hunks)
  • core/src/main/java/io/questdb/std/IntList.java (2 hunks)
  • core/src/test/java/io/questdb/test/std/IntListTest.java (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: New pull request (Check Changes Check changes)
  • GitHub Check: build
  • GitHub Check: build
🔇 Additional comments (2)
core/src/main/java/io/questdb/std/IntGroupSort.java (1)

59-127: Iterative quicksort removes the StackOverflowError risk.

The explicit stack built in the IntList tail is a solid fix; restoring pos in a finally-block is the right hygiene. Nice.

core/src/main/java/io/questdb/std/IntList.java (1)

322-322: Call-site looks correct (high is exclusive); verify no other direct callers remain
Passing groupHi = pos / groupSize matches the half-open [lo, hi) contract.
Sandbox rg returned "No files were searched" — run locally to confirm no remaining callers:
rg -nP -C3 -uu --type=java 'IntGroupSort.quickSort\s*('
rg -nP -C3 -uu '\bquickSort\s*('

bluestreak01
bluestreak01 previously approved these changes Sep 18, 2025
bluestreak01
bluestreak01 previously approved these changes Sep 24, 2025
@jerrinot
Copy link
Copy Markdown
Contributor

I only reviewed and tested 4f38f32 and it look fine - the test failure is no longer reproducible.

@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 25 / 25 (100.00%)

file detail

path covered line new line coverage
🔵 io/questdb/std/IntList.java 3 3 100.00%
🔵 io/questdb/std/IntGroupSort.java 22 22 100.00%

@bluestreak01 bluestreak01 merged commit 0a1a4e5 into master Sep 24, 2025
35 checks passed
@bluestreak01 bluestreak01 deleted the fix-error-index-drop branch September 24, 2025 21:50
bluestreak01 pushed a commit that referenced this pull request Oct 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exception in drop index, table suspended

4 participants