perf(sql): avoid expensive row counting in EXPLAIN query with LIMIT#6540
Merged
bluestreak01 merged 93 commits intomasterfrom Dec 22, 2025
Merged
perf(sql): avoid expensive row counting in EXPLAIN query with LIMIT#6540bluestreak01 merged 93 commits intomasterfrom
bluestreak01 merged 93 commits intomasterfrom
Conversation
skipRows() -> skipBaseRows() skippedRows -> baseRowsToSkip rowsToSkip -> pendingBaseRowsToSkip
... to resolveSizeAndGotoTop()
It no longer resolves actual cursor size when LIMIT bounds aren't negative.
.. and skipRows
Contributor
[PR Coverage check]😍 pass : 199 / 201 (99.00%) file detail
|
bluestreak01
approved these changes
Dec 22, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In the current code,
LimitRecordCursorFactory.toPlan()callscalculateSize()to resolve the effective LIMIT bounds. This executes the complete query. This PR letstoPlan()work with the information that's already available and never make the expensive call.The PR also resolves other related issues with
LimitRecordCursorFactory:calculateSize()inhasNext()when LIMIT bounds are non-negative (no need to count from the back)calculateSize()insize(), propagating -1 if the size isn't cheaply availableOn the other hand, the new code calls
baseCursor.size()eagerly inof(). This provides the invariant: "base row count missing indicates the expensive count is unavoidable".The PR clears up the semantics of LIMIT, which are currently in contradiction with the specification in the docs.
Cleaned-up LIMIT semantics
The main complication in the semantics stems from LIMIT args being allowed to be both positive and negative, and even a mix of both. Here are the new rules (
mandnare positive numbers, so-mis a negative number):LIMIT n=LIMIT n, 0=LIMIT 0, n=LIMIT n,=LIMIT , n: take firstnrowsLIMIT m, n: take rows from the zero-based range[m, n)(left-inclusive, right-exclusive). Ifm > n, implicitly swap them.LIMIT -n=LIMIT -n, 0=LIMIT -n,: take lastnrowsLIMIT -m, -n: take rows from the range[-m, -n), where -1 denotes the last row, -2 next-to-last, and so on. Ifm < n, implicitly swap them.LIMIT m, -n: take rows from the range[m, -n). In other words, skip the firstmrows, and drop the lastnrows. These arguments will not be swapped.Additional bugfixes
Since it touches
LimitRecordCursorFactory.calculateSize()andskipRows(), this PR surfaced a number of bugs in these methods in other cursor classes. It fixes them.