perf(sql): improve query scalability on large multicore machines#6459
perf(sql): improve query scalability on large multicore machines#6459bluestreak01 merged 8 commits intomasterfrom
Conversation
puzpuzpuz
added
SQL
WalkthroughAdds a new SQL parallel filter dispatch limit configuration parameter and infrastructure to control in-flight task dispatch for parallel filter operations. Introduces new Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes
Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
puzpuzpuz
force-pushed
the
puzpuzpuz_query_scalability_fix
branch
from
ef5e690 to
f5434e7
Compare
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (7)
core/src/main/resources/io/questdb/site/conf/server.conf (1)
734-736: Clarify default/auto semantics for the new dispatch limit (optional).The new option is well placed and described, but consider briefly stating what happens when it’s unset or set to non‑positive values (e.g., auto‑calculated vs. “unlimited”) to keep this consistent with nearby Cairo SQL tuning comments.
core/src/main/java/io/questdb/griffin/engine/PerWorkerLocks.java (1)
41-55: Cache-line padding and index changes look correct; consider reusing a shared cache-line constant.The padding strategy (
INTS_PER_SLOT * workerCountand indexing viaINTS_PER_SLOT * id) preserves external semantics (slotremains a logical worker index) and is bounds-safe while reducing false sharing on the lock array. If the codebase already exposes a cache-line size constant (e.g., inOs), you could deriveINTS_PER_SLOTfrom it to avoid duplicating the 64‑byte assumption, but that’s purely cosmetic.Also applies to: 62-63, 76-77, 87-88
core/src/test/java/io/questdb/test/PropServerConfigurationTest.java (1)
2020-2042: Assertion forgetSqlParallelFilterDispatchLimit()is appropriate; consider a small default/validation test.This new check correctly ties the test server.conf value (100) through
PropServerConfigurationand theCairoConfiguration/wrapper API. As a follow‑up, you might add a focused test that covers the default dispatch limit when the property is unset (and possibly how zero/negative values are handled) to lock in the intended behavior.core/src/test/java/io/questdb/test/cairo/fuzz/ParallelGroupByFuzzTest.java (2)
362-378: Revisit “exactly PAGE_FRAME_MAX_ROWS” partition cases vs new MIN-based constantsThe branches commented as “Add partition with exactly PAGE_FRAME_MAX_ROWS” / “PAGE_FRAME_MAX_ROWS + 1” now use
MIN_PAGE_FRAME_MAX_ROWSinstead of the actualconfiguration.getSqlPageFrameMaxRows(). That means you only hit a true “exactly max rows” partition when the runtime max equals the minimum; otherwise you’re just near the limit. If the intent is to exercise specific equality branches in count_distinct merge logic, consider deriving these row counts fromconfiguration.getSqlPageFrameMaxRows()(or updating comments to say “around the page-frame size”).
1943-1946: Timeout fuzzing via sharedrndis fine; adjust comments to match randomized frame countsBoth timeout tests now choose
tripWhenTicksfrom the sharedrnd(nextLong(39)/nextLong(48)withMath.max(10, …)), which is deterministic per test instance and gives a reasonable spread of timeout points. However, the comments stating “Page frame count is 40.” (and “Page frame count is 40 and shard count is 8.”) are no longer strictly accurate oncepageFrameMaxRowsis randomized, so it would be clearer to soften or update those comments to avoid confusion when debugging.Also applies to: 2693-2696
core/src/main/java/io/questdb/cairo/sql/async/PageFrameSequence.java (1)
498-507: Dispatch-limit handling indispatch(int)looks correct; consider guarding non-positive limitsUsing
totalDispatched = dispatchStartFrameIndex - (collectedFrameIndex + 1)to capreducePubSeq.next()whentotalDispatched >= dispatchLimitgives a clean notion of “in-flight but not yet collected” frames, and treating that as equivalent to a full queue preserves the existing control flow (including work-stealing and local reduction). One caveat: if a caller ever passesdispatchLimit <= 0, this will bypass the queue entirely and force all work throughworkLocally(), which may or may not be what you want; you could defensively clampdispatchLimitto at least 1 or assert on invalid values at thenext(int)entry point.Also applies to: 526-529
core/src/main/java/io/questdb/PropServerConfiguration.java (1)
455-456: Confirm intended default forcairo.sql.parallel.filter.dispatch.limitThe new config is wired end-to-end (field, constructor init, and CairoConfiguration getter) and looks mechanically correct, but the default is
Math.min(queryWorkers, 32), i.e., it caps the in‑flight dispatch limit at 32 on large machines. The PR description mentions a default ofmax(cpu_cores, 32), which would grow with core count instead; these are meaningfully different behaviors on big boxes. Please double-check which behavior you want and either (a) adjust the default expression here or (b) update the docs/description and any tests expecting themax(cpu_cores, 32)semantics. Also consider whether you want to validate that configured values are non‑negative / non‑zero, or explicitly document how0/ negative values are interpreted by the filter cursors.Also applies to: 1842-1843, 3885-3888
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (22)
core/src/main/java/io/questdb/PropServerConfiguration.java(4 hunks)core/src/main/java/io/questdb/PropertyKey.java(1 hunks)core/src/main/java/io/questdb/cairo/CairoConfiguration.java(1 hunks)core/src/main/java/io/questdb/cairo/CairoConfigurationWrapper.java(1 hunks)core/src/main/java/io/questdb/cairo/DefaultCairoConfiguration.java(1 hunks)core/src/main/java/io/questdb/cairo/sql/RecordCursor.java(1 hunks)core/src/main/java/io/questdb/cairo/sql/async/PageFrameSequence.java(6 hunks)core/src/main/java/io/questdb/griffin/engine/PerWorkerLocks.java(4 hunks)core/src/main/java/io/questdb/griffin/engine/orderby/LimitedSizePartiallySortedLightRecordCursor.java(2 hunks)core/src/main/java/io/questdb/griffin/engine/table/AsyncFilteredNegativeLimitRecordCursor.java(4 hunks)core/src/main/java/io/questdb/griffin/engine/table/AsyncFilteredRecordCursor.java(10 hunks)core/src/main/java/io/questdb/griffin/engine/table/AsyncGroupByAtom.java(4 hunks)core/src/main/java/io/questdb/griffin/engine/table/AsyncJitFilteredRecordCursorFactory.java(2 hunks)core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursor.java(1 hunks)core/src/main/java/io/questdb/griffin/engine/table/VirtualFunctionRecordCursor.java(1 hunks)core/src/main/resources/io/questdb/site/conf/server.conf(1 hunks)core/src/test/java/io/questdb/test/PropServerConfigurationTest.java(1 hunks)core/src/test/java/io/questdb/test/ServerMainTest.java(2 hunks)core/src/test/java/io/questdb/test/cairo/fuzz/ParallelGroupByFuzzTest.java(6 hunks)core/src/test/java/io/questdb/test/griffin/ParallelFilterTest.java(1 hunks)core/src/test/resources/server.conf(1 hunks)pkg/ami/marketplace/assets/server.conf(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-11-10T14:28:48.329Z
Learnt from: mtopolnik
Repo: questdb/questdb PR: 0
File: :0-0
Timestamp: 2025-11-10T14:28:48.329Z
Learning: In AsOfJoinDenseRecordCursorFactoryBase.java, the `backwardScanExhausted` flag is intentionally NOT reset in `toTop()` because backward scan results are reusable across cursor rewinds. The backward scan caches historical matches that remain valid when the cursor is rewound.
Applied to files:
core/src/main/java/io/questdb/griffin/engine/table/AsyncFilteredRecordCursor.javacore/src/main/java/io/questdb/griffin/engine/orderby/LimitedSizePartiallySortedLightRecordCursor.javacore/src/main/java/io/questdb/griffin/engine/table/AsyncFilteredNegativeLimitRecordCursor.java
📚 Learning: 2025-11-19T12:21:00.062Z
Learnt from: jerrinot
Repo: questdb/questdb PR: 6413
File: core/src/test/java/io/questdb/test/cutlass/pgwire/PGJobContextTest.java:11982-12002
Timestamp: 2025-11-19T12:21:00.062Z
Learning: QuestDB Java tests use a deterministic random seed. The test utilities (e.g., io.questdb.test.tools.TestUtils and io.questdb.std.Rnd) produce reproducible sequences, so rnd_* functions (including rnd_uuid4) yield deterministic outputs across runs. Do not flag tests in core/src/test/** that assert against values produced by rnd_* as flaky due to randomness.
Applied to files:
core/src/test/java/io/questdb/test/griffin/ParallelFilterTest.javacore/src/test/java/io/questdb/test/cairo/fuzz/ParallelGroupByFuzzTest.java
🧬 Code graph analysis (3)
core/src/main/java/io/questdb/griffin/engine/table/AsyncFilteredRecordCursor.java (1)
core/src/main/java/io/questdb/cairo/sql/PageFrameMemoryRecord.java (1)
PageFrameMemoryRecord(62-740)
core/src/test/java/io/questdb/test/griffin/ParallelFilterTest.java (1)
core/src/test/java/io/questdb/test/tools/TestUtils.java (1)
TestUtils(154-2690)
core/src/test/java/io/questdb/test/cairo/fuzz/ParallelGroupByFuzzTest.java (1)
core/src/test/java/io/questdb/test/tools/TestUtils.java (1)
TestUtils(154-2690)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (36)
- GitHub Check: build
- GitHub Check: New pull request (Coverage Report Coverage Report)
- GitHub Check: New pull request (SelfHosted Running tests with cover on linux-other)
- GitHub Check: New pull request (SelfHosted Running tests with cover on linux-pgwire)
- GitHub Check: New pull request (SelfHosted Running tests with cover on linux-cairo-sub)
- GitHub Check: New pull request (SelfHosted Running tests with cover on linux-cairo-root)
- GitHub Check: New pull request (SelfHosted Running tests with cover on linux-fuzz2)
- GitHub Check: New pull request (SelfHosted Running tests with cover on linux-fuzz1)
- GitHub Check: New pull request (SelfHosted Running tests with cover on linux-griffin-sub)
- GitHub Check: New pull request (Rust Test and Lint on linux-jdk17)
- GitHub Check: New pull request (SelfHosted Running tests with cover on linux-griffin-root)
- GitHub Check: New pull request (Hosted Running tests on windows-other-2)
- GitHub Check: New pull request (Hosted Running tests on windows-other-1)
- GitHub Check: New pull request (Hosted Running tests on windows-pgwire)
- GitHub Check: New pull request (Hosted Running tests on windows-cairo-2)
- GitHub Check: New pull request (Hosted Running tests on windows-cairo-1)
- GitHub Check: New pull request (Hosted Running tests on windows-fuzz2)
- GitHub Check: New pull request (Hosted Running tests on windows-fuzz1)
- GitHub Check: New pull request (Hosted Running tests on windows-griffin-sub)
- GitHub Check: New pull request (Hosted Running tests on windows-griffin-base)
- GitHub Check: New pull request (Hosted Running tests on mac-other)
- GitHub Check: New pull request (Hosted Running tests on mac-pgwire)
- GitHub Check: New pull request (Hosted Running tests on mac-cairo-fuzz)
- GitHub Check: New pull request (Hosted Running tests on mac-cairo)
- GitHub Check: New pull request (Hosted Running tests on mac-griffin)
- GitHub Check: New pull request (SelfHosted Other tests on linux-x86-graal)
- GitHub Check: New pull request (SelfHosted Other tests on linux-x64-zfs)
- GitHub Check: New pull request (SelfHosted Griffin tests on linux-x86-graal)
- GitHub Check: New pull request (SelfHosted Griffin tests on linux-arm64)
- GitHub Check: New pull request (SelfHosted Other tests on linux-arm64)
- GitHub Check: New pull request (SelfHosted Cairo tests on linux-x64-zfs)
- GitHub Check: New pull request (SelfHosted Griffin tests on linux-x64-zfs)
- GitHub Check: New pull request (SelfHosted Cairo tests on linux-x86-graal)
- GitHub Check: New pull request (SelfHosted Cairo tests on linux-arm64)
- GitHub Check: New pull request (Trigger Enterprise CI Trigger Enterprise Pipeline)
- GitHub Check: New pull request (Check Changes Check changes)
🔇 Additional comments (30)
core/src/main/java/io/questdb/griffin/engine/orderby/LimitedSizePartiallySortedLightRecordCursor.java (1)
118-118: Good optimization signal for LIMIT queries.The call to
expectLimitedIteration()correctly signals to the base cursor that limited iteration will occur, enabling potential dispatch and parallelization optimizations for LIMIT N queries. The placement before obtaining the base record is appropriate.core/src/main/java/io/questdb/griffin/engine/table/AsyncGroupByAtom.java (4)
241-257: LGTM - cleaner two-pass approach for heap size normalization.The separation of max-finding and application into distinct loops improves readability and makes the intent clear.
397-405: LGTM - accumulating function cardinality at fragment level.The design tracks cumulative cardinality per fragment across the query execution, with proper reset in
close(). Each worker operates on its own fragment, avoiding contention.
549-556: LGTM - clean helper extraction.The helper properly delegates to
getGroupByFunctions(slotId)to handle owner vs. worker slots.
659-665: LGTM - new field properly managed.The
totalFunctionCardinalityfield is correctly reset inclose()and accumulated inmaybeEnableSharding().core/src/main/java/io/questdb/PropertyKey.java (1)
98-103: New PropertyKey for dispatch limit is consistent and correct.The enum constant name and property path match the configuration files and follow existing naming patterns; this should integrate cleanly with validators and env overrides.
core/src/test/resources/server.conf (1)
156-160: Test config value for dispatch limit is consistent with the new tests.The new
cairo.sql.parallel.filter.dispatch.limit=100entry is correctly grouped with other parallel filter settings and aligns with the expectations inPropServerConfigurationTest.core/src/main/java/io/questdb/griffin/engine/table/VirtualFunctionRecordCursor.java (1)
89-92: LGTM!The delegation pattern is consistent with other cursor methods in this class and aligns with the parallel implementations in
SelectedRecordCursorand other wrapper cursors.core/src/test/java/io/questdb/test/ServerMainTest.java (1)
324-324: Test expectations updated correctly.The expected values align with the PR's configuration changes:
- Queue capacities now use formula
4 * queryWorkers(with 2 workers → 8)- New
cairo.sql.parallel.filter.dispatch.limitparameter added with value matching worker countAlso applies to: 403-403, 406-406
core/src/main/java/io/questdb/cairo/sql/RecordCursor.java (1)
144-151: Well-designed API extension.The new hook method follows good interface design practices:
- Default no-op implementation allows incremental adoption by cursor implementations
- Clear documentation explains the optimization intent for LIMIT N queries
- Consistent with existing default methods in the interface
core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursor.java (1)
62-65: LGTM!Correct delegation pattern that propagates the limited iteration signal through the cursor chain, consistent with other delegating methods in this class.
core/src/main/java/io/questdb/griffin/engine/table/AsyncFilteredNegativeLimitRecordCursor.java (3)
64-64: LGTM - dispatch limit integration.The
dispatchLimitfield is appropriately declared asfinalsince it's configuration-based and should remain constant for the cursor's lifetime.Also applies to: 85-85
183-183: Dispatch limit now applied to frame fetching.Passing
dispatchLimittoframeSequence.next()limits in-flight page frame tasks, which is the core optimization for improving latency on LIMIT N queries as described in the PR objectives.
234-240: I'll help you verify whether pattern matching for instanceof is consistently used throughout the codebase. Let me search for exception handling patterns to assess consistency.
<function_calls>
#!/bin/bashSearch for instanceof CairoException patterns to check consistency
rg -n 'instanceof\s+CairoException' --type java -C2 | head -60
</function_calls>
#!/bin/bashCount pattern matching vs traditional cast approach for CairoException
echo "=== Pattern matching style (instanceof Type var) ==="
rg -c 'instanceof\s+CairoException\s+\w+' --type java | wc -lecho -e "\n=== Traditional style (instanceof followed by cast) ==="
rg -n 'instanceof CairoException' --type java | grep -v 'instanceof CairoException [a-zA-Z_]' | head -20
</function_calls>
#!/bin/bashMore comprehensive search for all instanceof patterns with CairoException
rg 'instanceof\s+CairoException' --type java | wc -l
echo "---"
rg 'instanceof\s+CairoException\s+\w+' --type java | wc -l
</function_calls>core/src/main/java/io/questdb/cairo/CairoConfigurationWrapper.java (1)
999-1002: LGTM!The new delegation method follows the established pattern in this wrapper class and correctly forwards the call to the underlying configuration.
pkg/ami/marketplace/assets/server.conf (1)
549-550: LGTM!The commented configuration directive is properly documented and placed in the appropriate section. This follows the standard pattern for introducing new configuration options.
core/src/main/java/io/questdb/cairo/CairoConfiguration.java (1)
587-587: LGTM!The new configuration method is well-positioned alongside related parallel filter configuration accessors and follows the established naming convention.
core/src/main/java/io/questdb/cairo/DefaultCairoConfiguration.java (1)
1025-1028: LGTM - Unlimited default is appropriate for test configuration.Returning
Integer.MAX_VALUEeffectively disables the dispatch limit in the default test configuration, which is appropriate for ensuring tests aren't artificially constrained. Production configurations will provide actual values viaPropServerConfiguration.core/src/main/java/io/questdb/griffin/engine/table/AsyncFilteredRecordCursor.java (4)
156-160: Clarify the interaction betweenexpectLimitedIteration()andof().Both
expectLimitedIteration()(Line 159) andof()(Lines 409-410) set thedispatchLimitfield, but with potentially different logic:
expectLimitedIteration()unconditionally setsdispatchLimit = defaultDispatchLimitof()setsdispatchLimitbased on whetherrowsRemaining != Long.MAX_VALUEWhen are these methods called relative to each other? If
of()is always called during cursor initialization, doesexpectLimitedIteration()serve as an override mechanism, or is it meant to be called beforeof()? The current implementation doesn't clearly communicate the intended call sequence.Consider documenting the expected call sequence or consolidating the dispatch limit initialization logic to avoid potential confusion.
409-410: Verify dispatch limit initialization logic.The comment states "put a cap the number of in-flight page frame tasks in case of LIMIT N query", and the code sets
dispatchLimit = rowsRemaining != Long.MAX_VALUE ? defaultDispatchLimit : Integer.MAX_VALUE.However, there's also an
expectLimitedIteration()method (Lines 156-160) that sets the dispatch limit. Ensure the initialization logic between these two paths is intentional and well-coordinated.
73-79: LGTM!The constructor initialization is clean and properly ordered:
- Uses the constant
PageFrameMemoryRecord.RECORD_A_LETTERfor clarity- Initializes frame memory pool with appropriate cache capacity
- Reads the new dispatch limit configuration value
329-337: LGTM!The
fetchNextFrame()method correctly accepts and propagates thedispatchLimitparameter toframeSequence.next(). All call sites have been consistently updated to pass thedispatchLimitfield.core/src/test/java/io/questdb/test/griffin/ParallelFilterTest.java (2)
166-171: LGTM - Enhanced test coverage through randomization.The introduction of the
rndfield and its use to varyconvertToParquetimproves test coverage by exercising different execution paths. Per established patterns in QuestDB tests, the random seed is deterministic (viaTestUtils.generateRandom()), ensuring reproducibility across test runs.Based on learnings, QuestDB tests produce deterministic random sequences.
174-183: LGTM - Randomized configuration enhances edge case testing.The test now varies several configuration parameters:
CAIRO_PAGE_FRAME_SHARD_COUNT: 1-4 shardsCAIRO_SQL_PARALLEL_FILTER_DISPATCH_LIMIT: 1-4 in-flight tasksCAIRO_SQL_PARALLEL_WORK_STEALING_THRESHOLD: 1-16 thresholdThis randomization exercises various edge cases (e.g., single shard, minimal dispatch limit) while remaining deterministic due to the seeded RNG. The comment "We intentionally use small values... to exhibit various edge cases" aligns well with this approach.
core/src/main/java/io/questdb/griffin/engine/table/AsyncJitFilteredRecordCursorFactory.java (1)
81-139: LGTM - Parameter rename improves consistency.The parameter rename from
sharedQueryWorkerCounttoworkerCountsimplifies the signature while maintaining clarity. The internal field namesharedQueryWorkerCountis preserved for use in diagnostic output (Line 264 intoPlan()), which provides useful context when inspecting query plans.core/src/test/java/io/questdb/test/cairo/fuzz/ParallelGroupByFuzzTest.java (3)
66-75: Random seed promotion and ROW_COUNT definition look solidUsing
MIN_PAGE_FRAME_MAX_ROWSas the base forROW_COUNTand promotingrndto a field keeps the relationship between data volume and page-frame sizing explicit, whileTestUtils.generateRandom(LOG)preserves deterministic seeds across runs. No functional issues here.
Based on learnings, this remains deterministic across runs.
84-95: Randomized parallelism and dispatch configuration in setUp() is reasonableRandomizing
pageFrameMaxRows, shard count, reduce queue capacity, sharding/work-stealing thresholds, and the newCAIRO_SQL_PARALLEL_FILTER_DISPATCH_LIMIT(bounded byPAGE_FRAME_COUNT) increases fuzz coverage for a variety of parallel configurations while staying in safe ranges. This should help exercise the new dispatch-limit path without introducing obvious flakiness.
4081-4083: Assumptions intestParallelGroupByThrowsOnTimeoutalign with randomized setupThe new
Assume.assumeTrue(configuration.getGroupByShardingThreshold() <= 5)andROW_COUNT / configuration.getSqlPageFrameMaxRows() >= 20checks are consistent withsetUp()(threshold ∈ [1,50], page-frame max rows ∈ [100,109]), so the test only runs when there are many frames and a small sharding threshold, which is sensible for a timeout-focused scenario. Theinsert into tab ... long_sequence(" + ROW_COUNT + ")line continues to use the globalROW_COUNTand looks correct with the updated constants.Also applies to: 4121-4121
core/src/main/java/io/questdb/cairo/sql/async/PageFrameSequence.java (1)
326-343: next() overload and prepareForDispatch Javadoc remain consistentExposing a parameterless
next()that simply delegates tonext(Integer.MAX_VALUE)keeps existing callers source- and behavior-compatible, while allowing new users to pass an explicit dispatch limit. The updated Javadoc onprepareForDispatch()referencingnext(int)instead ofnext()correctly reflects this API shift; contract and threading assumptions stay the same.Also applies to: 430-435
core/src/main/java/io/questdb/PropServerConfiguration.java (1)
1830-1841: Parallel reduce/merge queue default scaling looks goodBumping
defaultReduceQueueCapacitytoMath.min(4 * queryWorkers, 256)and using it consistently for bothcairoPageFrameReduceQueueCapacityandcairoGroupByMergeShardQueueCapacitymatches the PR intent (better utilization on large-core machines) while still capping growth and keeping theceilPow2behavior. No issues from a correctness or config-compatibility perspective.
core/src/main/java/io/questdb/griffin/engine/table/AsyncGroupByAtom.java
Show resolved
Hide resolved
[PR Coverage check]😍 pass : 51 / 53 (96.23%) file detail
|
3 participants
Includes the following changes aimed to improve query engine scalability on multi-core machines:
MapFragment#totalFunctionCardinalitynot being updated in all cases inAsyncGroupAtom. Due to that, sharding stats were flapping from enable sharding to disable sharding between subsequent runs in case of queries with high-cardinalitycount_distinct().AsyncGroupAtomare now calculated as a sum of recorded group by function cardinalities instead of a maximum value. This way is a better estimate of the amount of work we need to do to merge all of the functions.cairo.page.frame.reduce.queue.capacityandcairo.sql.parallel.groupby.merge.shard.queue.capacityto4 * cpu_coreswith a cap of256. This improves worker thread utilization for parallel GROUP BY and filter queries.cairo.sql.parallel.filter.dispatch.limitconfig option (set tomax(cpu_cores, 32)by default) that limits the number of in-flight page frame tasks dispatched by parallel filter factories in case of LIMIT N queries. The goal is improve the latency of LIMIT queries.PerWorkerLocksto avoid false sharing.ParallelGroupByFuzzTestandParallelFilterTest.Benchmarks
ClickBench difference vs. the
masterbranch on c6a.metal, 500GB gp2: