Repository navigation
branch-4.2: [improvement](be) Use thread pool scan scheduler by default #68610 - #69508
Merged
Merged
Conversation
### What problem does this PR solve? Issue Number: None Related PR: apache#68610 Problem Summary: Both internal and external scans use the task-executor scheduler by default on branch-4.2, while the thread-pool scheduler is the intended stable default. Switch both defaults to the thread pool, retaining explicit task-executor opt-in. A row-ID fetch RPC callback runs on a plain thread-pool worker, so a Doris exception while decoding a request could terminate the BE; convert it to the response status before completing the RPC. Cherry-picked from apache#68610. Conflict resolution: branch-4.2 has no shared scan LIMIT counter or the master-side scanner/context APIs, so the master-only shared-LIMIT changes and tests were not ported. Keep the branch-native scanner implementation and adapt the scheduler/RPC tests to branch-4.2 headers. No unrelated master changes were imported. (cherry picked from commit e0bf487) ### Release note Internal and external scans use the thread-pool scan scheduler by default. Set enable_task_executor_in_internal_table or enable_task_executor_in_external_table to true to opt back into the task executor. ### Check List (For Author) - Test: Unit Test: 82 focused BE ASAN tests passed, including RowIdStorageReaderTest and WorkloadGroupManagerTest. - Regression test: query_p0/limit, 18 suites passed on an isolated local FE/BE cluster. - Manual test: ASAN BE+FE build, clang-format, check-format, FE checkstyle and git diff --check passed. - Behavior changed: Yes (default scheduler selection for both internal and external scans). - Does this need documentation: No
Member
Author
|
run buildall |
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
mrhhsg
marked this pull request as ready for review
October 24, 2026 03:08
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
yiguolei
approved these changes
Oct 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backport scope
Cherry-picked from #68610 for branch-4.2. This branch does not have the master-side shared scan LIMIT counter or the associated scanner/context APIs, so its existing scanner behavior is retained. The backport includes the scheduler defaults, the row-ID RPC exception boundary, and focused tests without importing unrelated master-side scan changes.
Tests
BUILD_TYPE=ASAN ./build.sh --be --fe -j 32— passed.query_p0/limitregression on an isolated local FE/BE cluster — 18 suites passed.clang-format.sh,check-format.sh, andgit diff --check— passed.Release note
Internal and external scans now use the thread-pool scheduler by default. Set
enable_task_executor_in_internal_tableorenable_task_executor_in_external_tabletotrueto opt back into the task executor for the respective scan type.