You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit e0bf487
Browse filesBrowse the repository at this point in the historyBrowse files
[improvement](be) Use thread pool scan scheduler by default (apache#68610)
### What problem does this PR solve?
Issue Number: None
Related PR: apache#51690, apache#58144, apache#58273, apache#65814, apache#67070
Problem Summary: Workload groups pick the scan scheduler for internal
and
external table scans from `enable_task_executor_in_internal_table` and
`enable_task_executor_in_external_table`. Both configs currently default
to
`true` on master, so every workload group creates
`TaskExecutorSimplifiedScanScheduler` (the time-sharing task executor)
for
local (`ls_*`) and remote (`rs_*`) scans. The time-sharing executor is
still
maturing (for example the self-deadlock fixed in apache#58273; the
internal-table
default was already reverted once in apache#58144), so the default should be
the
`ThreadPoolSimplifiedScanScheduler` for both internal and external
tables.
This change sets both configs' default values to `false`. The task
executor
can still be enabled explicitly through `be.conf`.
Making the thread pool path the default exposed four gaps.
1. Pending scanners after an exhausted shared LIMIT. For a `SELECT ...
LIMIT n`
over many tablets, the thread pool admission
(`ScannerContext::can_admit_scan_task`,
rewritten in apache#67070) keeps admitting the remaining pending scanners
after peer
scanners have exhausted the shared scan LIMIT. Admission is kept that
way on
purpose: holding pending scanners back based on the in-flight or
completed
task counters, as apache#65814 does on the TaskExecutor path, ties the
schedule to
bookkeeping that is easy to get wrong and can leave a context with
nothing to
wake the operator. Instead, `Scanner::get_block_after_projects()` now
checks
the shared LIMIT before it reads: once the counter is exhausted, the
scanner
returns an empty block, so an admitted scanner never reads its range for
rows
the operator will cut anyway. It does not report EOS for that: EOS means
the
underlying read is exhausted, while the LIMIT is applied by the scan
operator,
which stops the scanners after `reached_limit()`;
`ScannerContext::get_block_from_queue()` also finishes the context once
the
counter is exhausted with nothing in flight. The two pre-existing checks
in
`Scanner::get_block()` that set `eof` from the shared counter are
removed for
the same reason, so the shared LIMIT is handled in one place.
2. Rows held in scanner padding under a shared LIMIT.
`Scanner::get_block()`
charges the shared LIMIT counter as soon as rows pass the filters, but
with
projections `get_block_after_projects()` may keep a small block in
`_padding_block` and emit a later block first. Once peer scanners
exhaust the
counter, the context reaches EOS without running that scanner again, and
the
charged rows it holds are lost; for example, a parallel projected
`LIMIT 4096` could return 4086 rows. The counter is now decremented only
by
the rows a call actually returns, so held padding is never charged and
the
counter equals "LIMIT minus returned rows" between calls. The padding
batch
target is capped by the remaining LIMIT when a call starts, so a scanner
does
not read on just to fill half a batch when fewer rows are needed, and
the
read loop stops as soon as peers exhaust the LIMIT, returning the rows
it
already holds. Projected scans with selective filters keep coalescing
small
blocks. The scan does not cap its output at the LIMIT (the operator cuts
extra rows), so no further accounting is needed.
3. Exceptions in the row-ID fetch RPC callback. `multiget_data_v2`
submits
its callback to the remote scan scheduler; on a `ThreadPool` worker
there is
no exception boundary, and `RowIdStorageReader::read_by_rowids()`
constructs
`SlotDescriptor(pslot)` before its local `try`, so an unknown
`PTypeNode.type` would throw `doris::Exception` and terminate the BE.
The
callback now calls `RowIdStorageReader::read_by_rowids_for_rpc()`, which
converts a thrown exception into an error status on the response before
the
closure completes the RPC. `ExecEnv::set_id_manager()` is added under
`BE_TEST` for the unit test.
4. The shared LIMIT pointer of a scanner initialized while the counter
is
negative. `ScanLocalState::shared_scan_limit_ptr()` returned null
whenever
the live counter was negative, and `Scanner::init()` caches that pointer
once. Concurrent scanners can return more than the LIMIT before the
operator
makes its final cut, leaving the counter negative, so a scanner whose
local
state opened at that moment never checked or charged the shared LIMIT
and
could read a long selective tail after peers had satisfied the query.
The pointer is now returned whenever the operator has a SQL LIMIT, which
is
how `ScannerContext` already decides it.
### Release note
The BE configs `enable_task_executor_in_internal_table` and
`enable_task_executor_in_external_table` now default to `false`;
internal
and external table scans use the thread pool scan scheduler by default.
### Check List (For Author)
- Test:
- Unit Test: added
`WorkloadGroupManagerTest.DefaultScanSchedulersUseThreadPool`,
`ScannerContextTest.thread_pool_admission_ignores_shared_limit`,
`ScannerProjectionTest.shared_limit_exhausted_before_call_returns_empty_block_without_eos`,
`ScannerProjectionTest.shared_limit_counts_only_emitted_rows`,
`ScannerProjectionTest.shared_limit_drops_padding_after_peer_satisfies_limit`,
`ScannerProjectionTest.shared_limit_flushes_padding_before_reading_more`,
`ScannerProjectionTest.shared_limit_flushes_merged_padding_at_limit`,
`ScannerProjectionTest.shared_limit_counts_returned_rows_without_projection`,
`ScannerProjectionTest.shared_limit_counts_returned_rows_without_padding`,
`ScannerProjectionTest.shared_limit_keeps_padding_small_blocks`,
`ScannerProjectionTest.shared_limit_exhausted_by_own_rows_stops_reading_without_eos`,
`ScannerProjectionTest.shared_limit_exhausted_by_peer_mid_read_returns_held_rows_without_eos`,
`ScannerProjectionTest.shared_limit_observed_by_scanner_initialized_under_negative_counter`,
`ScannerProjectionTest.no_shared_limit_without_sql_limit`
and
`RowIdStorageReaderTest.RpcHelperReportsThrownExceptionOnThreadPoolPath`
(runs the RPC helper on a real `ThreadPoolSimplifiedScanScheduler`;
the test process aborts without the fix);
ran `ScannerProjectionTest.*`, `ScannerContextTest.*`,
`ScannerLateArrivalRfTest.*`, `WorkloadGroupManagerTest.*`,
`RowIdStorageReaderTest.*`, `SubmitExternalScanTasksTest.*`
- Regression test: ran on a local cluster with the new
default: query_p0/limit (19 suites) and
correctness_p0/test_shared_scan_limit_pending_tasks
- Behavior changed: Yes (default scan scheduler for internal and
external
table scans changes from the task executor to the thread pool; it can be
switched back via the two BE configs)
- Does this need documentation: No
0 commit comments