Skip to content

Commit 3c593b0

Browse files
committed
[feature](be) Add file range read scheduler
### What problem does this PR solve? Issue Number: Part of apache#67611 Related PR: apache#67292 Problem Summary: Query read-ahead needs to submit a complete batch of planned file ranges without blocking the scanner or coupling execution to page and cache-block policy. Add a generic asynchronous range reader with atomic query and BE resident-buffer byte admission, exact private buffers, worker-owned IO context, cancellation, shutdown, and terminal-state notification. Each admitted range is submitted directly as one task to the shared SegmentPrefetchThreadPool, which owns task queuing and controls physical read concurrency. The scheduler keeps no private task queue or range/concurrency quota. Byte reservations follow the last buffer reference so submitted and completed read-ahead data share the same memory bound. Account only accepted executor tasks as submitted; separate budget/executor rejection from read failure and queued cancellation. Record source/successful-source/local/remote bytes, timing, and BE cumulative IO totals independently of Profile lifetime, with accounting and cancellation-race tests. ### Release note None ### Check List (For Author) - Test: Unit Test - Component tests are included with the implementation. Focused ASAN/PCH results after this history fold are recorded in apache#67611 and apache#67612. - Repository clang-format 16, build hygiene and git diff --check. - Behavior changed: Yes (component functionality includes its observability and applicable cache-reuse checks; scheduling limits and query-result semantics are unchanged) - Does this need documentation: Yes (tracking issue apache#67611, split PR apache#67612, and the read-ahead observability document)
1 parent 0295d45 commit 3c593b0

3 files changed

Lines changed: 1796 additions & 0 deletions

File tree

0 commit comments

Comments
 (0)