Repository navigation
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
bobhan1
force-pushed
the
fix/phase2-no-pch-build
branch
4 times, most recently
from
September 10, 2026 08:20
e46032a to
5c594ab
Compare
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67292, apache#67579 Problem Summary: Column read-ahead uses paired high and low byte watermarks to decide when to extend its window. Pending-byte accounting also changes after Page Cache hits and rejected submissions, even when the scan has not advanced. Replace the watermarks with one compressed-byte window size per column role and a trigger page position. Each window uses its second selected page as the trigger, or its only page for a one-page window. First access and a jump beyond planned coverage establish a window through the same path. Process current-batch pages in scan order so a batch crossing the trigger can append the following window in the same plan. Keep mandatory current-batch pages, cache probing, IO coalescing, asynchronous reads, and writeback on their existing paths. ### Release note Replace the eager/lazy high and low watermark settings with read_ahead_eager_window_bytes (4 MiB) and read_ahead_lazy_window_bytes (128 KiB). Query page read-ahead advances by scan position and retains its existing query and BE buffer budgets. ### Check List (For Author) - Test: Unit Test — 132 targeted ASAN tests, including 24 ColumnReadAhead cases and related scanner, rowid/TopN, range IO, and cache-reader tests; clang-format 16, build hygiene, and git diff --check passed. No cluster or regression workload was run. - Behavior changed: Yes — page-position triggers replace pending-byte watermarks; first access and large jumps share the same window initialization. - Does this need documentation: Yes — configuration and component interface comments are updated in this commit.
bobhan1
force-pushed
the
fix/phase2-no-pch-build
branch
from
September 11, 2026 04:49
a48e80c to
25eef4d
Compare
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67292, apache#67579 Problem Summary: Column read-ahead uses paired high and low byte watermarks to decide when to extend its window. Pending-byte accounting also changes after Page Cache hits and rejected submissions, even when the scan has not advanced. Replace the watermarks with one compressed-byte window size per column role and a trigger page position. Each window uses its second selected page as the trigger, or its only page for a one-page window. First access and a jump beyond planned coverage establish a window through the same path. Process current-batch pages in scan order so a batch crossing the trigger can append the following window in the same plan. Keep mandatory current-batch pages, cache probing, IO coalescing, asynchronous reads, and writeback on their existing paths. ### Release note Replace the eager/lazy high and low watermark settings with read_ahead_eager_window_bytes (4 MiB) and read_ahead_lazy_window_bytes (128 KiB). Query page read-ahead advances by scan position and retains its existing query and BE buffer budgets. ### Check List (For Author) - Test: Unit Test — 132 targeted ASAN tests, including 24 ColumnReadAhead cases and related scanner, rowid/TopN, range IO, and cache-reader tests; clang-format 16, build hygiene, and git diff --check passed. No cluster or regression workload was run. - Behavior changed: Yes — page-position triggers replace pending-byte watermarks; first access and large jumps share the same window initialization. - Does this need documentation: Yes — configuration and component interface comments are updated in this commit.
This was referenced Sep 11, 2026
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292 Problem Summary: Each hole-fill worker issued the source reads for a cache block sequentially. Keep block workers responsible for planning and writeback, and execute their disjoint source ranges in a separate shared pthread pool. A per-block concurrent ThreadPoolToken joins accepted reads before either submitting the completed buffer or dropping the block. Preserve queued reads during shutdown and allow the read pool to resize without interrupting active GETs. Make the shared S3 reader statistics atomic for concurrent foreground and background reads. ### Release note Partial File Cache blocks can fetch hole ranges concurrently. The new mutable BE setting hole_fill_read_threads_per_be defaults to 64 and controls the shared background source-read pool independently of the block-worker count. ### Check List (For Author) - Test: Unit Test — 132 related BE unit tests passed through run-be-ut.sh with ASAN, PCH and -j100; 8 concurrency/statistics tests also passed 3 shuffled repetitions. clang-format v16, check-format, build hygiene and git diff --check passed. - Behavior changed: Yes — source reads within one partial cache block can execute concurrently, and failed blocks are released after their accepted reads finish. - Does this need documentation: Yes — the new BE setting and execution/lifecycle contracts are documented in code comments.
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292 Problem Summary: Each hole-fill worker issued the source reads for a cache block sequentially. Keep block workers responsible for planning and writeback, and execute their disjoint remote ranges in a separate shared HoleFillRemoteReadPool. A per-block concurrent ThreadPoolToken joins accepted reads before either submitting the completed buffer or dropping the block. Preserve queued reads during shutdown and allow the remote-read pool to resize without interrupting active GETs. Make the shared S3 reader statistics atomic for concurrent foreground and background reads. Initialize both hole-fill and asynchronous cache-write worker pools with their configured worker counts as the minimum, and retain at least one remote-read pool thread. ### Release note Partial File Cache blocks can fetch hole ranges concurrently. The new mutable BE setting hole_fill_remote_read_threads_per_be defaults to 64 and controls the shared background remote-read pool independently of the block-worker count. ### Check List (For Author) - Test: Unit Test — 62 existing BE unit tests passed through run-be-ut.sh with ASAN, PCH and -j100, covering AsyncCacheWriteManagerTest, PartialBlockWritebackManagerTest and PartialBlockWritebackOptionsTest. The naming and minimum-thread follow-up adds no new test cases. clang-format v16, check-format, build hygiene and git diff --check passed. - Behavior changed: Yes — remote reads within one partial cache block can execute concurrently, and failed blocks are released after their accepted reads finish. Worker pools start with nonzero minimum thread counts. - Does this need documentation: Yes — the new BE setting and execution/lifecycle contracts are documented in code comments.
bobhan1
force-pushed
the
fix/phase2-no-pch-build
branch
from
September 11, 2026 07:13
3796127 to
d8d795d
Compare
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292 Problem Summary: Each hole-fill worker issued the source reads for a cache block sequentially. Keep block workers responsible for planning and writeback, and execute their disjoint remote ranges in a separate shared HoleFillRemoteReadPool. Each block worker reuses one concurrent ThreadPoolToken across blocks, joining each block's accepted reads before either submitting the completed buffer or dropping the block. Destroy the token before signalling worker completion so the remote-read pool can be released safely. Preserve queued reads during shutdown and allow the remote-read pool to resize without interrupting active GETs. Make the shared S3 reader statistics atomic for concurrent foreground and background reads. Initialize both hole-fill and asynchronous cache-write worker pools with their configured worker counts as the minimum, and retain at least one remote-read pool thread. ### Release note Partial File Cache blocks can fetch hole ranges concurrently. The new mutable BE setting hole_fill_remote_read_threads_per_be defaults to 64 and controls the shared background remote-read pool independently of the block-worker count. ### Check List (For Author) - Test: Unit Test — 62 existing BE unit tests passed through run-be-ut.sh with ASAN, PCH and -j100, covering AsyncCacheWriteManagerTest, PartialBlockWritebackManagerTest and PartialBlockWritebackOptionsTest. The naming, minimum-thread and token-reuse follow-ups add no new test cases. clang-format v16, check-format, build hygiene and git diff --check passed. - Behavior changed: Yes — remote reads within one partial cache block can execute concurrently, and failed blocks are released after their accepted reads finish. Worker pools start with nonzero minimum thread counts. - Does this need documentation: Yes — the new BE setting and execution/lifecycle contracts are documented in code comments.
bobhan1
force-pushed
the
fix/phase2-no-pch-build
branch
from
September 11, 2026 07:24
d8d795d to
f63b6e1
Compare
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292 Problem Summary: Each hole-fill worker issued the source reads for a cache block sequentially. Keep block workers responsible for planning and writeback. Read a single hole range directly on its block worker and execute multiple disjoint ranges in a separate shared HoleFillRemoteReadPool. Each block worker reuses one concurrent ThreadPoolToken for multi-range blocks, joining accepted reads before either submitting the completed buffer or dropping the block. Destroy the token before signalling worker completion so the remote-read pool can be released safely. Preserve queued reads during shutdown and allow the remote-read pool to resize without interrupting active GETs. Make the shared S3 reader statistics atomic for concurrent foreground and background reads. Initialize both hole-fill and asynchronous cache-write worker pools with their configured worker counts as the minimum, and retain at least one remote-read pool thread. ### Release note Partial File Cache blocks can fetch hole ranges concurrently. The new mutable BE setting hole_fill_remote_read_threads_per_be defaults to 64 and controls the shared pool for multi-range hole reads. Single-range hole reads execute directly on block workers, outside this pool's concurrency limit. ### Check List (For Author) - Test: Unit Test — 62 existing BE unit tests passed through run-be-ut.sh with ASAN, PCH and -j100, covering AsyncCacheWriteManagerTest, PartialBlockWritebackManagerTest and PartialBlockWritebackOptionsTest. The follow-up refinements adapt existing cases without adding new tests. clang-format v16, check-format, build hygiene and git diff --check passed. - Behavior changed: Yes — remote reads within one partial cache block can execute concurrently, and failed blocks are released after their accepted reads finish. Single-range hole reads execute directly on block workers. Worker pools start with nonzero minimum thread counts. - Does this need documentation: Yes — the new BE setting and execution/lifecycle contracts are documented in code comments.
bobhan1
force-pushed
the
fix/phase2-no-pch-build
branch
from
September 11, 2026 07:38
f63b6e1 to
1cde489
Compare
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292 Problem Summary: A partial block can become active immediately after its first fragment arrives, leaving little opportunity to absorb subsequent foreground fragments before planning remote hole reads. Keep tasks mergeable in the existing queue for a configurable minimum interval measured from successful admission. Later fragments preserve that timestamp. Workers skip tasks whose delay has not elapsed and wait on the existing condition variable until the earliest deadline or cache-writer capacity retry. Runtime updates apply to queued tasks and wake workers without affecting active reads. Preserve complete-block direct writeback, queued-task eviction, cache invalidation checks, shutdown, and existing worker and remote-read concurrency. ### Release note Add mutable BE configuration hole_fill_merge_delay_ms, defaulting to 10 ms. When partial-block writeback is enabled, this setting gives queued blocks a short interval to merge foreground fragments before hole filling. Set it to 0 to disable the delay. Changing it affects existing queued tasks; merging another fragment does not restart the interval. ### Check List (For Author) - Test: Unit Test — 77 existing tests passed through run-be-ut.sh with ASAN, PCH, and -j100, covering AsyncCacheWriteManagerTest, PartialBlockWritebackManagerTest, PartialBlockWritebackOptionsTest, HoleFillPlannerTest, RangeCacheWritebackTest, and RangeWritebackDispatcherTest. Extended existing cases for minimum delay, stable admission deadlines, dynamic updates, active-task isolation, eviction, and shutdown. clang-format v16, check-format, build hygiene, and git diff --check passed. No cluster tests or clang-tidy were run. - Behavior changed: Yes — queued partial blocks wait for the configured aggregation interval before becoming active; complete-block writeback and active tasks keep their existing behavior. - Does this need documentation: Yes — the new configuration and queue timing contract are documented in code comments.
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292 Problem Summary: A partial block can become active immediately after its first fragment arrives, leaving little opportunity to absorb subsequent foreground fragments before planning remote hole reads. Keep tasks mergeable in the existing queue for a configurable minimum interval measured from successful admission. Later fragments preserve that timestamp. Workers skip tasks whose delay has not elapsed and wait on the existing condition variable until the earliest deadline or cache-writer capacity retry. Assert that a finite wakeup deadline has been selected before entering the timed wait. Runtime updates apply to queued tasks and wake workers without affecting active reads. Preserve complete-block direct writeback, queued-task eviction, cache invalidation checks, shutdown, and existing worker and remote-read concurrency. ### Release note Add mutable BE configuration hole_fill_merge_delay_ms, defaulting to 10 ms. When partial-block writeback is enabled, this setting gives queued blocks a short interval to merge foreground fragments before hole filling. Set it to 0 to disable the delay. Changing it affects existing queued tasks; merging another fragment does not restart the interval. ### Check List (For Author) - Test: Unit Test — 77 existing tests passed through run-be-ut.sh with ASAN, PCH, and -j100, covering AsyncCacheWriteManagerTest, PartialBlockWritebackManagerTest, PartialBlockWritebackOptionsTest, HoleFillPlannerTest, RangeCacheWritebackTest, and RangeWritebackDispatcherTest. Extended existing cases for minimum delay, stable admission deadlines, dynamic updates, active-task isolation, eviction, and shutdown. clang-format v16, check-format, build hygiene, and git diff --check passed. The subsequent DCHECK/comment-only addition passed formatting and static path review; build and unit tests were not rerun for that addition. No cluster tests or clang-tidy were run. - Behavior changed: Yes — queued partial blocks wait for the configured aggregation interval before becoming active; complete-block writeback and active tasks keep their existing behavior. - Does this need documentation: Yes — the new configuration and queue timing contract are documented in code comments.
bobhan1
force-pushed
the
fix/phase2-no-pch-build
branch
2 times, most recently
from
September 11, 2026 09:01
13266fa to
a28aaea
Compare
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67292, apache#67579 Problem Summary: Column read-ahead uses paired high and low byte watermarks to decide when to extend its window. Pending-byte accounting also changes after Page Cache hits and rejected submissions, even when the scan has not advanced. Replace the watermarks with one compressed-byte window size per column role and a trigger page position. Each window uses its second selected page as the trigger, or its only page for a one-page window. First access and a jump beyond planned coverage establish a window through the same path. Process current-batch pages in scan order so a batch crossing the trigger can append the following window in the same plan. Keep mandatory current-batch pages, cache probing, IO coalescing, asynchronous reads, and writeback on their existing paths. ### Release note Replace the eager/lazy high and low watermark settings with read_ahead_eager_window_bytes (4 MiB) and read_ahead_lazy_window_bytes (128 KiB). Query page read-ahead advances by scan position and retains its existing query and BE buffer budgets. ### Check List (For Author) - Test: Unit Test — 132 targeted ASAN tests, including 24 ColumnReadAhead cases and related scanner, rowid/TopN, range IO, and cache-reader tests; clang-format 16, build hygiene, and git diff --check passed. No cluster or regression workload was run. - Behavior changed: Yes — page-position triggers replace pending-byte watermarks; first access and large jumps share the same window initialization. - Does this need documentation: Yes — configuration and component interface comments are updated in this commit.
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292 Problem Summary: Each hole-fill worker issued the source reads for a cache block sequentially. Keep block workers responsible for planning and writeback. Read a single hole range directly on its block worker and execute multiple disjoint ranges in a separate shared HoleFillRemoteReadPool. Each block worker reuses one concurrent ThreadPoolToken for multi-range blocks, joining accepted reads before either submitting the completed buffer or dropping the block. Destroy the token before signalling worker completion so the remote-read pool can be released safely. Preserve queued reads during shutdown and allow the remote-read pool to resize without interrupting active GETs. Make the shared S3 reader statistics atomic for concurrent foreground and background reads. Initialize both hole-fill and asynchronous cache-write worker pools with their configured worker counts as the minimum, and retain at least one remote-read pool thread. ### Release note Partial File Cache blocks can fetch hole ranges concurrently. The new mutable BE setting hole_fill_remote_read_threads_per_be defaults to 64 and controls the shared pool for multi-range hole reads. Single-range hole reads execute directly on block workers, outside this pool's concurrency limit. ### Check List (For Author) - Test: Unit Test — 62 existing BE unit tests passed through run-be-ut.sh with ASAN, PCH and -j100, covering AsyncCacheWriteManagerTest, PartialBlockWritebackManagerTest and PartialBlockWritebackOptionsTest. The follow-up refinements adapt existing cases without adding new tests. clang-format v16, check-format, build hygiene and git diff --check passed. - Behavior changed: Yes — remote reads within one partial cache block can execute concurrently, and failed blocks are released after their accepted reads finish. Single-range hole reads execute directly on block workers. Worker pools start with nonzero minimum thread counts. - Does this need documentation: Yes — the new BE setting and execution/lifecycle contracts are documented in code comments.
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292 Problem Summary: A partial block can become active immediately after its first fragment arrives, leaving little opportunity to absorb subsequent foreground fragments before planning remote hole reads. Keep tasks mergeable in the existing queue for a configurable minimum interval measured from successful admission. Later fragments preserve that timestamp. Workers skip tasks whose delay has not elapsed and wait on the existing condition variable until the earliest deadline or cache-writer capacity retry. Assert that a finite wakeup deadline has been selected before entering the timed wait. Runtime updates apply to queued tasks and wake workers without affecting active reads. Preserve complete-block direct writeback, queued-task eviction, cache invalidation checks, shutdown, and existing worker and remote-read concurrency. ### Release note Add mutable BE configuration hole_fill_merge_delay_ms, defaulting to 10 ms. When partial-block writeback is enabled, this setting gives queued blocks a short interval to merge foreground fragments before hole filling. Set it to 0 to disable the delay. Changing it affects existing queued tasks; merging another fragment does not restart the interval. ### Check List (For Author) - Test: Unit Test — 77 existing tests passed through run-be-ut.sh with ASAN, PCH, and -j100, covering AsyncCacheWriteManagerTest, PartialBlockWritebackManagerTest, PartialBlockWritebackOptionsTest, HoleFillPlannerTest, RangeCacheWritebackTest, and RangeWritebackDispatcherTest. Extended existing cases for minimum delay, stable admission deadlines, dynamic updates, active-task isolation, eviction, and shutdown. clang-format v16, check-format, build hygiene, and git diff --check passed. The subsequent DCHECK/comment-only addition passed formatting and static path review; build and unit tests were not rerun for that addition. No cluster tests or clang-tidy were run. - Behavior changed: Yes — queued partial blocks wait for the configured aggregation interval before becoming active; complete-block writeback and active tasks keep their existing behavior. - Does this need documentation: Yes — the new configuration and queue timing contract are documented in code comments.
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 11, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579 Problem Summary: ReadAheadPlanTime starts in SegmentReadAhead::apply_plans after physical-column preparation. It therefore omits initialization, request validation, per-row page lookup and window maintenance, including work that produces an empty plan. Add elapsed-time counters for physical-column preparation, its initialization and validation stage, window discard, current-batch planning and nested window extension. Collect the inner durations per column call and report them before empty plans are dropped. Keep the existing range-planning timer unchanged for comparison with earlier profiles. Timers are scoped to column calls and window operations rather than individual row lookups. No read-ahead, IO coalescing, cache writeback or hole-fill decisions change. ### Release note Add ReadAheadColumnPlanTime and nested initialization, window-discard, current-batch and window-extension elapsed timers under PageReadAhead in query profiles. Parent timers include their children; the existing ReadAheadPlanTime retains its range-planning and submission scope. No configuration or bvar metrics change. ### Check List (For Author) - Test: Unit Test — 50 tests from 5 suites passed through run-be-ut.sh with ASAN and -j100, covering ReadAheadMetricsTest, ColumnReadAheadTest, SegmentReadAheadTest, SegmentIteratorReadAheadTest and the relevant ColumnReaderTest cases. Extended existing tests for timer hierarchy, delta reporting, reused plan outputs and empty-plan accounting. clang-format v16, check-format, build hygiene and git diff --check passed. No cluster tests or clang-tidy were run. - Behavior changed: No — timing collection and Profile output only. - Does this need documentation: No — timer boundaries and nesting are documented inline and represented in the Profile hierarchy.
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 14, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579 Problem Summary: The SSB Flat SF1000 planning-delay0 comparison attributed 5,242.6 seconds of accumulated column-planning elapsed time to the current-rowid loop excluding window extension, or 91.6% of column planning. Each scanner batch repeatedly validates rowids, locates the same physical pages for every row, and walks the existing window for every column. Window extension itself accounted for only 6.7 seconds. These are accumulated elapsed counters, not query wall time or thread CPU time. Build each scan column's candidate-page sequence once after segment pruning and submit the initial eager/lazy windows together. Advance subsequent windows directly from the ordinal page index at the data-page read entry, including Page Cache hits and sparse seeks. Retain high-page predictions while reverse scanners read an ascending batch. Keep point reads and dependent nested-column reads on the exact-rowid path, grouping rows by page. Reuse already buffered physical pages when column windows advance independently. Keep IO coalescing, cache-block completion, hole filling and all existing configuration defaults unchanged. Report page advancement separately through ReadAheadPageAdvanceTime; initialization includes candidate selection, while window-discard and window-extension counters remain sub-timers. This change has not yet been benchmarked on the SSB cluster. ### Release note Reduce query read-ahead planning work by preparing candidate pages once and advancing windows on data-page reads. Add ReadAheadPageAdvanceTime to the query Profile; ReadAheadWindowExtendTime is now a shared sub-timer under ReadAheadColumnPlanTime rather than a child of current-batch planning. ### Check List (For Author) - Test: Unit Test — 190 tests from 10 suites passed through run-be-ut.sh with ASAN and -j100, covering complete ColumnReaderTest, column/segment read-ahead, metrics, IO coalescing, range planning, asynchronous read scheduling, range writeback and hole filling. Added dense-scan, sparse/reverse progression, Page Cache, admission fallback and independent-window buffer-reuse coverage. clang-format v16, check-format, build hygiene and git diff --check passed. No cluster tests or clang-tidy were run. - Behavior changed: Yes — common scans plan once after pruning and advance on physical page reads instead of planning every rowid batch. Query results and synchronous fallback semantics are preserved. - Does this need documentation: Yes — the scan planning flow and Profile timer scope need a follow-up synchronization in the design repository.
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 14, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292, apache#67612 Problem Summary: Plan reset and timing aggregation, asynchronous read result construction and terminal-state checks, and partial-block request validation were implemented outside the objects whose state they describe. Move these operations to ColumnReadAheadPlan, FileRangeReadSubmitResult, FileRangeRead, and PartialBlockWritebackRequest. Keep partial-block buffer creation and remote reads on Task, while the manager retains queue admission, worker scheduling, and lifecycle accounting. Preserve vector capacity during plan reset, per-call statistics, lock boundaries, and the join of accepted remote reads before task cleanup. ### Release note None ### Check List (For Author) - Test: Unit Test — 198 tests from 11 suites passed through run-be-ut.sh with ASAN and -j100. Added plan-reset capacity/ownership, empty-plan timing aggregation, and rejected-result coverage. Existing read-ahead, scheduler, cache-writeback and hole-fill tests passed, including concurrent reads, failed submission/read cleanup, and shutdown. clang-format v16, check-format, build hygiene, and git diff --check passed. No cluster tests or clang-tidy were run. - Behavior changed: No — move object-local operations without changing read-ahead, IO, or writeback mechanisms. - Does this need documentation: No — internal responsibility cleanup; configuration and metric meanings are unchanged.
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 14, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67292, apache#67579, apache#67612 Problem Summary: Build each scan column's eligible physical page indexes once and advance compressed-byte windows from the page index resolved by the column reader. A window's second selected page triggers the following window. Retain predictions for ascending batches inside reverse scans; group exact-row requests by distinct pages. ColumnReadAheadPlan owns reset and per-call timing aggregation, retaining vector capacity. Window accounting stays with ColumnReadAhead. ### Release note Internal page-driven column windows for query read-ahead. ### Check List (For Author) - Test: Unit Test — component coverage is included with this change. The integrated source snapshot d5c427f passed 198 ASAN tests from 11 suites via run-be-ut.sh -j100. This individual intermediate commit was not built separately; target-branch validation is recorded on the PR. Repository clang-format v16, build hygiene and diff checks passed on the integrated tree. No cluster or clang-tidy run. - Behavior changed: Yes — introduces the opt-in component; this synchronization preserves existing defaults and IO/writeback semantics. - Does this need documentation: Yes — component scope and current behavior are tracked in apache#67611 and the related PR descriptions.
bobhan1
added a commit
to bobhan1/doris
that referenced
this pull request
Sep 14, 2026
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67292, apache#67579, apache#67612 Problem Summary: Coordinate all resolvable column plans, probe Page Cache, plan and submit ranges, and register exact page slices. Reuse a physical page's existing buffer when independent column windows overlap it. Keep synchronous fallback and per-page ownership accounting; expose planner and consumption statistics. ### Release note Internal segment-level coordination for concurrent data-page reads. ### Check List (For Author) - Test: Unit Test — component coverage is included with this change. The integrated source snapshot d5c427f passed 198 ASAN tests from 11 suites via run-be-ut.sh -j100. This individual intermediate commit was not built separately; target-branch validation is recorded on the PR. Repository clang-format v16, build hygiene and diff checks passed on the integrated tree. No cluster or clang-tidy run. - Behavior changed: Yes — introduces the opt-in component; this synchronization preserves existing defaults and IO/writeback semantics. - Does this need documentation: Yes — component scope and current behavior are tracked in apache#67611 and the related PR descriptions.
Issue Number: None
Related PR: None
Problem Summary: Background hole filling produces an owned complete cache-block buffer. Add a handoff to the asynchronous cache writer that preserves that buffer without another payload copy, while retaining epoch fencing, inflight deduplication, and queue admission. Also expose a point-in-time spare-capacity check so low-priority hole-fill reads start only when the write queue can accept a block without evicting foreground work.
None
- Test: Unit Test
- ./run-be-ut.sh -j100 --run '--filter=AsyncCacheWriteManagerTest.SubmitOwnedBlockPreservesBufferIdentity:AsyncCacheWriteManagerTest.ReportsOnlyUnusedPendingCapacity:AsyncCacheWriteManagerTest.SubmitBlockCopiesDeduplicatesAndRollsBackBackpressure:AsyncCacheWriteManagerTest.SubmitBlockRejectsStaleEpochAndAllocationFailure'
- Behavior changed: No
- Does this need documentation: No
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: A background hole-fill task can observe spare write capacity before its remote read and find the queue full at final handoff. Add a spare-capacity-only admission mode so this low-priority result is rejected instead of evicting an already queued foreground cache write. Normal submissions retain the existing oldest-queued replacement behavior.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- ./run-be-ut.sh -j100 --run '--filter=AsyncCacheWriteManagerTest.ReportsOnlyUnusedPendingCapacity:AsyncCacheWriteManagerTest.DropOldestReplacesOnlyOldestQueuedTaskAndKeepsInflightReaderAlive:AsyncCacheWriteManagerTest.SubmitOwnedBlockPreservesBufferIdentity'
- Behavior changed: No
- Does this need documentation: No
### What problem does this PR solve? Issue Number: None Related PR: apache#67292 Problem Summary: Query read-ahead can leave partially covered File Cache blocks that need bounded asynchronous completion before entering the cache writer. Add a BE-level manager that merges queued fragments, deduplicates active blocks, evicts the oldest queued block under pressure, fills block-local holes through dedicated remote-read workers, and hands completed buffers to the asynchronous cache writer without another copy. The worker set can grow or shrink at runtime; shrinking waits only for retiring workers to finish their active remote reads. Same-block submissions are detected before allocating another block buffer, and their fragment copies use a task-local lock so unrelated submissions do not hold the BE-wide queue lock behind a memcpy. ### Release note None ### Check List (For Author) - Test: Unit Test - ./run-be-ut.sh --run '--filter=PartialBlockWritebackManagerTest.*:PartialBlockWritebackOptionsTest.*' -j100 - Behavior changed: No; this commit adds an unwired internal component. - Does this need documentation: No
### What problem does this PR solve? Issue Number: None Related PR: apache#67292 Problem Summary: Background partial-block writeback needs production defaults and a BE-owned lifetime bounded by the File Cache lifecycle. Add validated hole-fill configuration, create the manager after File Cache initialization, and stop it before File Cache teardown. Make hole_fill_workers_per_be mutable so a runtime configuration update resizes the dedicated remote-read workers without rebuilding the manager; allow 1 through 128 workers and keep the default at 32. ### Release note Support runtime resizing of BE-wide hole-fill workers through hole_fill_workers_per_be. ### Check List (For Author) - Test: Unit Test - ./run-be-ut.sh --run '--filter=PartialBlockWritebackOptionsTest.*' -j100 - Behavior changed: Yes; runtime updates to hole_fill_workers_per_be resize the active hole-fill worker set. - Does this need documentation: No
### What problem does this PR solve? Issue Number: None Related PR: apache#67292 Problem Summary: Query read-ahead produces consumed variable-length buffers that can span complete and partial File Cache blocks. Add RangeCacheWriteback to split each consumed range, submit complete blocks directly to the existing asynchronous write path, and enqueue partial fragments for background hole filling under the same invalidation epoch captured before the remote read starts. ### Release note None ### Check List (For Author) - Test: Unit Test - ./run-be-ut.sh -j100 --run --filter=RangeCacheWritebackTest.* - Behavior changed: No; this commit adds an internal writeback router that is not connected to SegmentIterator yet. - Does this need documentation: No
Issue Number: None Related PR: apache#67292 Problem Summary: Query read-ahead must publish only buffers that were actually consumed, preserve caller-visible IO statistics, and honor the original cache-write policy. Centralize query read-ahead construction so SegmentIterator and exact row-id readers share the same byte windows, range planner, page-cache probe, scheduler, and writeback setup. Each asynchronous submission owns its statistics and File Cache epoch; the first consumed page publishes the completed range once, while no-write and remote-only requests skip cache writeback. Enable consumed query read-ahead ranges to populate File Cache when asynchronous cache writeback is enabled. - Test: Unit Test - ./run-be-ut.sh --run --filter=AsyncCachedRemoteFileReaderTest.*:SegmentReadAheadTest.*:SegmentIteratorReadAheadTest.*:ColumnReaderTest.*ReadAhead* -j100 - Behavior changed: Yes; consumed successful query read-ahead ranges now contribute IO statistics and enter cache writeback when policy permits. - Does this need documentation: Yes; covered by the read-ahead and hole-fill design document.
### What problem does this PR solve? Issue Number: None Related PR: apache#67292 Problem Summary: Short-circuit point queries read row-store data or missing column-store values through exact row IDs. Cold data pages previously followed the synchronous reader one page or column at a time. Create one request-scoped range-read context, initialize the row-store or all missing-column iterators against the shared read-ahead reader, submit the required data pages before decoding, and retain the existing row-cache and result-padding behavior. Read-ahead initialization or IO failure falls back to the original reader. Include the reusable row-id page preparation helpers, SegmentReadAhead exact-row entry point, and its tests in this point-query component. TopN second-stage integration remains unchanged from master and is outside this PR series. ### Release note Use query range read-ahead for point-query data pages when enable_query_read_ahead is enabled. ### Check List (For Author) - Test: Unit Test - ./run-be-ut.sh --run --filter=SegmentReadAheadTest.*:RowCacheTest.*:LookupConnectionCacheTest.* -j100 - Behavior changed: Yes; point-query data-page reads use the range read-ahead path when the feature is enabled. - Does this need documentation: Yes; covered by the read-ahead and hole-fill design document. # Conflicts: # be/src/service/point_query_executor.cpp # be/src/storage/tablet/base_tablet.h # Please enter the commit message for your changes. Lines starting # with '#' will be kept; you may remove them yourself if you want to. # An empty message aborts the commit. # # interactive rebase in progress; onto 3dc7341 # Last commands done (19 commands done): # pick b9396d9 # [feature](be) Connect read-ahead range writeback # pick 7ccfb97 # [feature](be) Extend range read-ahead to point queries # Next commands to do (21 remaining commands): # pick 9a6bcd5 # [fix](be) Scope filesystem error-code imports to their functions # pick 06c0770 # [improvement](be) Add page read-ahead and hole-fill observability # You are currently rebasing branch 'fix/phase2-no-pch-build' on '3dc73418a07'. # # Changes to be committed: # modified: be/src/service/point_query_executor.cpp # new file: be/src/storage/segment/rowid_read_ahead.cpp # new file: be/src/storage/segment/rowid_read_ahead.h # modified: be/src/storage/segment/segment_read_ahead.cpp # modified: be/src/storage/segment/segment_read_ahead.h # modified: be/src/storage/tablet/base_tablet.cpp # modified: be/src/storage/tablet/base_tablet.h # modified: be/test/storage/segment/segment_read_ahead_test.cpp # # Untracked files: # .codex #
### What problem does this PR solve? Issue Number: None Related PR: apache#67292 Problem Summary: With ENABLE_PCH=OFF and unity builds enabled, the IO unity translation unit includes err_utils.cpp before file_meta_cache.cpp. The namespace-scope ErrorCode using-directive then makes LIMIT_REACH ambiguous when the latter includes the generated protobuf enum. Move the using-directive into the two localfs_error overloads that need it, preserving the existing error-code mappings while preventing it from affecting sibling source files. ### Release note None ### Check List (For Author) - Test: Manual test. Reproduced the original ENABLE_PCH=OFF compile failure with ./build.sh --be -j100; build-support/clang-format.sh, build-support/check-format.sh, build-support/check-build-hygiene.sh, and git diff --check passed. Post-fix BE compilation is tracked in the compilation-only Draft PR. - Behavior changed: No - Does this need documentation: No
### What problem does this PR solve? Issue Number: apache#67611 (tracking; keep open) Related PR: apache#67292 Problem Summary: Page read-ahead, IO coalescing and partial-block writeback lack query-level attribution and BE-wide capacity and IO signals. Add named RuntimeProfile counters for planning, compressed candidate and cache-hit bytes, submission outcomes, source IO, page consumption and foreground writeback. Distinguish Query/BE budget rejection and executor rejection from failures of accepted reads, and account queued cancellations only for accepted tasks. Add successful-source and BE cumulative IO/consumption bytes so asynchronous tail work remains observable after the final Profile snapshot without delaying query completion. Record synchronous data-page fallback attempts, requested bytes and time at PageIO, including decode retries while excluding Page Cache hits and index/dictionary reads. Add aggregate hole-fill failure/read-time metrics and keep detailed failure causes in logs. Use COUNTER_UPDATE and scoped timers, with statistics owned independently of the reporting profile so outstanding reads can finish safely. Append point-query statistics to the existing sampled diagnostic log. ### Release note Expose PageReadAhead query profile counters and BE-wide metrics for read-ahead IO, data-page fallback, buffer capacity and background cache-block hole filling. ### Check List (For Author) - Test: Unit Test - 131 targeted ASAN tests from 18 suites passed via run-be-ut.sh -j100 with PCH enabled, as recorded in the original accounting commit. - clang-format 16, check-format, build hygiene and git diff --check passed before squashing. - The historical results above predate removal of TopN-specific collection; current branch validation is recorded in apache#67611. - Behavior changed: Yes (observability only; read scheduling, cancellation, fallback and writeback policies are unchanged) - Does this need documentation: Yes (doris-io/phase2_page_prefetch_and_io_coalescing/observability.md)
### What problem does this PR solve? Issue Number: apache#67611 (tracking; keep open) Related PR: apache#67292 Problem Summary: Consuming a read-ahead range served from File Cache could enqueue partial-block writeback and reread the rest of an already cached block from remote storage. Downloaded blocks were only detected later by the cache writer, after the redundant remote reads. Skip range writeback for local/inflight hits and combine inflight-buffer lookup with the existing cache probe to deduplicate blocks already cached or being filled. Perform the combined check before partial-block admission and once when a worker starts the task, outside the hole-fill queue mutex. Execute the planned hole reads without per-GET probes. Preserve existing queues, concurrency, active-task fragment handling, and final cache-writer deduplication and invalidation checks. ### Release note Avoid redundant remote hole-fill reads for blocks already cached or being filled. ### Check List (For Author) - Test: Unit Test - 105 targeted ASAN tests from 9 suites passed through run-be-ut.sh -j100 with PCH enabled on the same final source tree before squashing, including cached/downloading blocks, EOF, empty cells, queued and active completion races, inflight fast-path locking, mixed cached/uncached ranges, and local-hit read-ahead consumption. - clang-format 16, check-format, build hygiene, and git diff --check passed before squashing. - No cluster or regression workload was run. Clang-tidy was not run as requested. - Behavior changed: Yes (skip redundant best-effort writeback and hole-fill IO; query results and scheduling limits are unchanged) - Does this need documentation: No (cache-reuse fix with no configuration changes)
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67292, apache#67579 Problem Summary: Column read-ahead uses paired high and low byte watermarks to decide when to extend its window. Pending-byte accounting also changes after Page Cache hits and rejected submissions, even when the scan has not advanced. Replace the watermarks with one compressed-byte window size per column role and a trigger page position. Each window uses its second selected page as the trigger, or its only page for a one-page window. First access and a jump beyond planned coverage establish a window through the same path. Process current-batch pages in scan order so a batch crossing the trigger can append the following window in the same plan. Keep mandatory current-batch pages, cache probing, IO coalescing, asynchronous reads, and writeback on their existing paths. ### Release note Replace the eager/lazy high and low watermark settings with read_ahead_eager_window_bytes (4 MiB) and read_ahead_lazy_window_bytes (128 KiB). Query page read-ahead advances by scan position and retains its existing query and BE buffer budgets. ### Check List (For Author) - Test: Unit Test — 132 targeted ASAN tests, including 24 ColumnReadAhead cases and related scanner, rowid/TopN, range IO, and cache-reader tests; clang-format 16, build hygiene, and git diff --check passed. No cluster or regression workload was run. - Behavior changed: Yes — page-position triggers replace pending-byte watermarks; first access and large jumps share the same window initialization. - Does this need documentation: Yes — configuration and component interface comments are updated in this commit.
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292 Problem Summary: Each hole-fill worker issued the source reads for a cache block sequentially. Keep block workers responsible for planning and writeback. Read a single hole range directly on its block worker and execute multiple disjoint ranges in a separate shared HoleFillRemoteReadPool. Each block worker reuses one concurrent ThreadPoolToken for multi-range blocks, joining accepted reads before either submitting the completed buffer or dropping the block. Destroy the token before signalling worker completion so the remote-read pool can be released safely. Preserve queued reads during shutdown and allow the remote-read pool to resize without interrupting active GETs. Make the shared S3 reader statistics atomic for concurrent foreground and background reads. Initialize both hole-fill and asynchronous cache-write worker pools with their configured worker counts as the minimum, and retain at least one remote-read pool thread. ### Release note Partial File Cache blocks can fetch hole ranges concurrently. The new mutable BE setting hole_fill_remote_read_threads_per_be defaults to 64 and controls the shared pool for multi-range hole reads. Single-range hole reads execute directly on block workers, outside this pool's concurrency limit. ### Check List (For Author) - Test: Unit Test — 62 existing BE unit tests passed through run-be-ut.sh with ASAN, PCH and -j100, covering AsyncCacheWriteManagerTest, PartialBlockWritebackManagerTest and PartialBlockWritebackOptionsTest. The follow-up refinements adapt existing cases without adding new tests. clang-format v16, check-format, build hygiene and git diff --check passed. - Behavior changed: Yes — remote reads within one partial cache block can execute concurrently, and failed blocks are released after their accepted reads finish. Single-range hole reads execute directly on block workers. Worker pools start with nonzero minimum thread counts. - Does this need documentation: Yes — the new BE setting and execution/lifecycle contracts are documented in code comments.
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292 Problem Summary: A partial block can become active immediately after its first fragment arrives, leaving little opportunity to absorb subsequent foreground fragments before planning remote hole reads. Keep tasks mergeable in the existing queue for a configurable minimum interval measured from successful admission. Later fragments preserve that timestamp. Workers skip tasks whose delay has not elapsed and wait on the existing condition variable until the earliest deadline or cache-writer capacity retry. Assert that a finite wakeup deadline has been selected before entering the timed wait. Runtime updates apply to queued tasks and wake workers without affecting active reads. Preserve complete-block direct writeback, queued-task eviction, cache invalidation checks, shutdown, and existing worker and remote-read concurrency. ### Release note Add mutable BE configuration hole_fill_merge_delay_ms, defaulting to 10 ms. When partial-block writeback is enabled, this setting gives queued blocks a short interval to merge foreground fragments before hole filling. Set it to 0 to disable the delay. Changing it affects existing queued tasks; merging another fragment does not restart the interval. ### Check List (For Author) - Test: Unit Test — 77 existing tests passed through run-be-ut.sh with ASAN, PCH, and -j100, covering AsyncCacheWriteManagerTest, PartialBlockWritebackManagerTest, PartialBlockWritebackOptionsTest, HoleFillPlannerTest, RangeCacheWritebackTest, and RangeWritebackDispatcherTest. Extended existing cases for minimum delay, stable admission deadlines, dynamic updates, active-task isolation, eviction, and shutdown. clang-format v16, check-format, build hygiene, and git diff --check passed. The subsequent DCHECK/comment-only addition passed formatting and static path review; build and unit tests were not rerun for that addition. No cluster tests or clang-tidy were run. - Behavior changed: Yes — queued partial blocks wait for the configured aggregation interval before becoming active; complete-block writeback and active tasks keep their existing behavior. - Does this need documentation: Yes — the new configuration and queue timing contract are documented in code comments.
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579 Problem Summary: ReadAheadPlanTime starts in SegmentReadAhead::apply_plans after physical-column preparation. It therefore omits initialization, request validation, per-row page lookup and window maintenance, including work that produces an empty plan. Add elapsed-time counters for physical-column preparation, its initialization and validation stage, window discard, current-batch planning and nested window extension. Collect the inner durations per column call and report them before empty plans are dropped. Keep the existing range-planning timer unchanged for comparison with earlier profiles. Timers are scoped to column calls and window operations rather than individual row lookups. No read-ahead, IO coalescing, cache writeback or hole-fill decisions change. ### Release note Add ReadAheadColumnPlanTime and nested initialization, window-discard, current-batch and window-extension elapsed timers under PageReadAhead in query profiles. Parent timers include their children; the existing ReadAheadPlanTime retains its range-planning and submission scope. No configuration or bvar metrics change. ### Check List (For Author) - Test: Unit Test — 50 tests from 5 suites passed through run-be-ut.sh with ASAN and -j100, covering ReadAheadMetricsTest, ColumnReadAheadTest, SegmentReadAheadTest, SegmentIteratorReadAheadTest and the relevant ColumnReaderTest cases. Extended existing tests for timer hierarchy, delta reporting, reused plan outputs and empty-plan accounting. clang-format v16, check-format, build hygiene and git diff --check passed. No cluster tests or clang-tidy were run. - Behavior changed: No — timing collection and Profile output only. - Does this need documentation: No — timer boundaries and nesting are documented inline and represented in the Profile hierarchy.
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579 Problem Summary: The SSB Flat SF1000 planning-delay0 comparison attributed 5,242.6 seconds of accumulated column-planning elapsed time to the current-rowid loop excluding window extension, or 91.6% of column planning. Each scanner batch repeatedly validates rowids, locates the same physical pages for every row, and walks the existing window for every column. Window extension itself accounted for only 6.7 seconds. These are accumulated elapsed counters, not query wall time or thread CPU time. Build each scan column's candidate-page sequence once after segment pruning and submit the initial eager/lazy windows together. Advance subsequent windows directly from the ordinal page index at the data-page read entry, including Page Cache hits and sparse seeks. Retain high-page predictions while reverse scanners read an ascending batch. Keep point reads and dependent nested-column reads on the exact-rowid path, grouping rows by page. Reuse already buffered physical pages when column windows advance independently. Keep IO coalescing, cache-block completion, hole filling and all existing configuration defaults unchanged. Report page advancement separately through ReadAheadPageAdvanceTime; initialization includes candidate selection, while window-discard and window-extension counters remain sub-timers. This change has not yet been benchmarked on the SSB cluster. ### Release note Reduce query read-ahead planning work by preparing candidate pages once and advancing windows on data-page reads. Add ReadAheadPageAdvanceTime to the query Profile; ReadAheadWindowExtendTime is now a shared sub-timer under ReadAheadColumnPlanTime rather than a child of current-batch planning. ### Check List (For Author) - Test: Unit Test — 190 tests from 10 suites passed through run-be-ut.sh with ASAN and -j100, covering complete ColumnReaderTest, column/segment read-ahead, metrics, IO coalescing, range planning, asynchronous read scheduling, range writeback and hole filling. Added dense-scan, sparse/reverse progression, Page Cache, admission fallback and independent-window buffer-reuse coverage. clang-format v16, check-format, build hygiene and git diff --check passed. No cluster tests or clang-tidy were run. - Behavior changed: Yes — common scans plan once after pruning and advance on physical page reads instead of planning every rowid batch. Query results and synchronous fallback semantics are preserved. - Does this need documentation: Yes — the scan planning flow and Profile timer scope need a follow-up synchronization in the design repository.
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67579, apache#67292, apache#67612 Problem Summary: Plan reset and timing aggregation, asynchronous read result construction and terminal-state checks, and partial-block request validation were implemented outside the objects whose state they describe. Move these operations to ColumnReadAheadPlan, FileRangeReadSubmitResult, FileRangeRead, and PartialBlockWritebackRequest. Keep partial-block buffer creation and remote reads on Task, while the manager retains queue admission, worker scheduling, and lifecycle accounting. Preserve vector capacity during plan reset, per-call statistics, lock boundaries, and the join of accepted remote reads before task cleanup. ### Release note None ### Check List (For Author) - Test: Unit Test — 198 tests from 11 suites passed through run-be-ut.sh with ASAN and -j100. Added plan-reset capacity/ownership, empty-plan timing aggregation, and rejected-result coverage. Existing read-ahead, scheduler, cache-writeback and hole-fill tests passed, including concurrent reads, failed submission/read cleanup, and shutdown. clang-format v16, check-format, build hygiene, and git diff --check passed. No cluster tests or clang-tidy were run. - Behavior changed: No — move object-local operations without changing read-ahead, IO, or writeback mechanisms. - Does this need documentation: No — internal responsibility cleanup; configuration and metric meanings are unchanged.
### What problem does this PR solve? Issue Number: apache#67611 (tracking) Related PR: apache#67292, apache#67579 Problem Summary: Adjust the opt-in read-ahead and hole-fill defaults together: use a 1 MiB lazy-column window, a 1.0 background hole-fill amplification limit, a 1 GiB BE-wide pending partial-block budget, and zero deliberate fragment-aggregation delay. Align PartialBlockWritebackOptions and the existing production-default test with the registered settings. Keep planning, queueing, concurrency, validators, and configuration mutability unchanged. ### Release note For query read-ahead, which remains disabled by default, set read_ahead_lazy_window_bytes to 1048576 (1 MiB), hole_fill_max_read_amplification_ratio to 1.0, hole_fill_max_pending_bytes_per_be to 1073741824 (1 GiB), and hole_fill_merge_delay_ms to 0. ### Check List (For Author) - Test: Unit Test - 75 existing tests from 5 focused ASAN/PCH read-ahead and hole-fill suites passed via run-be-ut.sh -j100; no new tests. - clang-format 16 formatting/check, BE build hygiene, and git diff --check. - Behavior changed: Yes; adjust four defaults for opt-in read-ahead and hole filling without changing the mechanisms. - Does this need documentation: Yes; synchronize configuration values and commit mappings in apache#67292, apache#67579, and apache#67611.
### What problem does this PR solve? Issue Number: apache#67611 (tracking; this diagnostic does not complete the feature) Related PR: apache#67579 Problem Summary: Aggregate S3 and hole-fill byte counters cannot establish whether foreground reads and background hole filling fetched the same object intervals. Add disabled-by-default, bounded JSON tracing at actual S3 GETs and the read-ahead consumption / partial-block lifecycle boundaries. Carry stable range/task identities and source labels through IOContext, and take read-only cache/inflight coverage snapshots before whole-range misses. Provide an offline interval-sweep analyzer that counts extra successful transfers once, distinguishes within-source and cross-source overlap, links representative GETs to late fragments, and checks capture continuity and S3 byte reconciliation. Keep IO planning, merging, admission, concurrency and writeback decisions unchanged. No cluster or benchmark is run by this change. ### Release note Add mutable BE settings enable_read_io_trace=false and read_io_trace_max_events=1000000 for temporary S3/read-ahead/hole-fill diagnostics. The diagnostic logs can affect timing when enabled and are intended for isolated captures, not performance comparisons. See tools/read_io_trace/README.md for capture and analysis instructions. ### Check List (For Author) - Test: Unit Test - 73 targeted ASAN/PCH BE tests from 6 suites passed through run-be-ut.sh -j100. - 9 offline analyzer tests passed, including interval-union arithmetic, triple reads, source/query isolation, temporal examples, missing logs and reconciliation failure. - Repository clang-format 16, formatting check, BE header hygiene and git diff --check passed. - No cluster, external object storage workload, performance benchmark or clang-tidy run. - Behavior changed: Yes; opt-in diagnostic logging and coverage probes only. Read and writeback mechanisms are unchanged. - Does this need documentation: Yes; tools/read_io_trace/README.md is included in this commit.
### What problem does this PR solve? Issue Number: None Related PR: apache#67579 Problem Summary: Read-overlap diagnostics currently put every event in be.INFO and stop recording after a process-wide event limit, mixing diagnostic records with operational logs and truncating complete workload captures. Append serialized records to an owned memory batch and let a lazy background writer flush it to dedicated JSONL files by byte threshold or interval. Remove the event limit without adding queue-capacity limits or backpressure. Drain submitted records during normal and fast BE shutdown. Add flush checkpoints and pending-byte visibility so the offline analyzer can check complete captures and reported file errors. ### Release note Opt-in read IO diagnostics now use read_io_trace_dir (default: the BE log directory's read_io_trace subdirectory). read_io_trace_max_events is removed. New startup settings read_io_trace_flush_bytes and read_io_trace_flush_interval_ms default to 1 MiB and 1000 ms. enable_read_io_trace remains dynamically switchable and disabled by default. ### Check List (For Author) - Test: Unit Test - Incremental ASAN/PCH run-be-ut.sh -j100: 81 tests from 8 suites passed, including dedicated-file batching, interval flushing, concurrent producers, shutdown and actual file-error recovery. - Offline analyzer: 12 Python unit tests passed. - clang-format 16, BE header hygiene and git diff --check passed. No cluster workload or clang-tidy run. - Behavior changed: Yes; enabled diagnostic events go to batched standalone JSONL files instead of be.INFO, without a cumulative event limit. Read-ahead and hole-fill mechanisms are unchanged. - Does this need documentation: Yes; tools/read_io_trace/README.md is updated in this commit.
### What problem does this PR solve? Issue Number: None Related PR: apache#67579 Problem Summary: A longer hole-fill fragment aggregation window reduced duplicate foreground/background reads in the q4.1 experiment, but foreground writeback submission time increased. The existing Profile and interval trace cannot distinguish shared queue mutex contention from cache probes, buffer allocation, fragment merging, copies or diagnostic recording. Add complete/partial dispatch Profile timers and opt-in typed timing records for foreground submissions and locked worker scans. Serialize scan summaries outside the queue mutex and keep condition-variable sleeps outside scan timings. Extend the offline analyzer with stage totals, percentiles, queue-depth groups and slow examples, preserving process scope for scans shared by queries. Read, merge, admission and persistence decisions remain unchanged. ### Release note The existing enable_read_io_trace switch now also records writeback submission stages and hole-fill queue scans in the dedicated JSONL trace. Two lightweight Profile timers separate complete-block and partial-block submission even with tracing disabled. The doris_read_io_trace_record_time_ns counter measures diagnostic serialization and memory-append overhead. No new settings or default changes. ### Check List (For Author) - Test: Unit Test - Incremental ASAN/PCH run-be-ut.sh -j100: 87 tests from 9 suites passed. - Python offline analyzer: 15 tests passed. - Repository clang-format 16, check-format, BE header hygiene and git diff --check passed. - No cluster, external-object-store benchmark or clang-tidy run. - Behavior changed: Yes, additional diagnostic output and Profile counters; IO and queue mechanisms unchanged. - Does this need documentation: Yes, capture boundaries, timing interpretation and comparison workflow are documented in tools/read_io_trace/README.md in this commit.
### What problem does this PR solve? Issue Number: None Related PR: apache#67579 Problem Summary: With a 2000 ms hole-fill merge window, the archived q4.1 trace measured 389.63 s of cumulative foreground queue-mutex wait and 78.40 s of locked worker scans; 99.984% of examined tasks had not reached their merge deadline. Workers repeatedly probed every delayed task under the same mutex needed by foreground fragment submission. Keep the existing list ordered by admission time, moving stale-epoch replacements to the tail with a fresh merge window. Check the deadline before epoch/inflight probes and stop at the first delayed entry. A fully delayed queue now requires one examined entry per scan, verified with 1024 queued tasks. Continue past capacity-blocked due tasks so another cache writer can make progress. Preserve runtime delay updates, worker concurrency, fragment merging and hole-read planning. Workload latency gains have not been measured after this change. ### Release note Reduce foreground writeback contention when hole-fill fragment aggregation uses a delay, without adding configuration or changing its defaults. ### Check List (For Author) - Test: Unit Test - Incremental ASAN/PCH run-be-ut.sh -j100: 107 tests from 9 suites passed. - Coverage includes a 1024-task delayed queue, stale-epoch replacement order and renewed delay, runtime delay changes, cross-writer capacity bypass, active merges, eviction and shutdown. - Python trace analyzer: 15 tests passed. - Repository clang-format 16, check-format, BE header hygiene and git diff --check passed. - No cluster, external-object-store benchmark or clang-tidy run. - Behavior changed: Yes, delayed tasks are no longer repeatedly scanned or probed; stale replacements join the queue tail. Due-task checks, memory bounds, shutdown cleanup and concurrency limits remain in place. - Does this need documentation: Yes, visited-entry semantics are documented in tools/read_io_trace/README.md in this commit.
### What problem does this PR solve? Issue Number: None Related PR: apache#67579 Problem Summary: The archived q4.1 deadline-scan run removed queue-lock contention but still recorded 1,053,030 scans for about 34,479 activated blocks with 32 workers; 1,018,551 scans encountered an unexpired head. Multiple workers independently waited for the same deadline. Introduce one HoleFillDispatch thread for deadline and capacity decisions and keep block workers dedicated to execution with reusable remote-read tokens. Bound dispatched plus executing tasks by worker count, transfer queue nodes with splice, and suppress notifications that cannot advance the current wait. Preserve queued fragment merging, dynamic resizing and cross-writer capacity bypass, and cancel undelivered tasks at shutdown. No post-change workload speedup is claimed. Remove the offline tools/read_io_trace analyzer and its tests from this feature; retain the BE-side trace capture and diagnostic settings. ### Release note Use a single hole-fill dispatcher to avoid redundant deadline wakeups while preserving configured read concurrency and queued fragment aggregation. No new configuration or default changes. ### Check List (For Author) - Test: Unit Test - Pre-change incremental ASAN/PCH run-be-ut.sh -j100: 110 tests from 9 suites passed. - Coverage includes 32 workers with 1024 delayed tasks, full 32-way execution with overflow merging, idle and active resizing, handoff cancellation, epoch replacement, cache-writer capacity, eviction and remote reads. - Repository clang-format 16, check-format, BE header hygiene and git diff --check passed. - No cluster, external-object-store benchmark or clang-tidy run. - Behavior changed: Yes, one dispatcher owns timing and hands off at most the configured worker slots; queued and undelivered tasks are cancelled at shutdown. - Does this need documentation: Yes, dispatcher architecture and BE tracing are described in the active PRs.
### What problem does this PR solve? Issue Number: Part of apache#67611 Related PR: apache#67292, apache#67579 Problem Summary: Partial-block writeback should collect foreground fragments for 2000 ms by default and allow 4 GiB of pending buffers per BE. Default foreground range planning should introduce no gap or partial-block completion bytes, so set its amplification ratio and block-fill coverage to 1.0. Keep the existing 1 MiB lazy window, 4 MiB eager window, 1.0 hole-fill amplification ratio, 32 hole-fill workers and 64 remote read threads. Align PartialBlockWritebackOptions with the configured merge delay and retain explicit zero-delay scheduling in the existing immediate-writeback test fixture. No scheduling or IO algorithm changes. ### Release note When query read-ahead and partial-block writeback are enabled, the default merge delay is 2000 ms and pending-buffer budget is 4 GiB per BE. Foreground range merging and block completion default to a 1.0 amplification ratio and 1.0 minimum coverage respectively. Query read-ahead remains disabled by default. ### Check List (For Author) - Test: ASAN BE unit tests via run-be-ut.sh -j100; 121 tests from 9 suites passed. - Formatting, build hygiene and git diff --check passed. - Behavior changed: Yes, the four configurable defaults above. - Does this need documentation: Yes, synchronize the related PR descriptions and tracking issue.
### What problem does this PR solve? Issue Number: Part of apache#67611 Related PR: apache#67579, apache#67292 Problem Summary: The P0 regression aborts in MapFileColumnIterator::next_batch when a nested STRUCT/MAP lazy read leaves one child pruned. A skipped child retains its old ordinal while the active sibling seeks to the required items; the newly added prefetch assertion incorrectly requires their ordinals to match, even with query read-ahead disabled. Restrict the new read-ahead path to top-level scalar columns, remove complex-column preparation and dependent-item hooks, and restore the upstream ARRAY/MAP/STRUCT read implementations. Select the reader per column and retain the original segment prefetchers for complex columns. Point-query preparation likewise leaves complex iterators on Segment's existing path. Add coverage for divergent nested MAP ordinals, mixed scalar/complex scan routing, and complex point-query preparation. ### Release note Keep ARRAY/MAP/STRUCT on the existing read path when query read-ahead is enabled and prevent an abort during nested lazy materialization. ### Check List (For Author) - Test: Unit Test — 134 tests from 4 suites passed on the integrated apache#67579 tree with ./run-be-ut.sh -j100 --run (ASAN/PCH, no clean). Repository clang-format 16, header hygiene and git diff --check passed. No cluster or clang-tidy run. - Behavior changed: Yes — only top-level scalar columns use the new page read-ahead; legacy complex-column prefetch and common IO/writeback mechanisms are preserved. - Does this need documentation: Yes — scope and synchronization are recorded in apache#67611 and the related PR descriptions.
### What problem does this PR solve? Issue Number: Part of apache#67611 Related PR: apache#67292 Problem Summary: Read-ahead fetches with NO_WRITE and depends on asynchronous writeback after consumption. Enabling it without the asynchronous cache writer silently prevents cold data pages from filling File Cache. Gate both scan and point-query read-ahead on the async-write switch so this configuration retains synchronous cache filling. ### Release note Query read-ahead takes effect only when asynchronous File Cache writes are also enabled. ### Check List (For Author) - Test: Unit Test added for the disabled-writer combination and synchronous cold-read cache reuse; run as part of the repair-series validation. - Behavior changed: Yes, read-ahead remains inactive while asynchronous File Cache writes are disabled. - Does this need documentation: Yes, dependency is recorded in the PR description.
### What problem does this PR solve? Issue Number: Part of apache#67611 Related PR: apache#67292 Problem Summary: A delayed hole-fill task retained the inner remote reader, while destruction of its CachedRemoteFileReader wrapper closed that same inner reader. Later reads failed on HDFS and could abort DCHECK-enabled S3 builds. Use an aliasing shared_ptr whose pointer is the remote reader and whose owner is the cached wrapper, retaining that owner through each background task. ### Release note Keep background cache hole-fill readers open until their tasks finish. ### Check List (For Author) - Test: Unit Test added for submitting a delayed fragment, releasing the wrapper and writeback context, and completing cache fill; run as part of the repair-series validation. - Behavior changed: Yes, queued hole fills retain their cached reader's lifetime. - Does this need documentation: No, internal lifetime correction.
### What problem does this PR solve? Issue Number: apache#67611 Related PR: apache#67292 Problem Summary: A point request can pass the read-ahead enable check and then observe either mutable feature switch disabled while loading its segment. The row-store helper asserted that the factory still returned a read-ahead object, aborting the BE. Reuse the already loaded segment and the original synchronous column iterator when the factory returns no object. A deterministic test closes each switch between segment loading and factory creation, reproduces SIGABRT before the fix, and verifies identical row-store bytes after the fix. ### Release note Prevent BE aborts when query read-ahead or asynchronous File Cache writes are disabled during row-store point queries. ### Check List (For Author) - Test: ASAN/PCH BE unit test PointQueryReadAheadTest.RowStoreFallsBackWhenReadAheadSwitchChanges passed; the same test reproduced SIGABRT before the fix. Build hygiene and clang-format 16 passed. - Behavior changed: Yes; a runtime switch change falls back to the original reader. - Does this need documentation: No; preserve the existing dynamic switch contract. # Conflicts: # be/test/service/point_query_exector_test.cpp # Please enter the commit message for your changes. Lines starting # with '#' will be kept; you may remove them yourself if you want to. # An empty message aborts the commit. # # interactive rebase in progress; onto 3dc7341 # Last commands done (39 commands done): # pick 47d9e5e # [fix](be) Keep cached readers alive during background hole filling # pick 60f0f2e # [fix](be) Fall back when row-store read-ahead is disabled # Next command to do (1 remaining command): # pick 1d9b406 # [improvement](be) Admit read-ahead ranges within remaining byte budgets # You are currently rebasing branch 'fix/phase2-no-pch-build' on '3dc73418a07'. # # Changes to be committed: # modified: be/src/storage/tablet/base_tablet.cpp # modified: be/test/service/point_query_exector_test.cpp # # Untracked files: # .codex #
### What problem does this PR solve? Issue Number: apache#67611 Related PR: apache#67292 Problem Summary: A wide scan submits the initial windows of all columns together. Whole-batch byte admission rejected every range when their aggregate size exceeded a query or BE budget, discarding useful concurrency even when many ranges could fit. Admit ranges in input order with one linear pass under both budget locks, skip ranges that do not fit, and allocate only admitted buffers. Return null handles at skipped input positions so the existing planner mapping remains intact. Segment coordination registers admitted page slices and completes skipped pages for synchronous fallback. Preserve per-buffer reservation lifetime, allocation rollback, executor failures, and cancellation. Count partially budget-rejected batches without double-counting an executor rejection; configuration defaults, windows, coalescing, and hole filling are unchanged. ### Release note Retain partial read-ahead concurrency when a planned range batch exceeds its query or BE byte budget. ### Check List (For Author) - Test: ASAN/PCH BE unit tests through ./run-be-ut.sh --run -j100: 141 tests from 8 suites passed. Coverage includes partial query/BE admission, oversized ranges followed by smaller ranges, original page offsets, consumed-range callbacks, cancellation during allocation, allocation rollback, and executor rejection accounting. Clang-format 16, header hygiene and git diff --check passed. - Behavior changed: Yes; ranges that fit remain asynchronous while skipped ranges use the original synchronous path. - Does this need documentation: Yes; update the component behavior in the existing PR and tracking descriptions.
### What problem does this PR solve? Issue Number: None Related PR: apache#67292, apache#67579 Problem Summary: File Cache initialization rejects an S3 upload buffer size that is smaller than a cache block or not a multiple of the block size. The buffer size is mutable, but runtime updates did not register the same validation, so a later update could violate the layout assumed by cached reads and hole-fill deduplication. Register the existing alignment constraint as a config validator and cover accepted updates, rejected updates with value rollback, invalid block sizes, and File Cache disabled behavior. Cache probing, read-ahead, and hole-fill algorithms remain unchanged. ### Release note Reject S3 upload buffer configuration updates that violate File Cache block alignment while File Cache is enabled. ### Check List (For Author) - Test: Unit Test — 66 tests from 4 suites pass with ./run-be-ut.sh --run -j100 (ASAN/PCH), including configuration validation and partial-block/async cache writeback. Direct Release/PCH BE build passes with ./build.sh --be -j100. Clang-format 16, build hygiene, and diff checks pass. No cluster or clang-tidy run. - Behavior changed: Yes — invalid runtime S3 upload buffer updates return INVALID_ARGUMENT and retain the previous value; defaults and IO mechanisms are unchanged. - Does this need documentation: No — applies the existing File Cache startup constraint to runtime configuration updates.
### What problem does this PR solve? Issue Number: None Related PR: apache#67292, apache#67579 Problem Summary: Older binaries could persist non-aligned File Cache blocks after a runtime S3 upload-buffer update. These intervals survive cache reopening even after the buffer setting is corrected. Partial-block writeback deduplication used a grid-only cache probe, so encountering a restored irregular interval could abort the BE. Check coverage using the actual cached intervals, accepting contiguous DOWNLOADED or DOWNLOADING blocks without allocating cache cells or changing LRU state. Keep the inflight shortcut and the existing grid probe unchanged. ### Release note Avoid a BE abort during partial-block writeback when File Cache restores historical non-aligned blocks. ### Check List (For Author) - Test: The restored-cache regression test reproduced SIGABRT before the fix. All 92 related tests in 5 suites passed with run-be-ut.sh --run -j100 (ASAN/PCH), including restored irregular cache gaps and mixed downloaded/downloading coverage. A direct Release/PCH BE build passed with build.sh --be -j100. Clang-format 16, build hygiene and git diff --check passed. No cluster or clang-tidy run. - Behavior changed: Yes. Optional writeback deduplication supports actual cached block boundaries, including restored irregular blocks. - Does this need documentation: No. The PR descriptions record this compatibility fix; public configuration semantics are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: #67292 (full implementation)
Problem Summary:
This Draft PR was created for no-PCH compilation validation. It now carries the rebased Phase 2 component commits plus the compile fix, observability, hole-fill reuse, page-position windows, parallel hole reads, queued-fragment aggregation, planning timers, page-driven scan windows, object-owned operation cleanup, and configuration-default adjustments tracked below. Do not merge this validation PR; feature delivery is tracked in #67292.
With
ENABLE_PCH=OFFand unity builds enabled, the IO unity translation unit includeserr_utils.cppbeforefile_meta_cache.cpp. The namespace-scopeusing namespace ErrorCodeinerr_utils.cppremains visible when the latter includes the generated protobuf headers, makingPPlanFragmentCancelReason_MIN = LIMIT_REACHambiguous betweendoris::LIMIT_REACHanddoris::ErrorCode::LIMIT_REACH. PCH previously parsed these headers before the using-directive could affect them.Move the using-directive into the two
localfs_erroroverloads that need it. The additional fix changes one file with two insertions and two deletions and preserves the existing error-code mappings.Current synchronization (2026-10-29)
The S3 validator protects new runtime configuration updates. The coverage fix also handles historical non-aligned blocks restored from disk: writeback deduplication walks actual contiguous
DOWNLOADED/DOWNLOADINGintervals, retaining the inflight shortcut and the existing grid-onlyprobe(). The latest fix changes four cache/source-test files; windows, coalescing and hole-fill scheduling are preserved. Corresponding fixes have matching patch IDs. Both branches are rebased onto fixed upstream masterfc1b82283cb, with 42 chronological commits in #67579 and 21 component/fix commits in #67292. Their final source trees are identical. The latest rebase conflict was in scan initialization: retain master's cached bitmap cardinality and prepare scan read-ahead on the normal batch path. The RLE reverse-reach baseline remains 834 with 916 measured users; no dependency refactoring was added.Validation: Each branch independently passed 329 tests / 24 suites with
run-be-ut.sh --run -j100(ASAN/PCH) and a direct Release/PCH BE build withbuild.sh --be -j100, without a clean rebuild. Coverage includes range planning/scheduling, page-position windows, scalar and legacy complex-column reads, count-emission fast paths, point-query fallback, cache/inflight reuse, complete/partial writeback, hole filling, tracing and configuration validation. Full-checkout clang-format 16, build hygiene, gate self-tests, diff/range-diff and final tree-equality checks passed. No cluster or clang-tidy run.Synchronization checkpoint (2026-10-27, historical)
Both complete implementations use upstream master 3dc73418a07. #67579 retains 40 chronological commits; #67292 retains 19 component commits. Their final source trees are identical.
Validation: Each branch independently passed 236 tests / 13 suites with
run-be-ut.sh --run -j100(ASAN/PCH) and a direct Release/PCH BE build withbuild.sh --be -j100, without a clean rebuild. Build hygiene, gate self-tests, clang-format 16, diff/range-diff checks and final tree equality passed. No cluster or clang-tidy run.#67611 and #67612 are closed and are no longer synchronization targets. The collapsed entries below retain historical checkpoints.
2026-09-24 defaults synchronization and validation (historical)
The 2026-09-24 update changes defaults only; planning, queueing and concurrency mechanisms are unchanged. The shared master base remains 01efbca40a5; this update does not rebase onto a newer master.
hole_fill_merge_delay_mshole_fill_max_pending_bytes_per_beread_ahead_block_fill_min_coverageread_ahead_max_read_amplification_ratiohole_fill_max_read_amplification_ratioread_ahead_lazy_window_bytesread_ahead_eager_window_byteshole_fill_workers_per_behole_fill_remote_read_threads_per_behole_fill_merge_delay_msgives queued fragments 2 seconds to aggregate from first admission;0remains available to disable deliberate waiting. A 1.0 foreground amplification limit avoids merging across nonempty gaps, while full block-coverage eligibility prevents extra foreground completion bytes at the default. These settings do not cap total foreground/background duplicate traffic. Query read-ahead remains disabled by default.Source defaults commit 5f9e73ce8fe is folded into #67292's scan initialization (0b676dc846a), partial-block manager (0ca4025b827), hole-fill runtime (6c1d0f5bf69) and writeback test fixture (08a1fdd52d7). #67612 defines none of these nine consumer settings: its code/head are unchanged, and only its synchronization description is updated.
The two complete trees still differ only in #67579's validation-only
err_utils.cppno-PCH fix. Existing default-validation tests are aligned; the immediate-writeback fixture explicitly retains zero delay. No new tests or mechanism changes are added. The new ASAN/PCH runs userun-be-ut.sh -j100without clean; formatting, header hygiene, diff checks and component/tree comparisons pass. No cluster, object-store workload, new analyzer run or clang-tidy run is claimed. Synchronization means pushed, not merged.2026-09-22 synchronization and validation (historical)
All three branches are rebased onto the same upstream master 01efbca40a5. Recent interval diagnostics, batched trace files, submission/scan timing, deadline-ordered queue scans and dedicated hole-fill dispatch are synchronized by component.
Shared tracing is one additional infrastructure component. Hole-fill lifecycle timing, ordered scans and
HoleFillDispatchare folded into the partial-block manager; consumed-range timing/provenance and callback wiring are folded into their owning components. #67612 intentionally excludes scanner/point-query consumers and background hole filling. The two complete source trees differ only in #67579's validation-onlyerr_utils.cppno-PCH fix. TopN second-stage integration remains outside the series.Conflicts preserve upstream's column-reader tests and segment-loading refactor. Point-query reads use the new
TabletColumninterface and preserve slot-derived defaults before a light-schema-change column reaches the tablet schema; an added BE test covers that case. Scanner tests also use a schema-bearingMockSegment, matching upstream's move of table properties out ofStorageReadOptions; the obsolete options assignment is removed.All runs use the repository scripts, ASAN/PCH and
-j100, without a clean rebuild. The Python analyzer's 15 tests pass. Formatting, header hygiene, diff checks, component-prefix and tree-scope comparisons pass. No cluster, external-object-store benchmark or clang-tidy run; these results do not establish a workload speedup. Synchronization means pushed, not merged.Recent follow-up synchronization ledger
All entries above are pushed to the applicable PR branches. Shared infrastructure remains reviewable independently; this validation PR keeps the chronological development commits and its separate no-PCH compile fix. The tracking issue is #67611.
Earlier rebase, benchmark, validation and synchronization checkpoints
Current rebase (2026-09-20)
Rebased only this PR onto upstream master 0451443dd09. Rebase checkpoint: e3382f4aff0; all 30 chronological component commits are retained.
run-be-ut.sh -j100, 236 tests / 19 suites passed; 9 analyzer tests, clang-format 16, BE header hygiene and diff checks passed. No cluster or external-object-store workload was run.Only #67579 is rebased and pushed. Other PR branches, their descriptions and tracking issue #67611 are unchanged. The synchronization tables below describe the earlier dated checkpoint.
Latest follow-up: dedicated hole-fill dispatch (2026-09-22)
Current branch head: b3849c9d65d.
The follow-up q4.1 benchmark at the preceding deadline-scan commit reduced cumulative foreground queue-mutex wait to 27.8 ms, but still recorded 1,053,030 scans for approximately 34,479 activated blocks with 32 workers; 1,018,551 scans encountered an unexpired head. This commit targets the remaining repeated timers, wakeups and trace records. Those figures describe the preceding implementation; a workload benchmark of this new dispatcher is still pending.
flowchart LR Submit[Foreground fragments] --> Waiting[Bounded mergeable queue] Waiting --> Dispatch[HoleFillDispatch<br/>deadline and capacity checks] Dispatch --> Ready[Bounded worker handoff] Ready --> Workers[Block workers<br/>complete and submit blocks] Workers --> Remote[Shared remote-read pool<br/>multi-range GETs] Workers --> Writer[Async cache writer] Workers -. slot released .-> DispatchHoleFillDispatchthread owns merge-deadline and writer-capacity waits. Block workers consume ready tasks and retain their reusable remote-read tokens. The existing single-range inline path, parallel remote reads and configured concurrency remain unchanged.splicewithout allocating new handoff nodes. Active-task metrics include this bounded handoff, not just executing GETs.run-be-ut.sh -j100, 110 tests / 9 suites passed; 15 Python analyzer tests passed. Coverage includes 32 workers with 1024 delayed tasks, full 32-way execution with overflow fragment merging, idle/active resizing, undelivered-task cancellation, deadline/epoch changes, writer capacity, eviction and remote reads. Repository clang-format 16, format checks, BE header hygiene and diff checks passed.Architecture and trace interpretation. No cluster, external-object-store benchmark or clang-tidy run. Pushed only to #67579; #67292, #67612 and tracking issue #67611 remain unchanged.
Earlier follow-up: deadline-ordered hole-fill scans (2026-09-22)
Deadline-scan checkpoint: 1c59525a90f.
The archived q4.1 trace with a 2000 ms merge window measured 389.63 s of cumulative foreground queue-mutex wait and 78.40 s of locked worker scans. Of 219.53 million examined entries, 99.984% were still delayed. This follow-up removes that repeated delayed-task traversal:
run-be-ut.sh -j100, 107 tests / 9 suites passed; 15 Python analyzer tests passed. Tests cover a 1024-task delayed queue with exactly one visited entry per scan, replacement ordering and renewed delay, dynamic delay changes, cross-writer capacity bypass, active merges, eviction and shutdown. Clang-format 16, format check, BE header hygiene and diff checks passed.Updated trace interpretation. No post-change cluster benchmark or clang-tidy run; these tests establish the scan bound, not a query speedup. Pushed only to #67579; #67292, #67612 and tracking issue #67611 remain unchanged.
Earlier follow-up: hole-fill submission and queue-scan timing (2026-09-22)
Timing checkpoint: 7845fde5f6d. One additional diagnostic commit; no changes to read planning, merge delay, queue admission, concurrency or persistence.
The archived q4.1 comparison reduced foreground/hole-fill duplicate bytes with a 2000 ms merge window, but accumulated foreground writeback time increased from about 3.98 s to 271.85 s. Those measurements identify the submission path, not a proven lock-contention cause. This follow-up adds the missing measurements:
ReadAheadWritebackTimeinto complete-block and partial-block submission timers, available even with tracing disabled.enable_read_io_traceswitch, typed timing records separate cache probes, queue mutex wait/hold, block allocation, fragment mutex wait/hold, copies, and existing lifecycle recording. A range-completion event separates complete/partial dispatch.doris_read_io_trace_record_time_ns, exposes serialization/memory-append overhead. Nested timers and cumulative thread times are explicitly documented so they are not added together or mistaken for query wall time.run-be-ut.sh -j100, 87 tests / 9 suites passed; 15 Python analyzer tests passed. Clang-format 16, format check, BE header hygiene and diff checks passed. No cluster, external-object-store benchmark or clang-tidy run. These tests validate instrumentation, not a workload speedup.Timing capture and interpretation. Pushed only to #67579; #67292, #67612 and tracking issue #67611 remain unchanged.
Earlier follow-up: interval overlap diagnostics and batched files (2026-09-20)
Diagnostic checkpoint: 4c87dbbbea2. The diagnostic event instrumentation and dedicated-file writer are two additional commits on the existing chronological branch.
enable_read_io_trace=falseenables actual S3 GET intervals and outcomes, synchronous/read-ahead/hole-fill provenance, range/task identities, consumption/writeback, queued merges and active-ignored fragments.read_io_trace.<process>.<index>.jsonlfiles, notbe.INFO. Foreground threads serialize and append records to a memory batch; one lazy background writer swaps the batch under a short lock and performs file IO outside the lock.read_io_trace_max_events. There is no cumulative event cap, queue-capacity limit, or capacity-based backpressure/drop policy. The startup settings areread_io_trace_dir=""(defaults to the BE log directory'sread_io_trace/),read_io_trace_flush_bytes=1048576(1 MiB), andread_io_trace_flush_interval_ms=1000._exitpath drain submitted records. Pending-byte visibility and per-batch cumulative checkpoints let the analyzer check complete captures and reported writer failures; existingbe.INFOcaptures remain readable.doris_read_io_trace_pending_bytes=0before copying all process files. Reconcile against the same-windows3_file_reader_bytes_readdelta. Instrumentation can perturb timing; leave it disabled for performance measurements. Capture and analysis guide.run-be-ut.sh -j100, 81 tests / 8 suites passed; 12 analyzer tests, clang-format 16, BE header hygiene and diff checks passed. Coverage includes threshold/timer flushing, concurrent producers, shutdown and actual file-error recovery. No cluster, object-store workload or clang-tidy run. Reading, planning, merging and writeback mechanisms are unchanged.Synchronization is intentionally deferred for these diagnostics: pushed only to #67579. #67292, #67612, their descriptions and tracking issue #67611 remain unchanged as requested. The following section records the earlier synchronized checkpoint.
Previous synchronization checkpoint (2026-09-15)
At that checkpoint, the shared master base was
a6751eb3ac6b4667c3291544c5cfb1fae48e1556. At that checkpoint, heads were #67579 484f6cd106f, #67292 af40b6c9022, and #67612 cdec0499937. Synced means pushed to the target PR branch, not merged into master.#67579 keeps its chronological 29 commits. #67292 is organized into 17 component commits; its first five are exactly #67612. Complete-block submission, owned-buffer handoff and low-priority admission are one cache-writer component. The scheduler teardown order enters with query runtime ownership. The full source tree still differs from this validation branch only in the no-PCH
err_utils.cppfix.Follow-up commits on this branch
Regular scans now build candidate physical pages once after pruning and advance compressed-byte windows at data-page reads, including Page Cache hits. Eager and lazy initial windows are submitted together. Exact-row point/nested reads group row IDs by distinct pages. Profile timers distinguish initialization, physical-page advancement, exact-row planning, and window maintenance.
ColumnReadAheadPlanowns reset and timing aggregation; partial-block requests validate themselves and tasks own buffer creation/reads, while the manager keeps queue and lifetime accounting. The 2026-09-15 update changed four defaults only:read_ahead_lazy_window_bytes=1048576(1 MiB),hole_fill_max_read_amplification_ratio=1.0,hole_fill_max_pending_bytes_per_be=1073741824(1 GiB), andhole_fill_merge_delay_ms=0. Planning, concurrency, cache-block completion, and queue mechanisms are unchanged.TopN/global delayed-materialization second-stage integration remains outside the current PRs. Its specific reader/operator/RPC/test files match the shared master base; point-query helpers remain in the final component.
Validation
Both new runs use
run-be-ut.sh -j100, ASAN and PCH, without a clean rebuild. No tests were added; the existing production-default assertions were updated. The source run covers column windows, scanner initialization, partial-block workers/options and hole planning. The three changed files are identical in the full and validation branches; their only overall tree difference remains the validation-onlyerr_utils.cppfix. Repository formatting, BE build hygiene and diff checks pass. No cluster, external-object-store workload or clang-tidy was run; these checks do not establish a workload speedup.2026-09-14 validation before the default-value update
All runs use
run-be-ut.sh -j100, ASAN and PCH, without a clean rebuild. Full-tree validation ran at92dd0c2d85e; moving the existing scheduler teardown-order fix into its runtime component changed commit history only, and the final tree is byte-identical. The tests cover page-driven forward/reverse scans, Page Cache progression, exact-row preparation, plan reuse and statistics, independent-window buffer reuse, asynchronous reads and hole filling. Formatting, BE build hygiene and diff checks pass. No cluster, external-object-store workload or clang-tidy was run; these tests do not establish a workload speedup.17 component commits: correspondence across the three PRs
Release note
None for the additional compile fix. The Phase 2 feature changes are described in #67292.
Check List (For Author)
LIMIT_REACHcompilation failure withENABLE_PCH=OFFandENABLE_UNITY_BUILD=ONusing./build.sh --be -j100../build.sh --be -j100passed through compilation, linking, and packaging (Successfully build Doris), using the existing Release configuration withENABLE_PCH=OFFandENABLE_UNITY_BUILD=ON.build-support/clang-format.sh(clang-format 16),build-support/check-format.sh,build-support/check-build-hygiene.sh, andgit diff --checkpassed.