Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
7654cf7
[feature](be) Add linear file range coalescing
bobhan1 Aug 28, 2026
90cf5b6
[feature](be) Add cache-aware file range planning
bobhan1 Aug 28, 2026
85894d0
[feature](be) Add file range read scheduler
bobhan1 Aug 28, 2026
c64e719
[feature](be) Reuse remote-only reads for no-write ranges
bobhan1 Aug 28, 2026
563b5ba
[feature](be) Add query read-ahead runtime ownership
bobhan1 Aug 28, 2026
e2a4012
[improvement](be) Add opt-in file range IO tracing
bobhan1 Sep 22, 2026
8a08ae1
[fix](be) Scope filesystem error-code imports to their functions
bobhan1 Oct 15, 2026
4271da9
[feature](be) Add page-driven column read-ahead windows
bobhan1 Aug 28, 2026
e7dbfa1
[feature](be) Add segment range read-ahead coordination
bobhan1 Oct 29, 2026
00fa56b
[feature](be) Connect scalar column readers to range read-ahead
bobhan1 Sep 2, 2026
8190e0f
[feature](be) Initialize scan read-ahead once after pruning
bobhan1 Oct 29, 2026
d03f274
[feature](be) Add complete-block cache write handoff and admission
bobhan1 Aug 28, 2026
a547700
[feature](be) Split consumed ranges for cache writeback
bobhan1 Aug 28, 2026
7cfe76c
[feature](be) Add cache block hole-fill planning
bobhan1 Aug 28, 2026
a7e3899
[feature](be) Add background partial block writeback
bobhan1 Aug 28, 2026
999f5c6
[feature](be) Initialize background hole fill
bobhan1 Aug 28, 2026
6fd7bb6
[feature](be) Route consumed ranges to cache writeback
bobhan1 Aug 28, 2026
1ae1272
[feature](be) Connect read-ahead range writeback
bobhan1 Aug 28, 2026
fd6df2c
[feature](be) Extend range read-ahead to point queries
bobhan1 Sep 2, 2026
a881473
[fix](be) Validate S3 upload buffer alignment on config updates
bobhan1 Oct 29, 2026
b237ebc
[fix](be) Handle irregular cache blocks in writeback coverage checks
bobhan1 Oct 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[feature](be) Add background partial block writeback
### What problem does this PR solve?

Issue Number: Part of #67611

Related PR: #67292, #67579

Problem Summary: Partial cache blocks need bounded background completion without stalling query reads. Keep queued fragments mergeable under a configurable aggregation delay and a BE byte budget. One HoleFillDispatch thread owns admission-time-ordered deadline and writer-capacity checks, stops at the first future deadline, bypasses busy writers for other due tasks, and hands off at most the configured block-worker slots. Workers execute ready tasks with reusable remote-read tokens: one hole range is read inline and multiple ranges use the shared remote pool. Retain cache/inflight reuse, epoch checks, eviction, dynamic worker/pool/delay updates, and complete-block handoff. Trace fragment lifecycles and submission/queue timings with serialization outside the queue mutex. Cancel queued and undelivered work at shutdown and drain executing reads before destroying the remote pool.

### Release note

Add opt-in background cache-block hole filling with queued-fragment aggregation, dedicated dispatch, bounded worker handoff and parallel hole reads.

### Check List (For Author)

- Test: Unit Test
    - Component tests are included with the implementation; rebase and synchronization validation is recorded in the related PRs and tracking issue.
    - Repository clang-format 16, header hygiene and git diff --check passed before folding.
    - No cluster, external-object-store benchmark or clang-tidy run during synchronization.
- Behavior changed: Yes, includes final component behavior and applicable diagnostic instrumentation; default enable switches remain off.
- Does this need documentation: Yes, tools/read_io_trace/README.md and related PR/tracking descriptions.
  • Loading branch information
bobhan1 committed Oct 29, 2026
commit a7e38995a3c663606dc8286b7ae4512b95f5c70f
36 changes: 33 additions & 3 deletions be/src/io/cache/async_cache_write_manager.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -419,7 +419,7 @@ Status AsyncCacheWriteManager::start() {
RETURN_IF_ERROR(
ThreadPoolBuilder(fmt::format("AsyncFileCacheWrite-{}",
std::hash<std::string> {}(_cache->get_base_path())))
.set_min_threads(0)
.set_min_threads(static_cast<int>(worker_count))
.set_max_threads(static_cast<int>(worker_count))
.set_max_queue_size(128)
.build(&_worker_pool));
Expand Down Expand Up @@ -574,15 +574,45 @@ AsyncCacheWriteBlockSubmitResult AsyncCacheWriteManager::try_submit_owned_block(
}

bool AsyncCacheWriteManager::can_accept_without_eviction(size_t buffer_size) const {
return available_slots_without_eviction(buffer_size) > 0;
}

size_t AsyncCacheWriteManager::available_slots_without_eviction(size_t buffer_size) const {
DORIS_CHECK(buffer_size > 0);
std::lock_guard lock(_queue_mutex);
if (!_started.load(std::memory_order_acquire) || !_accepting.load(std::memory_order_acquire)) {
return false;
return 0;
}
DORIS_CHECK(_task_buffer_size == 0 || _task_buffer_size == buffer_size);
const size_t max_pending_bytes = _options.load(std::memory_order_acquire)->max_pending_bytes;
const size_t pending_bytes = _pending_bytes.load(std::memory_order_relaxed);
return buffer_size <= max_pending_bytes && pending_bytes <= max_pending_bytes - buffer_size;
if (pending_bytes >= max_pending_bytes) {
return 0;
}
return (max_pending_bytes - pending_bytes) / buffer_size;
}

bool AsyncCacheWriteManager::accepting() const {
return _started.load(std::memory_order_acquire) && _accepting.load(std::memory_order_acquire);
}

bool AsyncCacheWriteManager::should_skip_block_writeback(
const UInt128Wrapper& cache_hash, size_t offset, size_t size,
const CacheAdmissionContext& admission_ctx,
InflightWriteBufferIndex* inflight_index) const {
if (inflight_index != nullptr && inflight_index->lookup(cache_hash, offset) != nullptr) {
return true;
}
ReadStatistics stats;
const auto context = admission_ctx.to_cache_context(&stats);
auto result = _cache->probe(cache_hash, offset, size, context);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Handle irregular cached upload blocks before probing

Supported S3 upload settings can leave cache blocks that are not aligned to the global block grid: s3_write_buffer_size=5.5 MiB with 1 MiB cache blocks makes S3FileWriter populate intervals such as [5,5.5) and [5.5,6.5) MiB. After the former is evicted, consuming a read-ahead range such as [5.25,5.75) fetches remote bytes and submits a partial fragment for [5,6). This call then reaches BlockFileCache::probe(), whose DORIS_CHECK requires the resident block to start at 5 MiB; finding the retained 5.5 MiB block aborts the BE. Native queries with peer reads enabled previously used the synchronous, interval-aware get_or_set() path even with async writes enabled, so enabling read-ahead newly exposes this crash. Use an interval-aware coverage check here and cover partially evicted, non-multiple S3 upload layouts.

DORIS_CHECK(result.file_blocks.size() == 1);
const auto& block = result.file_blocks.front();
if (block == nullptr) {
return false;
}
const auto state = block->state();
return state == FileBlock::State::DOWNLOADED || state == FileBlock::State::DOWNLOADING;
}

Status AsyncCacheWriteManager::allocate_tracked_buffer(size_t size,
Expand Down
15 changes: 15 additions & 0 deletions be/src/io/cache/async_cache_write_manager.h
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,21 @@ class AsyncCacheWriteManager {
/// before the caller hands off its task.
bool can_accept_without_eviction(size_t buffer_size) const;

/// Return how many fixed-size tasks fit in the same point-in-time spare capacity. Background
/// schedulers use this to avoid launching multiple remote reads against one available slot.
size_t available_slots_without_eviction(size_t buffer_size) const;

/// Return whether the manager currently accepts submissions. This is a point-in-time hint;
/// submission methods still make the race-safe final decision.
bool accepting() const;

/// Check inflight buffers first, then probe one aligned cache block. Skip writeback if the
/// block is already cached or being filled. Each lookup releases its lock before the next;
/// this call must run outside the hole-fill queue mutex because probe may load cache metadata.
bool should_skip_block_writeback(const UInt128Wrapper& cache_hash, size_t offset, size_t size,
const CacheAdmissionContext& admission_ctx,
InflightWriteBufferIndex* inflight_index) const;

/// Allocate `size` payload bytes charged to the manager tracker and return them in `buffer`.
Status allocate_tracked_buffer(size_t size, AsyncCacheWriteBufferPtr* buffer);

Expand Down
Loading