Skip to content
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
6e312b9
[feature](be) Add linear file range coalescing
bobhan1 Aug 28, 2026
01b4553
[feature](be) Add cache-aware file range planning
bobhan1 Aug 28, 2026
bd0137c
[feature](be) Add file range read scheduler
bobhan1 Aug 28, 2026
e313aa9
[feature](be) Reuse remote-only reads for no-write ranges
bobhan1 Aug 28, 2026
f6b2bf2
[feature](be) Add column read-ahead byte windows
bobhan1 Aug 28, 2026
500fa70
[feature](be) Add segment range read-ahead coordination
bobhan1 Oct 29, 2026
14d225b
[feature](be) Add query read-ahead runtime ownership
bobhan1 Aug 28, 2026
2b37479
[feature](be) Connect column readers to range read-ahead
bobhan1 Sep 2, 2026
182bc8b
[feature](be) Integrate query range read-ahead
bobhan1 Aug 28, 2026
8aa1ad9
[refactor](be) Expose complete cache block submission
bobhan1 Aug 28, 2026
2751105
[feature](be) Split consumed ranges for cache writeback
bobhan1 Aug 28, 2026
5300af8
[feature](be) Add cache block hole-fill planning
bobhan1 Aug 28, 2026
b169fb7
[feature](be) Add zero-copy cache block handoff
bobhan1 Aug 28, 2026
785b567
[feature](be) Add low-priority cache write admission
bobhan1 Aug 28, 2026
f4b7f34
[feature](be) Add background partial block writeback
bobhan1 Aug 28, 2026
f16869f
[feature](be) Initialize background hole fill
bobhan1 Aug 28, 2026
400da5e
[feature](be) Route consumed ranges to cache writeback
bobhan1 Aug 28, 2026
81e6234
[feature](be) Connect read-ahead range writeback
bobhan1 Aug 28, 2026
1141837
[feature](be) Extend range read-ahead to point queries
bobhan1 Sep 2, 2026
8e27dce
[fix](be) Scope filesystem error-code imports to their functions
bobhan1 Sep 7, 2026
428a6ca
[improvement](be) Add page read-ahead and hole-fill observability
bobhan1 Sep 8, 2026
1345123
[fix](be) Skip redundant hole-fill reads for cached blocks
bobhan1 Sep 10, 2026
40a6b08
[improvement](be) Advance read-ahead windows by page position
bobhan1 Sep 11, 2026
d1d2cb9
[improvement](be) Parallelize hole-fill range reads
bobhan1 Sep 11, 2026
9236d84
[improvement](be) Delay hole filling to merge queued fragments
bobhan1 Sep 11, 2026
60a1d92
[improvement](be) Add column read-ahead planning timers
bobhan1 Sep 11, 2026
29eec89
[improvement](be) Advance scan read-ahead from data-page reads
bobhan1 Sep 14, 2026
446cfec
[refactor](be) Keep read-ahead operations with their owning objects
bobhan1 Sep 14, 2026
a6d93e4
[improvement](be) Adjust read-ahead and hole-fill defaults
bobhan1 Sep 15, 2026
7bdeba2
[improvement](be) Add opt-in read overlap diagnostics
bobhan1 Sep 20, 2026
48eed95
[improvement](be) Buffer read IO traces in dedicated files
bobhan1 Sep 20, 2026
508235b
[improvement](be) Trace hole-fill submission and queue-scan latency
bobhan1 Sep 22, 2026
53dcc8a
[improvement](be) Avoid scanning delayed hole-fill tasks
bobhan1 Sep 22, 2026
e0e61f3
[improvement](be) Dispatch hole-fill tasks from a dedicated thread
bobhan1 Sep 22, 2026
009cc67
[improvement](be) Tune read-ahead and hole-fill defaults
bobhan1 Sep 24, 2026
ac3214d
[fix](be) Keep complex columns on their original read path
bobhan1 Oct 15, 2026
38d9e37
[fix](be) Require async cache writes for query read-ahead
bobhan1 Oct 20, 2026
da1c24a
[fix](be) Keep cached readers alive during background hole filling
bobhan1 Oct 20, 2026
36cf25a
[fix](be) Fall back when row-store read-ahead is disabled
bobhan1 Oct 23, 2026
eccd7ce
[improvement](be) Admit read-ahead ranges within remaining byte budgets
bobhan1 Oct 23, 2026
a23fc63
[fix](be) Validate S3 upload buffer alignment on config updates
bobhan1 Oct 29, 2026
b4b194a
[fix](be) Handle irregular cache blocks in writeback coverage checks
bobhan1 Oct 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
[fix](be) Handle irregular cache blocks in writeback coverage checks
### What problem does this PR solve?

Issue Number: None

Related PR: #67292, #67579

Problem Summary: Older binaries could persist non-aligned File Cache blocks after a runtime S3 upload-buffer update. These intervals survive cache reopening even after the buffer setting is corrected. Partial-block writeback deduplication used a grid-only cache probe, so encountering a restored irregular interval could abort the BE. Check coverage using the actual cached intervals, accepting contiguous DOWNLOADED or DOWNLOADING blocks without allocating cache cells or changing LRU state. Keep the inflight shortcut and the existing grid probe unchanged.

### Release note

Avoid a BE abort during partial-block writeback when File Cache restores historical non-aligned blocks.

### Check List (For Author)

- Test: The restored-cache regression test reproduced SIGABRT before the fix. All 92 related tests in 5 suites passed with run-be-ut.sh --run -j100 (ASAN/PCH), including restored irregular cache gaps and mixed downloaded/downloading coverage. A direct Release/PCH BE build passed with build.sh --be -j100. Clang-format 16, build hygiene and git diff --check passed. No cluster or clang-tidy run.
- Behavior changed: Yes. Optional writeback deduplication supports actual cached block boundaries, including restored irregular blocks.
- Does this need documentation: No. The PR descriptions record this compatibility fix; public configuration semantics are unchanged.
  • Loading branch information
bobhan1 committed Oct 29, 2026
commit b4b194ada0b11bdc4f80d9a7c8f2103d13b7b0d2
9 changes: 1 addition & 8 deletions be/src/io/cache/async_cache_write_manager.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -605,14 +605,7 @@ bool AsyncCacheWriteManager::should_skip_block_writeback(
}
ReadStatistics stats;
const auto context = admission_ctx.to_cache_context(&stats);
auto result = _cache->probe(cache_hash, offset, size, context);
DORIS_CHECK(result.file_blocks.size() == 1);
const auto& block = result.file_blocks.front();
if (block == nullptr) {
return false;
}
const auto state = block->state();
return state == FileBlock::State::DOWNLOADED || state == FileBlock::State::DOWNLOADING;
return _cache->is_range_downloaded_or_downloading(cache_hash, offset, size, context);
}

Status AsyncCacheWriteManager::allocate_tracked_buffer(size_t size,
Expand Down
45 changes: 45 additions & 0 deletions be/src/io/cache/block_file_cache.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -918,6 +918,51 @@ std::vector<FileBlock::Range> BlockFileCache::downloaded_ranges(const UInt128Wra
return ranges;
}

bool BlockFileCache::is_range_downloaded_or_downloading(const UInt128Wrapper& hash, size_t offset,
size_t size, const CacheContext& context) {
DORIS_CHECK(size > 0);
const size_t end = offset + size;
DORIS_CHECK(end > offset);
SCOPED_CACHE_LOCK(_mutex, this);
auto file = _files.find(hash);
if (file == _files.end() && !_async_open_done) {
_storage->load_file_blocks_directly_unlocked(this, hash, context.tablet_id, cache_lock);
file = _files.find(hash);
}
if (file == _files.end()) {
return false;
}

const auto& blocks = file->second;
DORIS_CHECK(!blocks.empty());
auto iterator = blocks.lower_bound(offset);
if (iterator != blocks.begin()) {
--iterator;
}
// Old upload boundaries can leave [0, 512 KiB) and [512 KiB, 1.5 MiB). Walk their
// actual intervals rather than assuming one cached cell per aligned writeback block.
size_t covered = offset;
for (; iterator != blocks.end(); ++iterator) {
const auto& block = iterator->second.file_block;
const auto& range = block->range();
if (range.right < covered) {
continue;
}
if (range.left > covered) {
return false;
}
const auto state = block->state();
if (state != FileBlock::State::DOWNLOADED && state != FileBlock::State::DOWNLOADING) {
return false;
}
if (range.right >= end - 1) {
return true;
}
covered = range.right + 1;
}
return false;
}

FileBlocksProbeResult BlockFileCache::probe(const UInt128Wrapper& hash, size_t offset, size_t size,
const CacheContext& context) {
DORIS_CHECK(size > 0);
Expand Down
6 changes: 6 additions & 0 deletions be/src/io/cache/block_file_cache.h
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,12 @@ class BlockFileCache {
FileBlocksHolder get_or_set(const UInt128Wrapper& hash, size_t offset, size_t size,
CacheContext& context);

/// Whether existing DOWNLOADED/DOWNLOADING blocks fully cover `[offset, offset + size)`.
/// Handles arbitrary block boundaries, including restored upload blocks. May load metadata,
/// but never reserves cache cells, waits for downloads or touches LRU state.
bool is_range_downloaded_or_downloading(const UInt128Wrapper& hash, size_t offset, size_t size,
const CacheContext& context);

/// Probe the block-aligned `[offset, offset + size)` range without creating cache cells or
/// touching LRU state. The result contains one ordered slot per cache block; each slot is null
/// on miss or owns an existing block that starts at and covers the slot. A final short slot can
Expand Down
94 changes: 94 additions & 0 deletions be/test/io/cache/partial_block_writeback_manager_test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,100 @@ TEST_F(PartialBlockWritebackManagerTest, SkipsDownloadedBlockWithoutAllocating)
EXPECT_EQ(read_cached_block(cache.get(), hash), content);
}

TEST_F(PartialBlockWritebackManagerTest, SkipsMixedIrregularCoverageWithoutAllocating) {
auto cache = create_cache("partial_block_irregular_coverage");
auto* writer = cache->async_write_manager();
const auto hash = BlockFileCache::hash("partial_block_irregular_coverage");
const auto content = patterned_content('i');
ReadStatistics stats;
CacheContext context;
context.stats = &stats;

// Historical upload boundaries can leave [0, 1 KiB) and [1, 4 KiB) in one grid block.
auto left = cache->get_or_set(hash, 0, 1024, context);
ASSERT_EQ(left.file_blocks.size(), 1);
auto left_block = left.file_blocks.front();
ASSERT_EQ(left_block->get_or_set_downloader(), FileBlock::get_caller_id());
ASSERT_TRUE(left_block->append(Slice(content.data(), 1024)).ok());
ASSERT_TRUE(left_block->finalize().ok());
auto right = cache->get_or_set(hash, 1024, kBlockSize - 1024, context);
ASSERT_EQ(right.file_blocks.size(), 1);
auto right_block = right.file_blocks.front();
EXPECT_FALSE(writer->should_skip_block_writeback(hash, 0, kBlockSize, {}, nullptr));

ASSERT_EQ(right_block->get_or_set_downloader(), FileBlock::get_caller_id());
EXPECT_TRUE(writer->should_skip_block_writeback(hash, 0, kBlockSize, {}, nullptr));
EXPECT_TRUE(writer->should_skip_block_writeback(hash, 512, 1024, {}, nullptr));
EXPECT_TRUE(writer->should_skip_block_writeback(hash, 1024, 512, {}, nullptr));
EXPECT_FALSE(writer->should_skip_block_writeback(hash, 0, kBlockSize + 1, {}, nullptr));
const auto missing_hash = BlockFileCache::hash("partial_block_irregular_missing");
EXPECT_FALSE(writer->should_skip_block_writeback(missing_hash, 0, kBlockSize, {}, nullptr));
EXPECT_TRUE(cache->get_blocks_by_key(missing_hash).empty());

auto manager = create_manager();
auto reader = std::make_shared<ControlledFileReader>(content);
EXPECT_EQ(manager->try_submit(make_request(writer, nullptr, reader, hash, content, 0, 512)),
PartialBlockSubmitResult::CACHE_BLOCK_PRESENT);
EXPECT_EQ(manager->pending_count(), 0);
EXPECT_EQ(writer->buffer_memory_bytes(), 0);
EXPECT_EQ(reader->read_calls(), 0);
EXPECT_EQ(right_block->state(), FileBlock::State::DOWNLOADING);
EXPECT_EQ(right_block->get_downloader(), FileBlock::get_caller_id());
}

TEST_F(PartialBlockWritebackManagerTest, FillsRestoredIrregularCacheGap) {
const std::string name = "partial_block_restored_irregular_gap";
const auto hash = BlockFileCache::hash(name);
const auto content = patterned_content('h') + patterned_content('n');
{
auto cache = create_cache(name);
ReadStatistics stats;
CacheContext context;
context.stats = &stats;
// Persist [2, 6 KiB): it overlaps the [0, 4 KiB) target, with a hole at its start.
auto holder = cache->get_or_set(hash, kBlockSize / 2, kBlockSize, context);
ASSERT_EQ(holder.file_blocks.size(), 1);
auto block = holder.file_blocks.front();
ASSERT_EQ(block->get_or_set_downloader(), FileBlock::get_caller_id());
ASSERT_TRUE(block->append(Slice(content.data() + kBlockSize / 2, kBlockSize)).ok());
ASSERT_TRUE(block->finalize().ok());
}

auto cache = std::make_unique<BlockFileCache>((caches_dir / name).string(),
partial_writeback_cache_settings());
ASSERT_TRUE(cache->initialize().ok());
wait_until_cache_ready(*cache);
const auto restored = cache->get_blocks_by_key(hash);
ASSERT_EQ(restored.size(), 1);
EXPECT_EQ(restored.begin()->second->range().left, kBlockSize / 2);
EXPECT_EQ(restored.begin()->second->range().right, kBlockSize * 3 / 2 - 1);
auto* writer = cache->async_write_manager();
ASSERT_TRUE(writer->start().ok());
auto manager = create_manager();
auto reader = std::make_shared<ControlledFileReader>(content);

ASSERT_EQ(manager->try_submit(make_request(writer, nullptr, reader, hash, content, 1024, 1024)),
PartialBlockSubmitResult::QUEUED);
ASSERT_TRUE(wait_until([&]() { return manager->pending_count() == 0; }));
ASSERT_TRUE(wait_until([&]() { return writer->pending_count() == 0; }));
EXPECT_EQ(reader->read_calls(), 2);

ReadStatistics stats;
CacheContext context;
context.stats = &stats;
FileBlocks blocks;
bool fully_covered = false;
ASSERT_TRUE(cache->get_downloaded_blocks_if_fully_covered(hash, 0, kBlockSize, context, &blocks,
&fully_covered)
.ok());
ASSERT_TRUE(fully_covered);
ASSERT_EQ(blocks.size(), 2);
std::string cached(kBlockSize, '\0');
ASSERT_TRUE(blocks.front()->read(Slice(cached.data(), kBlockSize / 2), 0).ok());
ASSERT_TRUE(blocks.back()->read(Slice(cached.data() + kBlockSize / 2, kBlockSize / 2), 0).ok());
EXPECT_EQ(cached, content.substr(0, kBlockSize));
}

TEST_F(PartialBlockWritebackManagerTest, SkipsInflightBlockWithoutProbingCache) {
auto cache = create_cache("partial_block_inflight_before_probe");
auto* writer = cache->async_write_manager();
Expand Down
Loading