Skip to content

Commit 0a3595e

Browse files
authored
[improvement](index) Bound SNII posting workspace across spill and encoding (apache#67657)
### What problem does this PR solve? Related PR: apache#67537 (merged into master) **Under review: the error-cleanup and query-performance items below remain open.** This branch is rebased onto master at `e429e59bdfc` after apache#67537 merged. The master comparison now contains only the two spill-specific commits (46 changed files). The CommonGrams deletion and released-format alignment are already in master through apache#67537, so those removed lines do not recur in this PR diff. Problem Summary: SNII expands compact postings into complete document, frequency and position arrays during spill, run merging and final encoding. A frequent term or a very long document can therefore create a large allocation while the original posting arena is still live. Limiting the first spill alone moves the peak into the merge or encoder. Repeatedly merging an increasing run prefix also amplifies temporary I/O. This change reserves one hard posting workspace before allocating, shared across the writer's spill, merge and final encoding: - Append compact fragments to one spool with independently sealed, checksummed run ranges. The run directory itself can spill. - Merge contiguous groups using fan-in constrained by workspace and available file descriptors. Intermediate passes copy encoded fragments; final consumption combines boundary documents in their original position order. - Replay large position payloads and stage DD, window metadata and inline DD/PRX within the same budget. Ordinary admitted windows retain the existing resident encoder. - Keep input arena, dictionary keys/ranks, norms, NULL metadata, allocator retention and OS page cache outside this posting-workspace limit. It is an algorithmic workspace bound, not a whole-BE RSS limit. - Preserve sequential read-ahead across small-term boundaries. The added regression originally read 578,549,868 bytes from a 271,632-byte run because every term discarded the same 64 KiB cache. The persistent SNII layout and window policy established by apache#67537 are retained. Persistent readers are unchanged. Large streaming ZSTD frames can have different compressed bytes, so reader compatibility alone is not treated as evidence of equal query performance.
1 parent 7dc90d1 commit 0a3595e

47 files changed

Lines changed: 5301 additions & 1028 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎be/src/common/config.cpp‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1463,10 +1463,17 @@ DEFINE_mInt32(snii_index_build_max_memory_limit_percent, "10");
14631463
// ONLY the arena, so smaller triggers cut tiny runs for near-zero relief.
14641464
// Default 64 MiB.
14651465
DEFINE_mInt64(snii_forced_spill_min_arena_bytes, "67108864");
1466-
// Max spill-run files one SNII writer accumulates before its runs are
1467-
// merge-compacted into one (bounds the k-way merge fan-in and its open fds;
1468-
// every run is held open for the whole merge). 0 = uncapped. Default 64.
1466+
// Historical run-file cap: spill ranges now share one append-only spool, so
1467+
// ingestion does not accumulate one physical file per run. This knob additionally
1468+
// limits active inputs per final/intermediate merge group (minimum two). Workspace
1469+
// and fd limits also constrain fan-in; 0 uses those bounds without an extra cap.
14691470
DEFINE_mInt32(snii_spill_max_run_files_per_buffer, "64");
1471+
// Hard budget shared by a logical writer's posting spill, merge, and encoding
1472+
// buffers. Captured when an ingestion writer or compaction reporter is created.
1473+
// High ZSTD levels may require a larger budget for the compression context.
1474+
DEFINE_mInt64(snii_postings_workspace_bytes, "33554432");
1475+
DEFINE_Validator(snii_postings_workspace_bytes,
1476+
[](const int64_t value) -> bool { return value > 0; });
14701477
// dict path for chinese analyzer
14711478
DEFINE_String(inverted_index_dict_path, "${DORIS_HOME}/dict");
14721479
// The kuromoji (Japanese) analyzer

‎be/src/common/config.h‎

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1498,21 +1498,20 @@ DECLARE_mInt32(snii_index_build_max_memory_limit_percent);
14981498
// reclaims ONLY the posting arena -- the persistent vocab / pair-map
14991499
// structures survive it -- so honoring below a real floor degenerates into a
15001500
// storm of tiny runs whenever the memory over the share is dominated by
1501-
// persistent bytes (each run then costs a file, a sort and a merge-fd for
1501+
// persistent bytes (each run still adds framing, sorting and merge work for
15021502
// near-zero memory relief). THIS FLOOR, not any judgement about whether the
15031503
// overage is reachable, is what bounds forced spilling: it caps the cost at one
15041504
// >= floor-sized run per floor of arena growth per writer. Forced spilling
15051505
// therefore reclaims SPILLABLE memory only, never persistent memory.
15061506
// Default 64 MiB.
15071507
DECLARE_mInt64(snii_forced_spill_min_arena_bytes);
1508-
// G09 run-file cap: maximum spill-run files one SNII writer may accumulate;
1509-
// on the next spill past the cap, the existing runs are merge-compacted into
1510-
// a single run first (term stream unchanged). Bounds the final k-way merge's
1511-
// fan-in and, decisively, its simultaneously-open file descriptors -- every
1512-
// run of a buffer is reopened and held open for the whole merge, so unbounded
1513-
// run counts across ~100 concurrent writers can exhaust the BE nofile rlimit
1514-
// ("Too many open files" at run reopen). 0 disables the cap. Default 64.
1508+
// Historical run-file cap, now an additional limit on active inputs per merge
1509+
// group (minimum two). Ingestion runs share one append-only spool. Zero leaves
1510+
// fan-in bounded by the postings workspace and fd limits. Default 64.
15151511
DECLARE_mInt32(snii_spill_max_run_files_per_buffer);
1512+
// Shared hard posting-workspace budget, captured by new ingestion/compaction
1513+
// reporters. Positive bytes; default 32 MiB. High ZSTD levels may need more.
1514+
DECLARE_mInt64(snii_postings_workspace_bytes);
15161515
// dict path for chinese analyzer
15171516
DECLARE_String(inverted_index_dict_path);
15181517
// The kuromoji (Japanese) analyzer

‎be/src/storage/CMakeLists.txt‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,14 @@ pch_reuse(Storage)
5656
# - the three heaviest template-instantiation TUs (predicate creators) which
5757
# would dominate any batch they join
5858
set(STORAGE_UNITY_SKIP
59+
# Keep the spill helpers independent: the PRX encoder requests ZSTD's static
60+
# workspace API, and inserting these files into the existing unity batches
61+
# exposes an unrelated LIMIT_REACH enum collision in olap_scan_common.cpp.
62+
${CMAKE_CURRENT_SOURCE_DIR}/index/snii/writer/bounded_run_merge.cpp
63+
${CMAKE_CURRENT_SOURCE_DIR}/index/snii/writer/encoded_spill_run.cpp
64+
${CMAKE_CURRENT_SOURCE_DIR}/index/snii/writer/posting_byte_buffer.cpp
65+
${CMAKE_CURRENT_SOURCE_DIR}/index/snii/writer/posting_external_sort.cpp
66+
${CMAKE_CURRENT_SOURCE_DIR}/index/snii/writer/posting_prx_encoder.cpp
5967
${CMAKE_CURRENT_SOURCE_DIR}/index/inverted/inverted_index_compound_reader.cpp
6068
${CMAKE_CURRENT_SOURCE_DIR}/index/inverted/inverted_index_fs_directory.cpp
6169
${CMAKE_CURRENT_SOURCE_DIR}/index/inverted/tokenizer/basic/basic_tokenizer.cpp

‎be/src/storage/compaction/compaction.cpp‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1123,7 +1123,8 @@ Status Compaction::do_inverted_index_compaction() {
11231123
snii_merge_memory_reporter = std::make_shared<snii::writer::MemoryReporter>(
11241124
snii::writer::snii_build_consume_release(
11251125
snii::writer::BuildMemoryPopulation::kUnregistered),
1126-
spill_threshold, snii::writer::MemoryReporter::CapPolicy::kHardLimit);
1126+
spill_threshold, snii::writer::MemoryReporter::CapPolicy::kHardLimit,
1127+
static_cast<uint64_t>(config::snii_postings_workspace_bytes));
11271128
}
11281129
for (auto&& [column_uniq_id, index_metas] :
11291130
collect_index_compaction_domain(*_cur_tablet_schema, ctx)) {

‎be/src/storage/index/snii/compaction/posting_run_merger.cpp‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -274,14 +274,13 @@ Status MergedPostingRuns::fill(uint32_t target_docs, writer::TermPostingBuffer*
274274
*exhausted = true;
275275
return Status::OK();
276276
}
277-
const size_t position_count = run.positions_flat.size();
278277
writer::MutableTermPostingSpan destination;
279-
RETURN_IF_ERROR(out->grow_uninitialized(run.docids.size(), retain_positions_,
280-
position_count, &destination));
278+
RETURN_IF_ERROR(
279+
out->grow_uninitialized(run.docids.size(), retain_positions_, 0, &destination));
281280
std::ranges::copy(run.docids, destination.docids.begin());
282281
if (retain_positions_) {
283282
std::ranges::copy(run.freqs, destination.freqs.begin());
284-
std::ranges::copy(run.positions_flat, destination.positions_flat.begin());
283+
RETURN_IF_ERROR(out->append_positions(run.positions_flat));
285284
}
286285
#ifdef BE_TEST
287286
posting_run_copied_document_counter.fetch_add(run.docids.size(), std::memory_order_relaxed);

‎be/src/storage/index/snii/format/dict_block.cpp‎

Lines changed: 120 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -36,18 +36,20 @@ constexpr size_t kAnchorOffBytes = sizeof(uint32_t); // per-anchor offset u32
3636
// Estimate the encoded upper-bound byte size of one entry (no actual encoding; used by
3737
// estimated_bytes). Take the maximum varint width of each variable-length field plus payload bytes
3838
// to guarantee an upper bound.
39-
size_t estimate_entry_bytes(const DictEntry& e, IndexTier tier) {
39+
size_t estimate_entry_bytes(const DictEntry& e, IndexTier tier,
40+
uint64_t external_inline_prx_length = 0,
41+
uint64_t external_inline_frq_length = 0) {
4042
size_t body = 0;
4143
body += varint_len(static_cast<uint32_t>(e.term.size())); // prefix_len upper bound
4244
body += varint_len(static_cast<uint32_t>(e.term.size())); // suffix_len upper bound
4345
body += e.term.size(); // suffix bytes upper bound
4446
body += 1; // flags
4547
body += 10; // df upper bound
4648
if (e.kind == DictEntryKind::kInline) {
47-
body += 10 + e.frq_bytes.size();
49+
body += 10 + e.frq_bytes.size() + external_inline_frq_length;
4850
body += 1 + 10; // codec mode + dd uncomp_len
4951
if (tier >= IndexTier::kT2) {
50-
body += 10 + e.prx_bytes.size();
52+
body += 10 + e.prx_bytes.size() + external_inline_prx_length;
5153
}
5254
} else {
5355
body += 10 * 2; // frq_off/frq_len
@@ -59,6 +61,40 @@ size_t estimate_entry_bytes(const DictEntry& e, IndexTier tier) {
5961
return varint_len(static_cast<uint64_t>(body)) + body;
6062
}
6163

64+
Status append_dictionary_bytes(Slice bytes, const std::function<Status(Slice)>& append,
65+
uint32_t* crc, uint64_t* written) {
66+
RETURN_IF_ERROR(append(bytes));
67+
*crc = crc32c_extend(*crc, bytes);
68+
*written += bytes.size();
69+
return Status::OK();
70+
}
71+
72+
// Replace the empty inline PRX field emitted by the shared entry codec with a streamed field.
73+
Status append_external_inline_prx(
74+
Slice entry_bytes, uint64_t length,
75+
const std::function<Status(const std::function<Status(Slice)>&)>& visit,
76+
const std::function<Status(Slice)>& append) {
77+
ByteSource encoded(entry_bytes);
78+
uint64_t body_length = 0;
79+
RETURN_IF_ERROR(encoded.get_varint64(&body_length));
80+
DORIS_CHECK(body_length != 0 && entry_bytes.data()[entry_bytes.size() - 1] == 0);
81+
ByteSink header;
82+
header.put_varint64(body_length - 1 + varint_len(length) + length);
83+
RETURN_IF_ERROR(append(header.view()));
84+
RETURN_IF_ERROR(append(entry_bytes.subslice(encoded.position(), body_length - 1)));
85+
header.clear();
86+
header.put_varint64(length);
87+
RETURN_IF_ERROR(append(header.view()));
88+
uint64_t written = 0;
89+
RETURN_IF_ERROR(visit([&](Slice bytes) {
90+
RETURN_IF_ERROR(append(bytes));
91+
written += bytes.size();
92+
return Status::OK();
93+
}));
94+
DORIS_CHECK(written == length);
95+
return Status::OK();
96+
}
97+
6298
} // namespace
6399

64100
// ---- DictBlockBuilder ----
@@ -80,15 +116,17 @@ void DictBlockBuilder::add_entry(const DictEntry& entry) {
80116
++n_entries_;
81117
}
82118

83-
void DictBlockBuilder::add_entry(DictEntry&& entry) {
119+
void DictBlockBuilder::add_entry(DictEntry&& entry, uint64_t external_inline_prx_length,
120+
uint64_t external_inline_frq_length) {
84121
if (is_anchor(n_entries_)) {
85122
++n_anchors_;
86123
}
87124
// estimate_entry_bytes reads `entry`, so it MUST run before the move below:
88125
// sizing a moved-from (empty) entry would undercount entries_est_ and split
89126
// blocks incorrectly. finish() output is unaffected either way -- it depends
90127
// only on the entries actually queued, not on how they were appended.
91-
entries_est_ += estimate_entry_bytes(entry, tier_);
128+
entries_est_ += estimate_entry_bytes(entry, tier_, external_inline_prx_length,
129+
external_inline_frq_length);
92130
entries_.push_back(std::move(entry));
93131
++n_entries_;
94132
}
@@ -147,6 +185,83 @@ void DictBlockBuilder::finish(ByteSink* sink) const {
147185
sink->put_fixed32(crc32c(body.view()));
148186
}
149187

188+
Status DictBlockBuilder::finish_streamed(
189+
std::span<const uint64_t> inline_prx_lengths,
190+
const std::function<Status(uint32_t, const std::function<Status(Slice)>&)>& inline_prx,
191+
const std::function<Status(Slice)>& append) const {
192+
DORIS_CHECK(inline_prx_lengths.size() == entries_.size());
193+
size_t next = 0;
194+
return finish_streamed_sequential(
195+
[&](Slice*, uint64_t* length) {
196+
*length = inline_prx_lengths[next++];
197+
return Status::OK();
198+
},
199+
inline_prx, append);
200+
}
201+
202+
Status DictBlockBuilder::finish_streamed_sequential(
203+
const std::function<Status(Slice*, uint64_t*)>& next_inline_fields,
204+
const std::function<Status(uint32_t, const std::function<Status(Slice)>&)>& inline_prx,
205+
const std::function<Status(Slice)>& append) const {
206+
uint64_t written = 0;
207+
uint32_t crc = 0;
208+
auto covered = [&](Slice bytes) {
209+
return append_dictionary_bytes(bytes, append, &crc, &written);
210+
};
211+
ByteSink sink;
212+
sink.put_varint64(n_entries_);
213+
sink.put_u8(kDictBlockFormatVer);
214+
sink.put_u8(has_positions_ ? dict_block_flags::kHasPositions : 0U);
215+
sink.put_varint64(frq_base_);
216+
if (has_positions_) {
217+
sink.put_varint64(prx_base_);
218+
}
219+
RETURN_IF_ERROR(covered(sink.view()));
220+
std::vector<uint32_t> anchors;
221+
anchors.reserve(n_anchors_);
222+
ByteSink body;
223+
std::string_view previous;
224+
for (uint32_t i = 0; i < n_entries_; ++i) {
225+
const bool anchor = is_anchor(i);
226+
if (anchor && written > UINT32_MAX) {
227+
return Status::Error<ErrorCode::INVALID_ARGUMENT, false>(
228+
"dict block: anchor offset exceeds uint32");
229+
}
230+
if (anchor) {
231+
anchors.push_back(static_cast<uint32_t>(written));
232+
}
233+
sink.clear();
234+
const DictEntry& entry = entries_[i];
235+
Slice inline_frq;
236+
uint64_t inline_prx_length = 0;
237+
RETURN_IF_ERROR(next_inline_fields(&inline_frq, &inline_prx_length));
238+
RETURN_IF_ERROR(encode_dict_entry(entry, anchor ? std::string_view {} : previous, tier_,
239+
&sink, &body, inline_frq));
240+
if (inline_prx_length == 0) {
241+
RETURN_IF_ERROR(covered(sink.view()));
242+
} else {
243+
DORIS_CHECK(entry.kind == DictEntryKind::kInline && tier_ >= IndexTier::kT2 &&
244+
entry.prx_bytes.empty());
245+
RETURN_IF_ERROR(append_external_inline_prx(
246+
sink.view(), inline_prx_length,
247+
[&](const std::function<Status(Slice)>& visitor) {
248+
return inline_prx(i, visitor);
249+
},
250+
covered));
251+
}
252+
previous = entry.term;
253+
}
254+
sink.clear();
255+
for (uint32_t offset : anchors) {
256+
sink.put_fixed32(offset);
257+
}
258+
sink.put_fixed32(static_cast<uint32_t>(anchors.size()));
259+
RETURN_IF_ERROR(covered(sink.view()));
260+
sink.clear();
261+
sink.put_fixed32(crc);
262+
return append(sink.view());
263+
}
264+
150265
std::vector<uint8_t> DictBlockBuilder::finish_owned() const {
151266
ByteSink block;
152267
encode_covered(&block);

‎be/src/storage/index/snii/format/dict_block.h‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
#include <cstdint>
2222
#include <functional>
2323
#include <optional>
24+
#include <span>
2425
#include <string>
2526
#include <string_view>
2627
#include <vector>
@@ -86,7 +87,8 @@ class DictBlockBuilder {
8687
// inline entry is two std::vector<uint8_t> heap allocations plus the term
8788
// copy -- on the SPIMI build path.
8889
void add_entry(const DictEntry& entry);
89-
void add_entry(DictEntry&& entry);
90+
void add_entry(DictEntry&& entry, uint64_t external_inline_prx_length = 0,
91+
uint64_t external_inline_frq_length = 0);
9092

9193
// Upper-bound estimate of the serialized size of the current block (including
9294
// header + entries + anchor table + CRC footer), used by the upper layer to
@@ -103,6 +105,22 @@ class DictBlockBuilder {
103105
// CRC-covered bytes when the caller needs ownership of the complete block.
104106
std::vector<uint8_t> finish_owned() const;
105107

108+
// Same block layout, with selected inline PRX blobs supplied by the caller.
109+
// Their entries contain empty prx_bytes; lengths[i]==0 uses the ordinary
110+
// entry. A callback appends each retained blob through the covered sink so
111+
// the anchor offsets and block CRC include its bytes without materializing it.
112+
Status finish_streamed(
113+
std::span<const uint64_t> inline_prx_lengths,
114+
const std::function<Status(uint32_t, const std::function<Status(Slice)>&)>& inline_prx,
115+
const std::function<Status(Slice)>& append) const;
116+
117+
// Reads bounded inline DD bytes and a PRX length per entry, in order, so the
118+
// writer can retain both payload fields in shared replayable streams.
119+
Status finish_streamed_sequential(
120+
const std::function<Status(Slice*, uint64_t*)>& next_inline_fields,
121+
const std::function<Status(uint32_t, const std::function<Status(Slice)>&)>& inline_prx,
122+
const std::function<Status(Slice)>& append) const;
123+
106124
private:
107125
bool is_anchor(uint32_t index) const { return index % anchor_interval_ == 0; }
108126
void encode_covered(ByteSink* sink) const;

‎be/src/storage/index/snii/format/dict_entry.cpp‎

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020
#include <algorithm>
2121
#include <atomic>
2222

23+
#include "common/check.h"
2324
#include "storage/index/snii/common/slice.h"
2425

2526
namespace doris::snii::format {
@@ -122,9 +123,10 @@ void write_pod_ref(const DictEntry& e, IndexTier tier, ByteSink* sink) {
122123
sink->put_varint64(e.prx_len);
123124
}
124125

125-
void write_inline(const DictEntry& e, IndexTier tier, ByteSink* sink) {
126-
sink->put_varint64(static_cast<uint64_t>(e.frq_bytes.size()));
127-
sink->put_bytes(Slice(e.frq_bytes));
126+
void write_inline(const DictEntry& e, IndexTier tier, ByteSink* sink, Slice external_frq) {
127+
const Slice frq = external_frq.empty() ? Slice(e.frq_bytes) : external_frq;
128+
sink->put_varint64(static_cast<uint64_t>(frq.size()));
129+
sink->put_bytes(frq);
128130
// INLINE bytes are covered by the dict block crc32c: omit the redundant
129131
// per-region crc.
130132
write_region_meta(e, /*store_crc=*/false, sink);
@@ -135,7 +137,8 @@ void write_inline(const DictEntry& e, IndexTier tier, ByteSink* sink) {
135137
sink->put_bytes(Slice(e.prx_bytes));
136138
}
137139

138-
void write_body(const DictEntry& e, std::string_view prev, IndexTier tier, ByteSink* sink) {
140+
void write_body(const DictEntry& e, std::string_view prev, IndexTier tier, ByteSink* sink,
141+
Slice external_inline_frq) {
139142
write_term_key(e, prev, sink);
140143
sink->put_u8(pack_flags(e));
141144
sink->put_varint32(e.df);
@@ -145,7 +148,7 @@ void write_body(const DictEntry& e, std::string_view prev, IndexTier tier, ByteS
145148
return;
146149
}
147150
if (e.kind == DictEntryKind::kInline) {
148-
write_inline(e, tier, sink);
151+
write_inline(e, tier, sink, external_inline_frq);
149152
} else {
150153
write_pod_ref(e, tier, sink);
151154
}
@@ -258,7 +261,7 @@ Status encode_dict_entry(const DictEntry& entry, std::string_view prev_term, Ind
258261
}
259262

260263
Status encode_dict_entry(const DictEntry& entry, std::string_view prev_term, IndexTier tier,
261-
ByteSink* sink, ByteSink* body_scratch) {
264+
ByteSink* sink, ByteSink* body_scratch, Slice external_inline_frq) {
262265
if (sink == nullptr || body_scratch == nullptr || sink == body_scratch) {
263266
return Status::Error<ErrorCode::INVALID_ARGUMENT, false>(
264267
"dict_entry: sink and body_scratch must be non-null and distinct");
@@ -270,7 +273,9 @@ Status encode_dict_entry(const DictEntry& entry, std::string_view prev_term, Ind
270273
// CRC is not repeated at the entry level, to keep slim/inline low-frequency
271274
// terms maximally compact (spec §DICT block/§dict entry).
272275
body_scratch->clear();
273-
write_body(entry, prev_term, tier, body_scratch);
276+
DORIS_CHECK(external_inline_frq.empty() ||
277+
(entry.kind == DictEntryKind::kInline && entry.frq_bytes.empty()));
278+
write_body(entry, prev_term, tier, body_scratch, external_inline_frq);
274279
sink->put_varint64(static_cast<uint64_t>(body_scratch->size()));
275280
sink->put_bytes(body_scratch->view());
276281
return Status::OK();

‎be/src/storage/index/snii/format/dict_entry.h‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,9 +108,10 @@ Status encode_dict_entry(const DictEntry& entry, std::string_view prev_term, Ind
108108

109109
// Same encoding with caller-owned body scratch. The scratch is cleared before
110110
// use while retaining capacity, allowing block builders to avoid one allocation
111-
// per dictionary entry. sink and body_scratch must be distinct objects.
111+
// per dictionary entry. sink and body_scratch must be distinct objects. A nonempty
112+
// external_inline_frq replaces an empty frq_bytes field during bounded replay.
112113
Status encode_dict_entry(const DictEntry& entry, std::string_view prev_term, IndexTier tier,
113-
ByteSink* sink, ByteSink* body_scratch);
114+
ByteSink* sink, ByteSink* body_scratch, Slice external_inline_frq = {});
114115

115116
// Decodes one entry from the current position of src; term is reconstructed
116117
// from prev_term + suffix. Verifies the trailing CRC; out-of-range / CRC

‎be/src/storage/index/snii/format/frq_prelude.cpp‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,11 @@ Status build_frq_prelude(const FrqPreludeColumns& cols, ByteSink* out) {
194194
return Status::OK();
195195
}
196196

197+
void encode_frq_window_row(const WindowMeta& window, bool has_prx, uint64_t previous_last,
198+
ByteSink* output) {
199+
encode_window_row(window, has_prx, previous_last, output);
200+
}
201+
197202
namespace {
198203

199204
// Decoded header fields shared between parse phases.

0 commit comments

Comments
 (0)