Skip to content

Commit e39c032

Browse files
morningmanclaude
andcommitted
[opt](build) Prepare cutting the DataVariants instantiation amplifiers
Pure-additive / behavior-preserving preparation so that the hash-table variant machinery (exec/common/{agg,join,set,distinct_agg}_utils.h) can be cut out of exec/pipeline/dependency.h and rec_cte_shared_state.h next. Those two headers hold non-template in-class inline bodies (SharedState constructors/destructors/close paths, three std::visit dispatches) that reference the full variant types, and such bodies are semantically analyzed when the header is parsed: every one of the ~128 TUs that transitively include dependency.h instantiates the whole AggregatedDataVariants/JoinDataVariants/SetDataVariants surface at a flat ~0.85 CPU s per TU (~130 CPU s total), whether it uses them or not. No include edge is removed here: - dependency.h: move out of line into dependency.cpp everything whose body touches a variant -- AggSharedState ctor/dtor/_close_with_serialized_key/ _close_without_key, BucketedAggSharedState PerInstanceData ctor, dtor, _close and _close_one_agg_data, both HashJoinSharedState ctors, and a new user-declared SetSharedState ctor/dtor pair that takes over the make_unique<SetDataVariants>() default member initializer. All of it is per-query setup/teardown, so outlining costs nothing at runtime - dependency.h: forward-declare the variant structs, spell the members as plain std::unique_ptr/std::vector<size_t> instead of the utils-provided aliases, redeclare AggregateDataPtr, and directly include the light dependencies that used to ride in through the utils chain (core/arena.h, exprs/vexpr_fwd.h, util/stopwatch.hpp, cast_set/exception/ factory_creator, <list>/<queue>/<set>) - exec/common/join_op_utils.h (new): JoinOpVariants, is_asof_join*, and the AsofIndexGroup/AsofIndexVariant family move here out of join_utils.h, which re-exports it; dependency.h holds these by value, and this light header depends only on thrift enums, pdqsort and std containers -- not on the hash tables - rec_cte_shared_state.h: same treatment for the fourth amplifier; the emplace_block std::visit over DistinctDataVariants plus the brpc-heavy build_basic_param/send_data_to_targets bodies move to a new rec_cte_shared_state.cpp (CMake GLOB picks it up), with user-declared ctor/dtor so unique_ptr<DistinctDataVariants> works forward-declared - seed direct includes ahead of the cut: agg_utils.h into the three agg operator headers whose inline code walks method_variant, join_utils.h into join_build_sink_operator.h, distinct_agg_utils.h into rec_cte_source_operator.cpp, and the utils/template_helpers includes into dependency.cpp Verified with a full -fsyntax-only sweep: 0 failing TU(s) of 1365. Co-Authored-By: Claude Fable 5 <[email protected]>
1 parent f8b01ae commit e39c032

11 files changed

Lines changed: 490 additions & 346 deletions

‎be/src/exec/common/join_op_utils.h‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
// Licensed to the Apache Software Foundation (ASF) under one
2+
// or more contributor license agreements. See the NOTICE file
3+
// distributed with this work for additional information
4+
// regarding copyright ownership. The ASF licenses this file
5+
// to you under the Apache License, Version 2.0 (the
6+
// "License"); you may not use this file except in compliance
7+
// with the License. You may obtain a copy of the License at
8+
//
9+
// http://www.apache.org/licenses/LICENSE-2.0
10+
//
11+
// Unless required by applicable law or agreed to in writing,
12+
// software distributed under the License is distributed on an
13+
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
// KIND, either express or implied. See the License for the
15+
// specific language governing permissions and limitations
16+
// under the License.
17+
18+
#pragma once
19+
20+
// Lightweight join-op level types split out of join_utils.h so that
21+
// exec/pipeline/dependency.h (which holds JoinOpVariants / AsofIndexVariant
22+
// members by value) does not have to see the hash-table machinery.
23+
// Everything here depends only on thrift enums and std containers.
24+
25+
#include <gen_cpp/PlanNodes_types.h>
26+
#include <pdqsort.h>
27+
28+
#include <cstdint>
29+
#include <variant>
30+
#include <vector>
31+
32+
#include "common/compiler_util.h"
33+
34+
namespace doris {
35+
36+
using JoinOpVariants =
37+
std::variant<std::integral_constant<TJoinOp::type, TJoinOp::INNER_JOIN>,
38+
std::integral_constant<TJoinOp::type, TJoinOp::LEFT_SEMI_JOIN>,
39+
std::integral_constant<TJoinOp::type, TJoinOp::LEFT_ANTI_JOIN>,
40+
std::integral_constant<TJoinOp::type, TJoinOp::LEFT_OUTER_JOIN>,
41+
std::integral_constant<TJoinOp::type, TJoinOp::FULL_OUTER_JOIN>,
42+
std::integral_constant<TJoinOp::type, TJoinOp::RIGHT_OUTER_JOIN>,
43+
std::integral_constant<TJoinOp::type, TJoinOp::CROSS_JOIN>,
44+
std::integral_constant<TJoinOp::type, TJoinOp::RIGHT_SEMI_JOIN>,
45+
std::integral_constant<TJoinOp::type, TJoinOp::RIGHT_ANTI_JOIN>,
46+
std::integral_constant<TJoinOp::type, TJoinOp::NULL_AWARE_LEFT_ANTI_JOIN>,
47+
std::integral_constant<TJoinOp::type, TJoinOp::NULL_AWARE_LEFT_SEMI_JOIN>,
48+
std::integral_constant<TJoinOp::type, TJoinOp::ASOF_LEFT_INNER_JOIN>,
49+
std::integral_constant<TJoinOp::type, TJoinOp::ASOF_LEFT_OUTER_JOIN>>;
50+
51+
inline bool is_asof_join(TJoinOp::type join_op) {
52+
return join_op == TJoinOp::ASOF_LEFT_INNER_JOIN || join_op == TJoinOp::ASOF_LEFT_OUTER_JOIN;
53+
}
54+
55+
template <int JoinOpType>
56+
inline constexpr bool is_asof_join_op_v =
57+
JoinOpType == TJoinOp::ASOF_LEFT_INNER_JOIN || JoinOpType == TJoinOp::ASOF_LEFT_OUTER_JOIN;
58+
59+
template <int JoinOpType>
60+
inline constexpr bool is_asof_outer_join_op_v = JoinOpType == TJoinOp::ASOF_LEFT_OUTER_JOIN;
61+
62+
// ASOF JOIN index with inline values for cache-friendly branchless binary search.
63+
// IntType is the integer representation of the ASOF column value:
64+
// uint32_t for DateV2, uint64_t for DateTimeV2 and TimestampTZ.
65+
// Rows are sorted by asof_value during build, then materialized into SoA arrays
66+
// so probe-side binary search only touches the ASOF values hot path.
67+
template <typename IntType>
68+
struct AsofIndexGroup {
69+
using int_type = IntType;
70+
71+
struct Entry {
72+
IntType asof_value;
73+
uint32_t row_index; // 1-based, 0 = invalid/padding
74+
};
75+
76+
std::vector<Entry> entries;
77+
std::vector<IntType> asof_values;
78+
std::vector<uint32_t> row_indexes;
79+
80+
void add_row(IntType value, uint32_t row_idx) { entries.push_back({value, row_idx}); }
81+
82+
void sort_and_finalize() {
83+
if (entries.empty()) {
84+
return;
85+
}
86+
if (entries.size() > 1) {
87+
pdqsort(entries.begin(), entries.end(),
88+
[](const Entry& a, const Entry& b) { return a.asof_value < b.asof_value; });
89+
}
90+
91+
asof_values.resize(entries.size());
92+
row_indexes.resize(entries.size());
93+
for (size_t i = 0; i < entries.size(); ++i) {
94+
asof_values[i] = entries[i].asof_value;
95+
row_indexes[i] = entries[i].row_index;
96+
}
97+
98+
std::vector<Entry>().swap(entries);
99+
}
100+
101+
const IntType* values_data() const { return asof_values.data(); }
102+
103+
// Branchless lower_bound: first i where asof_values[i] >= target
104+
ALWAYS_INLINE size_t lower_bound(IntType target) const {
105+
size_t lo = 0, n = asof_values.size();
106+
while (n > 1) {
107+
size_t half = n / 2;
108+
lo += half * (asof_values[lo + half] < target);
109+
n -= half;
110+
}
111+
if (lo < asof_values.size()) {
112+
lo += (asof_values[lo] < target);
113+
}
114+
return lo;
115+
}
116+
117+
// Branchless upper_bound: first i where asof_values[i] > target
118+
ALWAYS_INLINE size_t upper_bound(IntType target) const {
119+
size_t lo = 0, n = asof_values.size();
120+
while (n > 1) {
121+
size_t half = n / 2;
122+
lo += half * (asof_values[lo + half] <= target);
123+
n -= half;
124+
}
125+
if (lo < asof_values.size()) {
126+
lo += (asof_values[lo] <= target);
127+
}
128+
return lo;
129+
}
130+
131+
// Semantics by (is_greater, is_strict):
132+
// (true, false): probe >= build -> find largest build value <= probe
133+
// (true, true): probe > build -> find largest build value < probe
134+
// (false, false): probe <= build -> find smallest build value >= probe
135+
// (false, true): probe < build -> find smallest build value > probe
136+
// Returns the build row index of the best match, or 0 if no match.
137+
template <bool IsGreater, bool IsStrict>
138+
ALWAYS_INLINE uint32_t find_best_match(IntType probe_value) const {
139+
if (asof_values.empty()) {
140+
return 0;
141+
}
142+
if constexpr (IsGreater) {
143+
size_t pos = IsStrict ? lower_bound(probe_value) : upper_bound(probe_value);
144+
return pos > 0 ? row_indexes[pos - 1] : 0;
145+
} else {
146+
size_t pos = IsStrict ? upper_bound(probe_value) : lower_bound(probe_value);
147+
return pos < asof_values.size() ? row_indexes[pos] : 0;
148+
}
149+
}
150+
};
151+
152+
// Type-erased container for all ASOF index groups.
153+
// DateV2 -> uint32_t, DateTimeV2/TimestampTZ -> uint64_t.
154+
using AsofIndexVariant = std::variant<std::monostate, std::vector<AsofIndexGroup<uint32_t>>,
155+
std::vector<AsofIndexGroup<uint64_t>>>;
156+
157+
} // namespace doris

‎be/src/exec/common/join_utils.h‎

Lines changed: 1 addition & 121 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
#include "exec/common/hash_table/hash_key_type.h"
2525
#include "exec/common/hash_table/hash_map_context.h"
2626
#include "exec/common/hash_table/join_hash_table.h"
27+
#include "exec/common/join_op_utils.h" // IWYU pragma: export (JoinOpVariants/AsofIndexGroup moved there)
2728

2829
namespace doris {
2930

@@ -45,32 +46,6 @@ decltype(auto) asof_column_dispatch(const IColumn* col, Func&& func) {
4546
return std::forward<Func>(func)(col);
4647
}
4748
}
48-
using JoinOpVariants =
49-
std::variant<std::integral_constant<TJoinOp::type, TJoinOp::INNER_JOIN>,
50-
std::integral_constant<TJoinOp::type, TJoinOp::LEFT_SEMI_JOIN>,
51-
std::integral_constant<TJoinOp::type, TJoinOp::LEFT_ANTI_JOIN>,
52-
std::integral_constant<TJoinOp::type, TJoinOp::LEFT_OUTER_JOIN>,
53-
std::integral_constant<TJoinOp::type, TJoinOp::FULL_OUTER_JOIN>,
54-
std::integral_constant<TJoinOp::type, TJoinOp::RIGHT_OUTER_JOIN>,
55-
std::integral_constant<TJoinOp::type, TJoinOp::CROSS_JOIN>,
56-
std::integral_constant<TJoinOp::type, TJoinOp::RIGHT_SEMI_JOIN>,
57-
std::integral_constant<TJoinOp::type, TJoinOp::RIGHT_ANTI_JOIN>,
58-
std::integral_constant<TJoinOp::type, TJoinOp::NULL_AWARE_LEFT_ANTI_JOIN>,
59-
std::integral_constant<TJoinOp::type, TJoinOp::NULL_AWARE_LEFT_SEMI_JOIN>,
60-
std::integral_constant<TJoinOp::type, TJoinOp::ASOF_LEFT_INNER_JOIN>,
61-
std::integral_constant<TJoinOp::type, TJoinOp::ASOF_LEFT_OUTER_JOIN>>;
62-
63-
inline bool is_asof_join(TJoinOp::type join_op) {
64-
return join_op == TJoinOp::ASOF_LEFT_INNER_JOIN || join_op == TJoinOp::ASOF_LEFT_OUTER_JOIN;
65-
}
66-
67-
template <int JoinOpType>
68-
inline constexpr bool is_asof_join_op_v =
69-
JoinOpType == TJoinOp::ASOF_LEFT_INNER_JOIN || JoinOpType == TJoinOp::ASOF_LEFT_OUTER_JOIN;
70-
71-
template <int JoinOpType>
72-
inline constexpr bool is_asof_outer_join_op_v = JoinOpType == TJoinOp::ASOF_LEFT_OUTER_JOIN;
73-
7449
template <class T>
7550
using PrimaryTypeHashTableContext = MethodOneNumber<T, JoinHashMap<T, HashCRC32Return32<T>, false>>;
7651

@@ -232,99 +207,4 @@ inline void try_convert_to_direct_mapping(
232207
primary_to_direct_mapping(context, key_columns, variant_ptrs);
233208
}
234209

235-
// ASOF JOIN index with inline values for cache-friendly branchless binary search.
236-
// IntType is the integer representation of the ASOF column value:
237-
// uint32_t for DateV2, uint64_t for DateTimeV2 and TimestampTZ.
238-
// Rows are sorted by asof_value during build, then materialized into SoA arrays
239-
// so probe-side binary search only touches the ASOF values hot path.
240-
template <typename IntType>
241-
struct AsofIndexGroup {
242-
using int_type = IntType;
243-
244-
struct Entry {
245-
IntType asof_value;
246-
uint32_t row_index; // 1-based, 0 = invalid/padding
247-
};
248-
249-
std::vector<Entry> entries;
250-
std::vector<IntType> asof_values;
251-
std::vector<uint32_t> row_indexes;
252-
253-
void add_row(IntType value, uint32_t row_idx) { entries.push_back({value, row_idx}); }
254-
255-
void sort_and_finalize() {
256-
if (entries.empty()) {
257-
return;
258-
}
259-
if (entries.size() > 1) {
260-
pdqsort(entries.begin(), entries.end(),
261-
[](const Entry& a, const Entry& b) { return a.asof_value < b.asof_value; });
262-
}
263-
264-
asof_values.resize(entries.size());
265-
row_indexes.resize(entries.size());
266-
for (size_t i = 0; i < entries.size(); ++i) {
267-
asof_values[i] = entries[i].asof_value;
268-
row_indexes[i] = entries[i].row_index;
269-
}
270-
271-
std::vector<Entry>().swap(entries);
272-
}
273-
274-
const IntType* values_data() const { return asof_values.data(); }
275-
276-
// Branchless lower_bound: first i where asof_values[i] >= target
277-
ALWAYS_INLINE size_t lower_bound(IntType target) const {
278-
size_t lo = 0, n = asof_values.size();
279-
while (n > 1) {
280-
size_t half = n / 2;
281-
lo += half * (asof_values[lo + half] < target);
282-
n -= half;
283-
}
284-
if (lo < asof_values.size()) {
285-
lo += (asof_values[lo] < target);
286-
}
287-
return lo;
288-
}
289-
290-
// Branchless upper_bound: first i where asof_values[i] > target
291-
ALWAYS_INLINE size_t upper_bound(IntType target) const {
292-
size_t lo = 0, n = asof_values.size();
293-
while (n > 1) {
294-
size_t half = n / 2;
295-
lo += half * (asof_values[lo + half] <= target);
296-
n -= half;
297-
}
298-
if (lo < asof_values.size()) {
299-
lo += (asof_values[lo] <= target);
300-
}
301-
return lo;
302-
}
303-
304-
// Semantics by (is_greater, is_strict):
305-
// (true, false): probe >= build -> find largest build value <= probe
306-
// (true, true): probe > build -> find largest build value < probe
307-
// (false, false): probe <= build -> find smallest build value >= probe
308-
// (false, true): probe < build -> find smallest build value > probe
309-
// Returns the build row index of the best match, or 0 if no match.
310-
template <bool IsGreater, bool IsStrict>
311-
ALWAYS_INLINE uint32_t find_best_match(IntType probe_value) const {
312-
if (asof_values.empty()) {
313-
return 0;
314-
}
315-
if constexpr (IsGreater) {
316-
size_t pos = IsStrict ? lower_bound(probe_value) : upper_bound(probe_value);
317-
return pos > 0 ? row_indexes[pos - 1] : 0;
318-
} else {
319-
size_t pos = IsStrict ? upper_bound(probe_value) : lower_bound(probe_value);
320-
return pos < asof_values.size() ? row_indexes[pos] : 0;
321-
}
322-
}
323-
};
324-
325-
// Type-erased container for all ASOF index groups.
326-
// DateV2 -> uint32_t, DateTimeV2/TimestampTZ -> uint64_t.
327-
using AsofIndexVariant = std::variant<std::monostate, std::vector<AsofIndexGroup<uint32_t>>,
328-
std::vector<AsofIndexGroup<uint64_t>>>;
329-
330210
} // namespace doris

‎be/src/exec/operator/aggregation_sink_operator.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919

2020
#include <stdint.h>
2121

22+
#include "exec/common/agg_utils.h"
2223
#include "exec/operator/operator.h"
2324
#include "runtime/exec_env.h"
2425
#include "runtime/runtime_profile.h"

‎be/src/exec/operator/bucketed_aggregation_sink_operator.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919

2020
#include <stdint.h>
2121

22+
#include "exec/common/agg_utils.h"
2223
#include "exec/operator/operator.h"
2324
#include "runtime/exec_env.h"
2425
#include "runtime/runtime_profile.h"

‎be/src/exec/operator/join_build_sink_operator.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717

1818
#pragma once
1919

20+
#include "exec/common/join_utils.h"
2021
#include "exec/operator/operator.h"
2122

2223
namespace doris {

‎be/src/exec/operator/rec_cte_source_operator.cpp‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@
1717

1818
#include "exec/operator/rec_cte_source_operator.h"
1919

20+
#include "exec/common/distinct_agg_utils.h"
21+
2022
namespace doris {
2123

2224
RecCTESourceLocalState::RecCTESourceLocalState(RuntimeState* state, OperatorXBase* parent)

‎be/src/exec/operator/streaming_aggregation_operator.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323

2424
#include "common/status.h"
2525
#include "core/block/block.h"
26+
#include "exec/common/agg_utils.h"
2627
#include "exec/operator/operator.h"
2728
#include "runtime/runtime_profile.h"
2829

0 commit comments

Comments
 (0)