Skip to content

Commit f8b01ae

Browse files
morningmanclaude
andcommitted
[opt](build) Repair the BE UT build after the include-edge cuts
The four cut-edge commits on this branch were validated with a -fsyntax-only sweep over be/src only, so the 1024 be/test TUs were never compiled against the slimmed graph. A full `ninja -k 0` run (2422 targets) surfaced 8 failing TUs in 4 families. Nothing here restores a cut edge, and no production header gains an include; the RLE fix in fact tightens the invariant decoder.h already claimed. - core/wide_integer_from_double.cpp: -Werror,-Wunused-macros. Where the 80-bit long double exists, wide_integer_impl.h takes its `#if (LDBL_MANT_DIG == 64)` branch and never evaluates the `#elif defined(...)` that reads DORIS_WIDE_INTEGER_FROM_DOUBLE_IMPL_TU, so the definition looked unused. The existing instantiation guard now reads the macro itself. `defined(...)` must stay the left operand -- the preprocessor short-circuits `&&`, so a false left term skips it and the warning returns. - format/parquet/decoder.h and format_v2/parquet/reader/native/decoder.h: outline BaseDictDecoder's default ctor next to the dtor that was already outlined. A defaulted-in-class ctor is defined in every TU that constructs a derived decoder and it odr-uses ~unique_ptr<RleBatchDecoder<uint32_t>>, so the header's own comment -- that the complete type is only needed in decoder.cpp -- did not hold. Both trees had the defect; only the format/ one had a test reaching it. - block_test / column_variant_v2_test: declare the RuntimeProfile and HybridSorter they use directly. They had been riding pod_array.h -> thread_context.h and column.h -> exec/sort/hybrid_sorter.h. - asof_join_test and the two fix_length_dict_decoder tests: include-order- sensitive, so the cost stays in the tests rather than in a production header. AsofIndexGroup::sort_and_finalize calls the global pdqsort, which ADL cannot reach from std::vector's iterators; FixLengthDictDecoder::_decode_values dereferences a unique_ptr<RleBatchDecoder<uint32_t>> that depends on no template parameter. Both bind where the template is parsed, not where it is instantiated, so the supplying include has to come first. The two decoder tests carry clang-format off/on because the formatter would otherwise sort the include back after the header under test and break the build again. Verified: `sh run-be-ut.sh` links test/doris_be_test with 0 failures; all 10 files pass clang-format --dry-run --Werror. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01V3BVeKrPS3jQ2bagcuGHBQ
1 parent 624d039 commit f8b01ae

10 files changed

Lines changed: 47 additions & 5 deletions

File tree

‎be/src/core/wide_integer_from_double.cpp‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,14 @@
2626

2727
#include "core/wide_integer.h"
2828

29-
#if !(LDBL_MANT_DIG == 64)
29+
// The `defined(...)` term below is what marks the macro above as used, and it has to stay the
30+
// left operand: the preprocessor short-circuits `&&`, so a false left term would skip it and the
31+
// definition would read as an unused macro (-Werror,-Wunused-macros). That happens on exactly the
32+
// platforms this file exists for the inverse of -- where the 80-bit long double exists,
33+
// wide_integer_impl.h takes its `#if (LDBL_MANT_DIG == 64)` branch and never evaluates the
34+
// `#elif defined(...)` that would otherwise consume the macro. Reading it here is also honest:
35+
// these instantiations exist precisely because this TU is the impl TU.
36+
#if defined(DORIS_WIDE_INTEGER_FROM_DOUBLE_IMPL_TU) && !(LDBL_MANT_DIG == 64)
3037

3138
// Every wide::integer specialization used by Doris (see core/extended_types.h)
3239
// gets its from-double conversion emitted here. A new specialization that is

‎be/src/format/parquet/decoder.cpp‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,8 @@
3333

3434
namespace doris {
3535

36+
BaseDictDecoder::BaseDictDecoder() = default;
37+
3638
BaseDictDecoder::~BaseDictDecoder() = default;
3739

3840
Status BaseDictDecoder::set_data(Slice* data) {

‎be/src/format/parquet/decoder.h‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -94,10 +94,12 @@ class Decoder {
9494

9595
class BaseDictDecoder : public Decoder {
9696
public:
97-
BaseDictDecoder() = default;
9897
// Out-of-line: member unique_ptr<RleBatchDecoder<uint32_t>> only needs the
99-
// complete type where the dtor/set_data/skip_values are defined (decoder.cpp),
98+
// complete type where the ctor/dtor/set_data/skip_values are defined (decoder.cpp),
10099
// keeping the costly RLE template machinery out of every includer of this header.
100+
// The ctor counts: a defaulted-in-class one would be defined in every TU that
101+
// constructs a derived decoder, and it odr-uses the member's destructor.
102+
BaseDictDecoder();
101103
~BaseDictDecoder() override;
102104

103105
// Set the data to be decoded

‎be/src/format_v2/parquet/reader/native/decoder.cpp‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,8 @@
3333

3434
namespace doris::format::parquet::native {
3535

36+
BaseDictDecoder::BaseDictDecoder() = default;
37+
3638
BaseDictDecoder::~BaseDictDecoder() = default;
3739

3840
Status BaseDictDecoder::set_data(Slice* data) {

‎be/src/format_v2/parquet/reader/native/decoder.h‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -131,11 +131,13 @@ class Decoder : public ParquetDecodeSource {
131131

132132
class BaseDictDecoder : public Decoder {
133133
public:
134-
BaseDictDecoder() = default;
135134
// Out-of-line together with every member that dereferences _index_batch_decoder:
136135
// keeping the RleBatchDecoder<uint32_t> machinery out of this header stops every
137136
// includer from instantiating the RLE decode chain; the complete type lives only
138-
// in decoder.cpp. All of these are per-page/per-batch virtual calls.
137+
// in decoder.cpp. All of these are per-page/per-batch virtual calls. The ctor is
138+
// out of line for the same reason: a defaulted-in-class one would be defined in
139+
// every TU that constructs a derived decoder, and it odr-uses the member's dtor.
140+
BaseDictDecoder();
139141
~BaseDictDecoder() override;
140142

141143
// Set the data to be decoded

‎be/test/core/block/block_test.cpp‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@
6565
#include "exec/common/sip_hash.h"
6666
#include "runtime/descriptor_helper.h"
6767
#include "runtime/descriptors.h"
68+
#include "runtime/runtime_profile.h"
6869
#include "testutil/column_helper.h"
6970
#include "util/debug_points.h"
7071
#include "util/defer_op.h"

‎be/test/core/column/column_variant_v2_test.cpp‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@
6161
#include "exec/common/hash_table/hash_map_context.h"
6262
#include "exec/common/hash_table/string_hash_map.h"
6363
#include "exec/common/sip_hash.h"
64+
#include "exec/sort/hybrid_sorter.h"
6465
#include "exprs/function/parse/variant_jsonb_parse.h"
6566
#include "exprs/function/parse/variant_string_parse.h"
6667
#include "util/jsonb_writer.h"

‎be/test/exec/operator/asof_join_test.cpp‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,13 @@
1616
// under the License.
1717

1818
#include <gtest/gtest.h>
19+
// Must precede exec/common/join_utils.h below. AsofIndexGroup::sort_and_finalize calls the
20+
// global pdqsort, which ADL cannot reach from std::vector's iterators, so the name has to be
21+
// visible where the template is defined rather than where it is instantiated. join_utils.h
22+
// deliberately does not include it: core/column/column.h stopped pulling exec/sort/hybrid_sorter.h
23+
// (pdqsort + timsort) out of 808 TUs, and production TUs reach join_utils.h through headers that
24+
// already supply pdqsort. Only this test pays for it.
25+
#include <pdqsort.h>
1926

2027
#include <initializer_list>
2128
#include <type_traits>

‎be/test/format/parquet/fix_length_dict_decoder_empty_dict_test.cpp‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,16 @@
1818
#include <gtest/gtest.h>
1919

2020
#include "core/column/column_vector.h"
21+
// Must precede the header below. FixLengthDictDecoder::_decode_values dereferences the inherited
22+
// unique_ptr<RleBatchDecoder<uint32_t>>, and that type depends on no template parameter, so the
23+
// call is bound when the template is parsed rather than when it is instantiated. decoder.h only
24+
// forward-declares RleBatchDecoder to keep the RLE machinery out of its ~530 includers, so this TU
25+
// supplies the complete type up front. Only this test pays for it.
26+
// clang-format off
27+
#include "util/rle_encoding.h" // IWYU pragma: keep
28+
2129
#include "format/parquet/fix_length_dict_decoder.hpp"
30+
// clang-format on
2231

2332
namespace doris {
2433

‎be/test/format/parquet/fix_length_dict_decoder_test.cpp‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,16 @@
1515
// specific language governing permissions and limitations
1616
// under the License.
1717

18+
// Must precede the header under test. FixLengthDictDecoder::_decode_values dereferences the
19+
// inherited unique_ptr<RleBatchDecoder<uint32_t>>, and that type depends on no template parameter,
20+
// so the call is bound when the template is parsed rather than when it is instantiated. decoder.h
21+
// only forward-declares RleBatchDecoder to keep the RLE machinery out of its ~530 includers, so
22+
// this TU supplies the complete type up front. Only this test pays for it.
23+
// clang-format off
24+
#include "util/rle_encoding.h" // IWYU pragma: keep
25+
1826
#include "format/parquet/fix_length_dict_decoder.hpp"
27+
// clang-format on
1928

2029
#include <gtest/gtest.h>
2130

0 commit comments

Comments
 (0)