Skip to content

Commit 5d53eb3

Browse files
csun5285claude
andcommitted
[refactor](be) Replace PrimitiveTypeConvertor with StorageLayout keyed by FieldType
### What problem does this PR solve? Issue Number: None Related PR: #68608, #68609 Problem Summary: How a compute-layer value becomes the value a segment stores is written down in several places. PrimitiveTypeConvertor<PT> gives the V1 DATE, DATETIME and DECIMAL layouts to the key encodings of RowCursor and the BKD index; KeyCoderTraitsForFloat canonicalises NaN on its own; and the comparison and IN-list predicates repeat the V1 conversions by hand to probe a bloom filter. The key-encodable types are listed as (FieldType, PrimitiveType) pairs so that the key helpers can be keyed by PrimitiveType. Add storage/storage_layout.h. For every fixed-width FieldType, StorageLayout<FT> gives the StorageValue a segment stores, the PrimitiveValue the compute layer holds, and to_storage from one to the other: a bit cast, the quiet NaN for every NaN of FLOAT/DOUBLE, and the V1 packing of DATE, DATETIME and DECIMAL. Make the PrimitiveType <-> FieldType mapping in storage_field_type.h constexpr, so that a template picks its layout with StorageLayout<primitive_type_to_storage_field_type(PT)>; data_type.h now declares FieldType itself instead of including that header. Move the users onto it: full_encode_field_as_key and so the RowCursor and BKD query key encodings, the BKD +/- infinity sentinels, the NaN handling of KeyCoderTraitsForFloat, and the bloom filter probes of the comparison and IN-list predicates. The key-encodable types are listed by FieldType alone. Delete PrimitiveTypeConvertor, whose last user was a test helper. Every conversion is the one it replaces, so no stored or encoded byte changes. The one difference is that a bloom filter probe now canonicalises a NaN, and no bloom filter index is built on FLOAT or DOUBLE. This is the second piece split out of #68608; the page encoders and decoders move onto StorageLayout in later pieces. ### Release note None ### Check List (For Author) - Test: Unit Test - StorageLayoutTest (new): the StorageValue size of every fixed-width FieldType, the V1 DATE, DATETIME and DECIMAL packing, NaN canonicalisation and bit casts - KeyCoderTest (key order of full_encode_field_as_key for every key type), TestRowCursor, StorageFieldTypeTest, DateBloomFilterTest, BloomFilterIndexReaderWriterTest, BlockColumnPredicateTest, PredicateCreatorTest, InvertedIndexReaderTest (BKD range queries of every key type), SniiBkd*, and SegmentFlusherFormatTest and SegmentFlusherTransformFormatTest with strict golden comparison - Behavior changed: No - Does this need documentation: No Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
1 parent 7dc90d1 commit 5d53eb3

16 files changed

Lines changed: 596 additions & 454 deletions

‎be/src/core/data_type/data_type.h‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,11 +38,11 @@
3838
#include "core/cow.h"
3939
#include "core/data_type/define_primitive_type.h"
4040
#include "core/data_type/primitive_type.h"
41-
#include "core/data_type/storage_field_type.h"
4241
#include "core/data_type_serde/data_type_serde.h"
4342
#include "core/types.h"
4443

4544
namespace doris {
45+
enum class FieldType;
4646
class PColumnMeta;
4747
class PTypeDesc;
4848
class PTypeNode;

‎be/src/core/data_type/primitive_type.h‎

Lines changed: 0 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -555,50 +555,6 @@ struct PrimitiveTypeTraits<TYPE_UINT64> {
555555
using ColumnType = ColumnOffset64;
556556
};
557557

558-
template <PrimitiveType PT>
559-
struct PrimitiveTypeConvertor {
560-
using CppType = typename PrimitiveTypeTraits<PT>::CppType;
561-
using StorageFieldType = typename PrimitiveTypeTraits<PT>::StorageFieldType;
562-
563-
static inline StorageFieldType&& to_storage_field_type(CppType&& value) {
564-
return static_cast<StorageFieldType&&>(std::forward<CppType>(value));
565-
}
566-
567-
static inline const StorageFieldType& to_storage_field_type(const CppType& value) {
568-
return *reinterpret_cast<const StorageFieldType*>(&value);
569-
}
570-
};
571-
572-
template <>
573-
struct PrimitiveTypeConvertor<TYPE_DATE> {
574-
using CppType = typename PrimitiveTypeTraits<TYPE_DATE>::CppType;
575-
using StorageFieldType = typename PrimitiveTypeTraits<TYPE_DATE>::StorageFieldType;
576-
577-
static inline StorageFieldType to_storage_field_type(const CppType& value) {
578-
return StorageFieldType(cast_set<uint32_t>(value.to_olap_date()));
579-
}
580-
};
581-
582-
template <>
583-
struct PrimitiveTypeConvertor<TYPE_DATETIME> {
584-
using CppType = typename PrimitiveTypeTraits<TYPE_DATETIME>::CppType;
585-
using StorageFieldType = typename PrimitiveTypeTraits<TYPE_DATETIME>::StorageFieldType;
586-
587-
static inline StorageFieldType to_storage_field_type(const CppType& value) {
588-
return value.to_olap_datetime();
589-
}
590-
};
591-
592-
template <>
593-
struct PrimitiveTypeConvertor<TYPE_DECIMALV2> {
594-
using CppType = typename PrimitiveTypeTraits<TYPE_DECIMALV2>::CppType;
595-
using StorageFieldType = typename PrimitiveTypeTraits<TYPE_DECIMALV2>::StorageFieldType;
596-
597-
static inline StorageFieldType to_storage_field_type(const CppType& value) {
598-
return {value.int_value(), value.frac_value()};
599-
}
600-
};
601-
602558
inline TTypeDesc create_type_desc(PrimitiveType type, int precision = 0, int scale = 0) {
603559
TTypeDesc type_desc;
604560
std::vector<TTypeNode> node_type;

‎be/src/core/data_type/storage_field_type.cpp‎

Lines changed: 0 additions & 210 deletions
This file was deleted.

0 commit comments

Comments
 (0)