Repository navigation
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
/review |
There was a problem hiding this comment.
Request changes: the production refactor is sound, but the test-contract migration is incomplete and normal BE-UT configurations cannot pass as submitted.
Critical checkpoint conclusions:
- Goal and proof: Centralizing the cached single-row data column removes caller-side physical-shape checks and preserves production behavior, but the changed/new tests do not prove a passing cross-build contract; two blocking test failures are called out inline.
- Scope and clarity: The change is conceptually focused. The accessor migration covers all repository consumers; the related
VMatchPredicateremoval was verified as a no-op for valid plans because MATCH retains a nonconstant slot child. - Concurrency: The wrapper is immutable after construction and existing shared pointers are populated during expression open. No new thread, lock, atomic, dependency-order, or deadlock behavior was introduced; valid scan constant-false paths still signal readiness as before.
- Lifecycle, memory, and nullability: The wrapper owns the unwrapped
ColumnPtr;FunctionContextclones share that ownership, so cachedStringRefvalues remain live. Nullable and array/map/struct row-zero semantics and COW ownership are preserved, with no new untracked allocation or reference cycle. - Configuration and compatibility: No configuration, FE-BE variable, serialization, storage-format, or rolling-upgrade contract changes. The internal source API migration is complete across the repository.
- Parallel paths and special checks: File, mock, OLAP, generic scan, scalar-function, collection-IN, constant rebroadcast, and ANN paths were traced. The one-row check is a valid production invariant, but its direct test/mock callers and build-mode-specific failure behavior were not fully migrated.
- Test coverage and results: Existing scan-normalization cases now fail during setup, and the new unconditional death test fails under
NDEBUG. The visible macOS BE check is compile-only withMAKE_TEST=OFF, so it does not cover either issue. No local builds or tests were run because this review runner prohibits builds. - Observability: No new runtime path requires logs, metrics, or tracing.
- Transactions, persistence, and writes: Not applicable; the PR changes neither transaction state nor data-write/persistence behavior.
- Performance: Constant unwrapping remains no-copy and is performed once at wrapper construction; no hot-path regression or redundant allocation was found.
- Additional user focus: none was provided. The complete PR was reviewed without further narrowing.
Review completion: converged in Round 2 with two inline findings and no unresolved candidate.
| } else { | ||
| _column_ptr = std::move(column); | ||
| } | ||
| DORIS_CHECK_EQ(_column_ptr->size(), 1); |
There was a problem hiding this comment.
[P1] Update the existing two-row mock-constant tests
This invariant is reached immediately by the existing non-disabled ScanNormalizePredicate.test_eval_const_conjuncts3 and test_eval_const_conjuncts4: lines 130 and 154 pass two-row non-const columns through MockFnCall::set_const_expr_col(), which constructs this wrapper outside any death/exception assertion. The normal BE-UT CMake glob includes exec/*.cpp, so those tests now abort or throw before reaching their remaining code. (The green macOS check only builds BE with MAKE_TEST=OFF.) Please update/remove those malformed fixtures, or assert the new failure at construction, together with this invariant.
|
|
||
| TEST(ColumnPtrWrapperTest, RejectsNonConstMultiRowColumn) { | ||
| auto column = ColumnHelper::create_column<DataTypeInt64>({7, 8}); | ||
| EXPECT_DEATH(static_cast<void>(ColumnPtrWrapper(column)), ""); |
There was a problem hiding this comment.
[P1] Assert the release-mode exception path here
This only behaves as a death assertion while DCHECKs are enabled. With NDEBUG, DORIS_CHECK_EQ uses the release comparison macro and doris_check_fail() throws doris::Exception; GoogleTest does not count leaving a death-test statement by exception as death. The existing be/test/common/check_test.cpp handles this contract by capturing the exception under NDEBUG and using EXPECT_DEATH only under DCHECK_IS_ON(). Please mirror that split (or use a build-mode-neutral helper), otherwise release BE-UT fails this new test.
|
run buildall |
TPC-H: Total hot run time: 28896 ms |
TPC-DS: Total hot run time: 158807 ms |
ClickBench: Total hot run time: 23.84 s |
|
/review |
There was a problem hiding this comment.
Review opinion: no new blocking findings at 82d95715c456989a9987e5b383c96bf9ee31d509. The two earlier inline concerns are addressed at this head, and the change looks ready from a code-review perspective.
Critical checkpoint conclusions:
- Goal and proof: The change consistently stores cached constants as owned, single-row physical data columns, unwraps
ColumnConstwithout copying, and recreatesColumnConstonly when broadcasting. Productionget_const_col()executes for one row and validates size and declared type before caching. The added wrapper tests cover direct data, const unwrapping, and build-mode-specific invariant failure; the surviving scan tests cover both const-wrapped and direct one-row Boolean constants. - Scope and clarity: The refactor is focused. Repository-wide searches found no stale public-field access or missed
ColumnPtrWrapperproducer/consumer, and the mechanical accessor changes preserve the existing row-zero semantics. - Data correctness, nullability, and lifecycle:
ColumnConstdelegatesField,StringRef, null, and complex-value reads to nested row zero, so eager unwrapping preserves scalar, nullable, array/map/struct, JSON, regex, date/time, IN-set, and ANN behavior. The wrapper owns the nested column, and clonedFunctionContexts retain shared ownership; retained buffers remain live with no COW mutation, dangling reference, or ownership cycle. - Concurrency: Constant wrappers are populated at fragment open before thread-local cloning and are read-only afterward. Scan false/NULL normalization still sets EOS and wakes the scan dependency before scanner preparation. No new thread, lock, atomic, lock-order, blocking, or deadlock behavior is introduced.
- Error handling: Null and one-row wrapper preconditions fail loudly. Production expression size/type mismatches are returned as
Statusbefore wrapper construction, and existing status/exception boundaries remain intact. - Parallel paths and special checks: File, OLAP, Mock, and shared scan normalization paths, function-state consumers, constant rebroadcast, MATCH index/slow execution, and ANN TopN/range extraction were traced. Valid MATCH retains a slot child, so the removed root constant-cache call was a no-op; child constants and analyzer state are still initialized for original and cloned contexts. ANN keeps an owning nested Float32 pointer and preserves outer-null, empty, inner-null, array-shape, element-type, and dimension checks.
- Configuration and compatibility: No configuration, serialization, storage-format, FE-BE variable, external symbol, transaction, persistence, data-write, or rolling-upgrade contract changes apply.
- Test coverage and results: The two earlier test-contract findings are fixed at this head. Live
BE UT,COMPILE,Clang Formatter, and relevant regression checks are passing. No local build or test was run because this review runner explicitly prohibits builds. - Observability and performance: No new runtime path needs logs or metrics. Normalization is no-copy, removes repeated const-shape dispatch, and adds no hot-loop allocation or obvious CPU/memory regression.
- Additional user focus: No additional focus was provided; the complete PR was reviewed without narrowing.
Review completion: converged in Round 1. The main risk scan, two normal full-review partitions, and a separate risk-focused review completed; all returned no new valuable findings, every risk item has a concrete conclusion, and the final changed-file/unresolved-candidate sweep is clean. No new inline comments are needed.
### What problem does this PR solve? Issue Number: N/A Related PR: #66614 Problem Summary: Root cause: expression constness does not guarantee a physical `ColumnConst`. Cached constant expressions can return either a const wrapper or a single-row data column, while scan consumers sometimes cast the result directly to `ColumnConst`. Store an owned single-row data column in `ConstantExprResult`, unwrap `ColumnConst` without copying, and expose read-only accessors. Migrate scan, scalar-function, and ANN consumers, and recreate `ColumnConst` when broadcasting the cached value. Preserve current function initialization and NULL handling. ### Release note Handle constant expressions that return ordinary single-row columns consistently in scan predicate processing. ### Check List (For Author) - Test: - ASAN BE UT build passed. Ran 276 tests across 16 affected suites: 275 passed and the existing `MathFunctionTest.random_test` was skipped. Coverage includes zero-copy normalization, nullable values and ownership, arrays with nested NULLs, invalid multi-row input, broadcasting, scan predicates, function constant arguments, and ANN extraction. - clang-format 16, `git diff --check`, and BE build hygiene checks passed. - clang-tidy checked all 43 modified C++ files. New diagnostics were fixed and rechecked; existing complexity diagnostics remain on unchanged lines in column and LIKE tests. - Behavior changed: Yes. Scan consumers accept cached single-row constants regardless of their original physical representation; malformed multi-row cache inputs fail at construction. - Does this need documentation: No. No SQL syntax, configuration, or serialization changes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label
What problem does this PR solve?
Cached constant expressions could expose either a
ColumnConstor a single-row data column, forcing callers to inspect the physical column type. This change makesColumnPtrWrapperstore a private, single-row data column, unwrapsColumnConstwithout copying, and createsColumnConstonly when broadcasting the cached value. It also removes redundant constant evaluation fromVMatchPredicate::open().Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)