Skip to content

Commit 2824ba6

Browse files
committed
[improvement](be) Avoid redundant ASCII scans in translate
### What problem does this PR solve? Issue Number: N/A Related PR: #69223 Problem Summary: translate checks from/to ASCII status unconditionally even though the additional result is used only for two constant mappings. A non-ASCII source with a varying replacement column therefore scans the whole replacement column before using the same UTF-8 implementation. All-ASCII inputs also repeat mapping scans. Dispatch on mapping constness first and evaluate each needed ASCII predicate once. Non-ASCII constant mappings skip the source check, while varying mappings preserve source-first short-circuiting. The same conversion kernels are selected and SQL results remain unchanged. No performance benchmark was run. ### Release note Avoid redundant ASCII detection when evaluating translate with constant or varying mapping arguments. ### Check List (For Author) - Test: Passed build-support/check-format.sh with clang-format 16.0.6 and git diff --check. Attempted the existing mixed constant-map unit test using run-be-ut.sh -j 48 --run --filter=function_string_test.function_translate_mixed_const_map_test with ASAN. Build hygiene passed during configure, but CMake could not find OpenMP_C, so compilation and test execution did not run. - Behavior changed: No SQL semantic changes; reduces dispatch scanning. - Does this need documentation: No
1 parent 2d888fe commit 2824ba6

1 file changed

Lines changed: 16 additions & 13 deletions

File tree

‎be/src/exprs/function/function_string_misc.cpp‎

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -969,20 +969,23 @@ class FunctionTranslate : public IFunction {
969969
const auto* col_from = assert_cast<const ColumnString*>(argument_columns[1].get());
970970
const auto* col_to = assert_cast<const ColumnString*>(argument_columns[2].get());
971971

972-
bool is_ascii = col_source->is_ascii() && col_from->is_ascii() && col_to->is_ascii();
973-
bool from_to_ascii = col_from->is_ascii() && col_to->is_ascii();
974972
auto impl_vectors = impl_vectors_utf8<false>;
975-
if (col_const[1] && col_const[2] && is_ascii) {
976-
impl_vectors = impl_vectors_ascii<true>;
977-
} else if (col_const[1] && col_const[2] && from_to_ascii) {
978-
// from/to are ASCII constants; decide per source row so mostly-ASCII
979-
// source columns (e.g. URL with a few multi-byte chars) never fall into
980-
// the unordered_map-per-character path for ASCII rows.
981-
impl_vectors = impl_vectors_mixed<true>;
982-
} else if (col_const[1] && col_const[2]) {
983-
impl_vectors = impl_vectors_utf8<true>;
984-
} else if (is_ascii) {
985-
impl_vectors = impl_vectors_ascii<false>;
973+
if (col_const[1] && col_const[2]) {
974+
// Non-ASCII mappings require UTF-8 processing regardless of the source.
975+
const bool mapping_ascii = col_from->is_ascii() && col_to->is_ascii();
976+
if (mapping_ascii) {
977+
const bool source_ascii = col_source->is_ascii();
978+
impl_vectors = source_ascii ? impl_vectors_ascii<true> : impl_vectors_mixed<true>;
979+
} else {
980+
impl_vectors = impl_vectors_utf8<true>;
981+
}
982+
} else {
983+
// A non-ASCII source needs no mapping scans when the mappings vary by row.
984+
const bool all_ascii =
985+
col_source->is_ascii() && col_from->is_ascii() && col_to->is_ascii();
986+
if (all_ascii) {
987+
impl_vectors = impl_vectors_ascii<false>;
988+
}
986989
}
987990
impl_vectors(col_source, col_from, col_to, col_res.get());
988991
block.get_by_position(result).column = std::move(col_res);

0 commit comments

Comments
 (0)