Repository navigation
update - #9
Merged
Merged
update#9
Conversation
In this CL, I move the entry's deleter out of LRUCache's mutex block, which can let others access this cache without waiting free cache entry.
In this change, channel close is finished into two phases. So we can close channels parallel, which can make query faster.
In Doris, one block has 1024 rows. 1. If the previous ScanKey scan rows multiple blocks, and also the final block has 1024 rows just right. 2. The current ScanKey scan rows with number less than one block. Under the two conditions, if not seek block, the position of prefix shortkey columns is wrong.
Reproduce: 1. start a routine load, send a routine load task to BE 2. BE executes task successfully and commit to FE. 3. Commit request failed on FE because database is renamed(throw db not found exception) 4. After commit failed, BE will send rollback request to FE. 5. FE receive this rollback request and mistakenly update the routine load progress, because the number of loaded rows in this rollback request's attachment is larger than 0
Now size of HyperLogLog struct is so large that it lead the rowset is too small when ingesting data. In this CL, registers in HyperLogLog are only created when it is needed. When ingesting data, it's normal case that there are only few values in one HyperLogLog.
The function named assign conjuncts has been invoked before creating aggregation plan node. If the empty set node is the child of aggregation node, the conjuncts will be assign to empty set node which could not be executed correctly in Backend. It will thrown the exception "couldn't resolve slot descriptor" for query which has both empty set node and aggregation node. For example: select sum(pv) from test where type != 1 and 1=0 group by type; This commit fix this bug. It remove conjuncts for empty set node.
StringLiteral could be cast to VARCHAR or CHAR. The default value of lead and lag function could be 'String' when the column type is CHAR or VARCHAR.
The current load process is: Tablet Sink -> Tablet Channel Mgr -> Tablets Channel -> Delta Writer -> MemTable -> Flush to disk In the path of Tablets Channel -> DeltaWriter -> MemTable -> Flush to disk, the following operations are performed: Insert tuple into different memtables according to tablet ID When the memtable size reaches the threshold, it is written to disk. The above operations are equivalent to single thread execution for a single load task. In fact, the insertion of memtable and the flush of memtable can be executed synchronously. Perform these operation in single thread prevents the insertion of memtable from being delayed due to slow disk writing. In the new implementation, I added a MemTableFlushExecutor class with a set of flush queues and corresponding worker threads. By default, each data directory uses two worker threads for flush, which can be modified by the parameter flush_thread_num_per_store of BE. DeltaWriter will push the full memtable to MemTableFlushExecutor for flush operation and generate a new memtable for receiving new data. This design can improve the performance of load large files. In single host testing, the time to load a 1GB text file is reduced from 48 seconds to 29 seconds.
…compaction (#1866) In previous compaction, only rowsets will be taken into consideration. Doing streaming load, the singleton rowset may is made up of many overlapping segments. Scanning these overlapping segments will result in read amplification. To address this problem, overlapping segments should be taken into consideration when doing cumulative compaction to reduce read amplification.
hll_hash should support the inline function in broker load and should not support the inline function in hadoop load.
Each load job has several load tasks, and each task is a query plan with serveral plan fragments. Each plan fragment report query profile independently. So we need to collect each plan fragment's report, separately.
xy720
pushed a commit
that referenced
this pull request
Feb 21, 2025
WRITE of size 1 at 0x6160007e86f0 thread T1983 (Pipe_normal [wo) #0 0x55fc8065b975 in std::__atomic_base<bool>::store(bool, std::memory_order) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:457:2 #1 0x55fc8065b975 in std::__atomic_base<bool>::operator=(bool) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:349:2 #2 0x55fc8065b975 in std::atomic<bool>::operator=(bool) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:80:22 #3 0x55fc8065b975 in doris::pipeline::PipelineTask::set_running(bool) /root/doris/be/src/pipeline/pipeline_task.h:192:47 #4 0x55fc8065b975 in doris::pipeline::TaskScheduler::_do_work(int)::$_0::operator()() const /root/doris/be/src/pipeline/task_scheduler.cpp:121:23 #5 0x55fc8065b975 in doris::Defer<doris::pipeline::TaskScheduler::_do_work(int)::$_0>::~Defer() /root/doris/be/src/util/defer_op.h:37:16 #6 0x55fc8065b975 in doris::pipeline::TaskScheduler::_do_work(int) /root/doris/be/src/pipeline/task_scheduler.cpp:162:5 #7 0x55fc4c57cd19 in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:608:24 #8 0x55fc4c55395e in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 #9 0x55fc4c55395e in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5 #10 0x7f9ee3d25608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8 apache#11 0x7f9ee3fd2132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95 0x6160007e86f0 is located 624 bytes inside of 632-byte region [0x6160007e8480,0x6160007e86f8) freed by thread T1981 (Pipe_normal [wo) here: #0 0x55fc47aa680d in operator delete(void*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x3376e80d) (BuildId: 865149e62959581e) #1 0x55fc8059db84 in std::default_delete<doris::pipeline::PipelineTask>::operator()(doris::pipeline::PipelineTask*) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2 #2 0x55fc8059db84 in std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >::~unique_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4 #3 0x55fc8059db84 in void std::destroy_at<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15 #4 0x55fc8059db84 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7 #5 0x55fc8059db84 in void std::_Destroy_aux<false>::__destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*>(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6 #6 0x55fc8059db84 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*>(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7 #7 0x55fc8059db84 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7 #8 0x55fc8059db84 in std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >::~vector() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680:2 #9 0x55fc8052571c in void std::destroy_at<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15 #10 0x55fc8052571c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7 apache#11 0x55fc8052571c in void std::_Destroy_aux<false>::__destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6 apache#12 0x55fc8052571c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7 apache#13 0x55fc8052571c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7 apache#14 0x55fc8052571c in std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > > >::_M_erase_at_end(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1796:6 apache#15 0x55fc8052571c in std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > > >::clear() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1499:9 apache#16 0x55fc8052571c in doris::pipeline::PipelineFragmentContext::~PipelineFragmentContext() /root/doris/be/src/pipeline/pipeline_fragment_context.cpp:142:12 apache#17 0x55fc47ad30cc in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6 apache#18 0x55fc80658d57 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:11 apache#19 0x55fc80658d57 in std::__shared_ptr<doris::TaskExecutionContext, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31 apache#20 0x55fc80658d57 in doris::pipeline::close_task(doris::pipeline::PipelineTask*, doris::Status) /root/doris/be/src/pipeline/task_scheduler.cpp:100:1 apache#21 0x55fc8065aa17 in doris::pipeline::TaskScheduler::_do_work(int) /root/doris/be/src/pipeline/task_scheduler.cpp:160:36 apache#22 0x55fc4c57cd19 in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:608:24 apache#23 0x55fc4c55395e in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 apache#24 0x55fc4c55395e in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5 apache#25 0x7f9ee3d25608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
xy720
pushed a commit
that referenced
this pull request
Feb 26, 2025
Schema scanner runs on a separate thread which is executed asynchronously. We should make sure all context used not be freed once it is scheduled. ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002f33eb2 at pc 0x55e085dccbe3 bp 0x7f345c0e1f10 sp 0x7f345c0e1f08 READ of size 1 at 0x613002f33eb2 thread T2776 (FragmentMgrAsyn) #0 0x55e085dccbe2 in std::__atomic_base::load(std::memory_order) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9 #1 0x55e085dccbe2 in std::atomic::operator bool() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:87:22 #2 0x55e085dccbe2 in doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5 #3 0x55e085dccbe2 in void std::__invoke_impl(std::__invoke_other, doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #4 0x55e085dccbe2 in std::enable_if, void>::type std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #5 0x55e085dccbe2 in std::_Function_handler::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #6 0x55e050f081ca in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24 #7 0x55e050ede467 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 #8 0x7f376ef5aac2 in start_thread nptl/pthread_create.c:442:8 #9 0x7f376efec84f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
xy720
pushed a commit
that referenced
this pull request
Feb 27, 2025
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88 READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn) #0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9 #1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&) /home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44 #2 0x5559d8f289c0 in doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5 #3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other, doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #4 0x5559d8f289c0 in std::enable_if, void>::type std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24 #7 0x5559a4117877 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 #8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8 #9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
xy720
pushed a commit
that referenced
this pull request
Mar 19, 2025
### What problem does this PR solve?
```
/root/doris/be/src/vec/common/pod_array.h:510:29: runtime error: null pointer passed as argument 2, which is declared to never be null
/root/ldb_toolchain/bin/../usr/include/string.h:43:28: note: nonnull attribute specified here
#0 0x55cb2c2cea1e in void doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 15ul>::insert_assume_reserved<char const*, char const*>(char const*, char const*) /root/doris/be/src/vec/common/pod_array.h:510:9
#1 0x55cb2c2ce8a7 in void doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 15ul>::insert<char const*, char const*>(char const*, char const*) /root/doris/be/src/vec/common/pod_array.h:472:9
#2 0x55cb5a0b0d50 in doris::vectorized::StringOP::push_value_string(std::basic_string_view<char, std::char_traits<char>> const&, unsigned long, doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 15ul>&, doris::vectorized::PODArray<unsigned int, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 15ul>&) /root/doris/be/src/vec/functions/function_string.h:128:15
#3 0x55cb5d6843a2 in doris::Status doris::vectorized::FunctionStringParseUrl::vector_parse<false, true>(doris::vectorized::ColumnStr<unsigned int> const*, std::vector<doris::UrlParser::UrlPart, std::allocator<doris::UrlParser::UrlPart>>&, int, doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 15ul>&, doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 15ul>&, doris::vectorized::PODArray<unsigned int, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 15ul>&) /root/doris/be/src/vec/functions/function_string.h:2806:17
#4 0x55cb5d683e9d in auto doris::vectorized::FunctionStringParseUrl::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const::'lambda'(auto, auto)::operator()<std::integral_constant<bool, false>, std::integral_constant<bool, true>>(auto, auto) const /root/doris/be/src/vec/functions/function_string.h:2783:13
#5 0x55cb5d683c39 in auto std::__invoke_impl<doris::Status, doris::vectorized::FunctionStringParseUrl::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const::'lambda'(auto, auto), std::integral_constant<bool, false>, std::integral_constant<bool, true>>(std::__invoke_other, auto&&, std::integral_constant<bool, false>&&, std::integral_constant<bool, true>&&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14
#6 0x55cb5d683af5 in std::__invoke_result<auto, std::integral_constant<bool, false>, std::integral_constant<bool, true>>::type std::__invoke<doris::vectorized::FunctionStringParseUrl::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const::'lambda'(auto, auto), std::integral_constant<bool, false>, std::integral_constant<bool, true>>(auto&&, std::integral_constant<bool, false>&&, std::integral_constant<bool, true>&&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96:14
#7 0x55cb5d682dd3 in std::__detail::__variant::__gen_vtable_impl<std::__detail::__variant::_Multi_array<std::__detail::__variant::__deduce_visit_result<doris::Status> (*)(doris::vectorized::FunctionStringParseUrl::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const::'lambda'(auto, auto)&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>&&)>, std::integer_sequence<unsigned long, 0ul, 1ul>>::__visit_invoke(doris::vectorized::FunctionStringParseUrl::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const::'lambda'(auto, auto)&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>&&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/variant:1032:11
#8 0x55cb5d682ad5 in decltype(auto) std::__do_visit<std::__detail::__variant::__deduce_visit_result<doris::Status>, doris::vectorized::FunctionStringParseUrl::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const::'lambda'(auto, auto), std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>>(auto&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>&&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/variant:1758:15
#9 0x55cb5d67766b in std::invoke_result<auto, std::__conditional<is_lvalue_reference_v<std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>>>::type<std::variant_alternative<0ul, std::remove_reference<decltype(__variant::__as(std::declval<std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>>()))>::type>::type&, std::variant_alternative<0ul, std::remove_reference<decltype(__variant::__as(std::declval<std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>>()))>::type>::type&&>, std::__conditional<is_lvalue_reference_v<std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>>>::type<std::variant_alternative<0ul, std::remove_reference<decltype(__variant::__as(std::declval<std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>>()))>::type>::type&, std::variant_alternative<0ul, std::remove_reference<decltype(__variant::__as(std::declval<std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>>()))>::type>::type&&>>::type std::visit<doris::vectorized::FunctionStringParseUrl::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const::'lambda'(auto, auto), std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>>(auto&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>&&, std::variant<std::integral_constant<bool, false>, std::integral_constant<bool, true>>&&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/variant:1859:9
#10 0x55cb5d676604 in doris::vectorized::FunctionStringParseUrl::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /root/doris/be/src/vec/functions/function_string.h:2783:13
apache#11 0x55cb51a43fd4 in doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /root/doris/be/src/vec/functions/function.h:434:26
apache#12 0x55cb583f7dff in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:119:16
apache#13 0x55cb583e6de9 in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:244:12
apache#14 0x55cb583e5f53 in doris::vectorized::PreparedFunctionImpl::default_implementation_for_nulls(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool, bool*) const /root/doris/be/src/vec/functions/function.cpp:216:9
apache#15 0x55cb583f7939 in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:110:5
apache#16 0x55cb583e6de9 in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:244:12
apache#17 0x55cb583e7069 in doris::vectorized::PreparedFunctionImpl::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:250:12
apache#18 0x55cb51a3fd95 in doris::vectorized::IFunctionBase::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool) const /root/doris/be/src/vec/functions/function.h:193:19
apache#19 0x55cb51a2797f in doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector<unsigned int, std::allocator<unsigned int>>&) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:187:5
apache#20 0x55cb51a28a77 in doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:220:12
apache#21 0x55cb51b5f3b4 in doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vexpr_context.cpp:61:5
apache#22 0x55cb519a6732 in doris::vectorized::Scanner::_do_projections(doris::vectorized::Block*, doris::vectorized::Block*) /root/doris/be/src/vec/exec/scan/scanner.cpp:200:9
apache#23 0x55cb519a1b13 in doris::vectorized::Scanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) /root/doris/be/src/vec/exec/scan/scanner.cpp:82:16
apache#24 0x55cb5192a59d in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:241:5
apache#25 0x55cb51931c38 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:148:21
apache#26 0x55cb519314ef in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:147:31
apache#27 0x55cb519312fe in void std::__invoke_impl<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14
apache#28 0x55cb5193123e in std::enable_if<is_invocable_r_v<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&>, void>::type std::__invoke_r<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2
apache#29 0x55cb51930e45 in std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9
apache#30 0x55cb216f8e3f in std::function<void ()>::operator()() const /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9
apache#31 0x55cb51940ec6 in doris::vectorized::SimplifiedScanScheduler::submit_scan_task(doris::vectorized::SimplifiedScanTask)::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.h:149:65
apache#32 0x55cb51940e7e in void std::__invoke_impl<void, doris::vectorized::SimplifiedScanScheduler::submit_scan_task(doris::vectorized::SimplifiedScanTask)::'lambda'()&>(std::__invoke_other, doris::vectorized::SimplifiedScanScheduler::submit_scan_task(doris::vectorized::SimplifiedScanTask)::'lambda'()&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14
apache#33 0x55cb51940dbe in std::enable_if<is_invocable_r_v<void, doris::vectorized::SimplifiedScanScheduler::submit_scan_task(doris::vectorized::SimplifiedScanTask)::'lambda'()&>, void>::type std::__invoke_r<void, doris::vectorized::SimplifiedScanScheduler::submit_scan_task(doris::vectorized::SimplifiedScanTask)::'lambda'()&>(doris::vectorized::SimplifiedScanScheduler::submit_scan_task(doris::vectorized::SimplifiedScanTask)::'lambda'()&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2
apache#34 0x55cb51940745 in std::_Function_handler<void (), doris::vectorized::SimplifiedScanScheduler::submit_scan_task(doris::vectorized::SimplifiedScanTask)::'lambda'()>::_M_invoke(std::_Any_data const&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9
apache#35 0x55cb216f8e3f in std::function<void ()>::operator()() const /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9
apache#36 0x55cb28bd9844 in doris::FunctionRunnable::run() /root/doris/be/src/util/threadpool.cpp:64:27
apache#37 0x55cb28bb84b9 in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:616:24
apache#38 0x55cb28bfd263 in void std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(std::__invoke_memfun_deref, void (doris::ThreadPool::*&)(), doris::ThreadPool*&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:74:14
apache#39 0x55cb28bfd068 in std::__invoke_result<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>::type std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96:14
apache#40 0x55cb28bfcfa0 in void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::__call<void, 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:506:11
apache#41 0x55cb28bfcd95 in void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::operator()<void>() /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:591:17
apache#42 0x55cb28bfcc8e in void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14
apache#43 0x55cb28bfcbce in std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>(std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2
apache#44 0x55cb28bfc665 in std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>>::_M_invoke(std::_Any_data const&) /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9
apache#45 0x55cb216f8e3f in std::function<void ()>::operator()() const /root/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9
apache#46 0x55cb28b74241 in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5
apache#47 0x55cb2142be0a in asan_thread_start(void*) crtstuff.c
apache#48 0x7f17840221c9 in start_thread (/lib64/libpthread.so.0+0x81c9) (BuildId: 7c4add5c7a885e6ff4ce17867d6a2286e4420eec)
apache#49 0x7f1784a118d2 in clone (/lib64/libc.so.6+0x398d2) (BuildId: 4ee3325955e3b55b6805f33959b7cb77745ad625)
xy720
pushed a commit
that referenced
this pull request
Apr 8, 2025
…iteral with one more subfield causing BE coredump (apache#49485) Fixed the issue of inserting into a struct type string literal with one more subfield causing BE coredump some situation like this blow will make BE core ``` create table t(a int, b int, s struct<a:int>) PROPERTIES ("replication_allocation" = "tag.location.default: 1"); insert into t values(1,1,'{1,2}'); ``` core info: ``` [WARNING!] /sys/kernel/mm/transparent_hugepage/enabled: [always] madvise never, Doris not recommend turning on THP, which may cause the BE process to use more memory and cannot be freed in time. Turn off THP: `echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled` start BE in local mode ================================================================= ==2818976==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000f17a90 at pc 0x55eba7fdd69c bp 0x7f03243dbd30 sp 0x7f03243dbd28 READ of size 8 at 0x603000f17a90 thread T928 (brpc_light) #0 0x55eba7fdd69b in std::__shared_ptr<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2>::get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1291:16 #1 0x55eba7fdd649 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:990:66 #2 0x55eba7fda316 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::operator->() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:984:9 #3 0x55ebcfc8b19e in doris::vectorized::DataTypeStructSerDe::deserialize_one_cell_from_json(doris::vectorized::IColumn&, doris::Slice&, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_struct_serde.cpp:200:25 #4 0x55ebdac49c8b in doris::vectorized::ConvertImplGenericFromString::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:618:36 #5 0x55ebda2234d0 in doris::Status std::__invoke_impl<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #6 0x55ebda22323a in std::enable_if<is_invocable_r_v<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>, doris::Status>::type std::__invoke_r<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:114:9 #7 0x55ebda222e71 in std::_Function_handler<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long), doris::Status (*)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)>::_M_invoke(std::_Any_data const&, doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #8 0x55ebd9b3f1d2 in std::function<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long)>::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long) const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 #9 0x55ebda1ad7c7 in doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:2241:17 #10 0x55ebda1acee2 in doris::Status std::__invoke_impl<doris::Status, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)&, doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bo@@@ ```
xy720
pushed a commit
that referenced
this pull request
Apr 8, 2025
…he#49700) ### What problem does this PR solve? When a BE is down, global object should shut down following a specific order to ensure BE will not crash. Because all running queries will finish once fragment mgr finished, stream mgr must be released before it because it may wake up some blocked tasks. #0 __GI___pthread_sigmask (how=2, newmask=<optimized out>, oldmask=0x0) at ./nptl/pthread_sigmask.c:43 #1 0x00007f842b2bb71e in PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #2 0x00007f842b2bc206 in JVM_handle_linux_signal () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #3 <signal handler called> #4 ___pthread_mutex_lock (mutex=0x1b8) at ./nptl/pthread_mutex_lock.c:80 #5 0x000055c5f03edcca in __gthread_mutex_lock (__mutex=0x1b8) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/x86_64-linux-gnu/c++/11/bits/gthr-default.h:749 #6 std::mutex::lock (this=0x1b8) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_mutex.h:100 #7 std::lock_guard<std::mutex>::lock_guard (__m=..., this=<optimized out>) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_mutex.h:229 #8 doris::RuntimeProfile::add_info_string (this=0x0, key=..., value=...) at /home/zcp/repo_center/doris_master/doris/be/src/util/runtime_profile.cpp:354 #9 0x000055c5f9cccaa7 in doris::pipeline::PipelineTask::_state_transition (this=0x7f84015b4000, new_state=doris::pipeline::PipelineTask::State::RUNNABLE) at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/pipeline_task.h:339 #10 0x000055c5fa53aec5 in doris::pipeline::PipelineTask::wake_up (this=0x7fffbd28a8e8, dep=<optimized out>) at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/pipeline_task.cpp:748 apache#11 0x000055c5f9cba3f5 in doris::pipeline::Dependency::set_ready (this=0x7f832f848310) at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/dependency.cpp:72 apache#12 0x000055c5f9c53bcd in doris::vectorized::VDataStreamRecvr::SenderQueue::set_source_ready (this=0x7f839d485880) at /home/zcp/repo_center/doris_master/doris/be/src/vec/runtime/vdata_stream_recvr.cpp:142 apache#13 doris::vectorized::VDataStreamRecvr::SenderQueue::close (this=0x7f839d485880) at /home/zcp/repo_center/doris_master/doris/be/src/vec/runtime/vdata_stream_recvr.cpp:301 apache#14 0x000055c5f9c55b39 in doris::vectorized::VDataStreamRecvr::close (this=0x7f83fbe78940) at /home/zcp/repo_center/doris_master/doris/be/src/vec/runtime/vdata_stream_recvr.cpp:488 apache#15 0x000055c5f9c4dc68 in doris::vectorized::VDataStreamMgr::~VDataStreamMgr (this=0x7f83f6d99da0) at /home/zcp/repo_center/doris_master/doris/be/src/vec/runtime/vdata_stream_mgr.cpp:58 apache#16 0x000055c5f00cb651 in doris::ExecEnv::destroy (this=0x55c5fdf7a198 <doris::ExecEnv::GetInstance()::s_exec_env>) at /home/zcp/repo_center/doris_master/doris/be/src/runtime/exec_env_init.cpp:796 apache#17 0x000055c5ef387909 in main (argc=<optimized out>, argv=<optimized out>) at /home/zcp/repo_center/doris_master/doris/be/src/service/doris_main.cpp:635
xy720
pushed a commit
that referenced
this pull request
Apr 8, 2025
…r/signal to avoid mult… (apache#49739) …ithread read and write wrapper ### What problem does this PR solve? ```cpp ================================================================= ==28655==ERROR: AddressSanitizer: heap-use-after-free on address 0x61000331f350 at pc 0x5577863cea6c bp 0x7fac1d2a9ef0 sp 0x7fac1d2a9ee8 READ of size 1 at 0x61000331f350 thread T1933 (Pipe_normal [wo) #0 0x5577863cea6b in doris::RuntimeFilterWrapper::debug_string[abi:cxx11]() /root/doris/be/src/runtime_filter/runtime_filter_wrapper.cpp:585:46 #1 0x55778632f8f5 in doris::RuntimeFilter::_debug_string[abi:cxx11]() const /root/doris/be/src/runtime_filter/runtime_filter.cpp:128:61 #2 0x55778638d946 in doris::RuntimeFilterConsumer::debug_string[abi:cxx11]() const /root/doris/be/src/runtime_filter/runtime_filter_consumer.h:61:57 #3 0x55778634ed24 in doris::RuntimeFilterConsumer::_set_state(doris::RuntimeFilterConsumer::State) /root/doris/be/src/runtime_filter/runtime_filter_consumer.h:125:43 #4 0x55778634c6fa in doris::RuntimeFilterConsumer::acquire_expr(std::vector<std::shared_ptr<doris::vectorized::VRuntimeFilterWrapper>, std::allocator<std::shared_ptr<doris::vectorized::VRuntimeFilterWrapper> > >&) /root/doris/be/src/runtime_filter/runtime_filter_consumer.cpp:67:9 #5 0x5577a1fbdd1b in doris::RuntimeFilterConsumerHelper::acquire_runtime_filter(std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > >&) /root/doris/be/src/runtime_filter/runtime_filter_consumer_helper.cpp:90:9 #6 0x5577b9aec26c in doris::pipeline::ScanLocalState<doris::pipeline::OlapScanLocalState>::open(doris::RuntimeState*) /root/doris/be/src/pipeline/exec/scan_operator.cpp:100:5 #7 0x5577ba853e05 in doris::pipeline::PipelineTask::_open() /root/doris/be/src/pipeline/pipeline_task.cpp:217:32 #8 0x5577ba8608e0 in doris::pipeline::PipelineTask::execute(bool*) /root/doris/be/src/pipeline/pipeline_task.cpp:407:9 #9 0x5577ba8b4bb5 in doris::pipeline::TaskScheduler::_do_work(int) /root/doris/be/src/pipeline/task_scheduler.cpp:147:9 #10 0x5577869c537c in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:619:24 apache#11 0x55778699ba1e in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 apache#12 0x55778699ba1e in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:496:5 apache#13 0x7fb57bef3608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8 apache#14 0x7fb57c1a0132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95 0x61000331f350 is located 16 bytes inside of 192-byte region [0x61000331f340,0x61000331f400) freed by thread T817 (brpc_light) here: #0 0x557781e1c80d in operator delete(void*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x3395280d) (BuildId: 45c9f8cc52f91b28) #1 0x557781e778b8 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::operator=(std::__shared_count<(__gnu_cxx::_Lock_policy)2> const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:721:15 #2 0x55778634d1d1 in std::__shared_ptr<doris::RuntimeFilterWrapper, (__gnu_cxx::_Lock_policy)2>::operator=(std::__shared_ptr<doris::RuntimeFilterWrapper, (__gnu_cxx::_Lock_policy)2> const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1148:69 #3 0x55778634d1d1 in std::shared_ptr<doris::RuntimeFilterWrapper>::operator=(std::shared_ptr<doris::RuntimeFilterWrapper> const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:359:65 #4 0x55778634d1d1 in doris::RuntimeFilterConsumer::signal(doris::RuntimeFilter*) /root/doris/be/src/runtime_filter/runtime_filter_consumer.cpp:76:14 #5 0x557785dd52c3 in auto doris::FragmentMgr::apply_filterv2(doris::PPublishFilterRequestV2 const*, butil::IOBufAsZeroCopyInputStream*)::$_0::operator()<std::shared_ptr<doris::RuntimeFilterConsumer> >(std::shared_ptr<doris::RuntimeFilterConsumer>&) const /root/doris/be/src/runtime/fragment_mgr.cpp:1295:72 #6 0x557785dd52c3 in void std::__invoke_impl<void, doris::FragmentMgr::apply_filterv2(doris::PPublishFilterRequestV2 const*, butil::IOBufAsZeroCopyInputStream*)::$_0&, std::shared_ptr<doris::RuntimeFilterConsumer>&>(std::__invoke_other, doris::FragmentMgr::apply_filterv2(doris::PPublishFilterRequestV2 const*, butil::IOBufAsZeroCopyInputStream*)::$_0&, std::shared_ptr<doris::RuntimeFilterConsumer>&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #7 0x557785dd52c3 in std::__invoke_result<doris::FragmentMgr::apply_filterv2(doris::PPublishFilterRequestV2 const*, butil::IOBufAsZeroCopyInputStream*)::$_0&, std::shared_ptr<doris::RuntimeFilterConsumer>&>::type std::__invoke<doris::FragmentMgr::apply_filterv2(doris::PPublishFilterRequestV2 const*, butil::IOBufAsZeroCopyInputStream*)::$_0&, std::shared_ptr<doris::RuntimeFilterConsumer>&>(doris::FragmentMgr::apply_filterv2(doris::PPublishFilterRequestV2 const*, butil::IOBufAsZeroCopyInputStream*)::$_0&, std::shared_ptr<doris::RuntimeFilterConsumer>&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 #8 0x557785dd52c3 in std::ranges::in_fun_result<__gnu_cxx::__normal_iterator<std::shared_ptr<doris::RuntimeFilterConsumer>*, std::vector<std::shared_ptr<doris::RuntimeFilterConsumer>, std::allocator<std::shared_ptr<doris::RuntimeFilterConsumer> > > >, doris::FragmentMgr::apply_filterv2(doris::PPublishFilterRequestV2 const*, butil::IOBufAsZeroCopyInputStream*)::$_0> std::ranges::__for_each_fn::operator()<__gnu_cxx::__normal_iterator<std::shared_ptr<doris::RuntimeFilterConsumer>*, std::vector<std::shared_ptr<doris::RuntimeFilterConsumer>, std::allocator<std::shared_ptr<doris::RuntimeFilterConsumer> > > >, __gnu_cxx::__normal_iterator<std::shared_ptr<doris::RuntimeFilterConsumer>*, std::vector<std::shared_ptr<doris::RuntimeFilterConsumer>, std::allocator<std::shared_ptr<doris::RuntimeFilterConsumer> > > >, std::identity, doris::FragmentMgr::apply_filterv2(doris::PPublishFilterRequestV2 const*, butil::IOBufAsZeroCopyInputStream*)::$_0>(__gnu_cxx::__normal_iterator<std::shared_ptr<doris::RuntimeFilterConsumer>*, std::vector<std::shared_ptr<doris::RuntimeFilterConsumer>, std::allocator<std::shared_ptr<doris::RuntimeFilterConsumer> > > >, __gnu_cxx::__normal_iterator<std::shared_ptr<doris::RuntimeFilterConsumer>*, std::vector<std::shared_ptr<doris::RuntimeFilterConsumer>, std::allocator<std::shared_ptr<doris::RuntimeFilterConsumer> > > >, doris::FragmentMgr::apply_filterv2(doris::PPublishFilterRequestV2 const*, butil::IOBufAsZeroCopyInputStream*)::$_0, std::identity) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/ranges_algo.h:187:4 #9 0x557785dd52c3 in _ZNKSt6ranges13__for_each_fnclIRSt6vectorISt10shared_ptrIN5doris21RuntimeFilterConsumerEESaIS6_EESt8identityZNS4_11FragmentMgr14apply_filterv2EPKNS4_23PPublishFilterRequestV2EPN5butil26IOBufAsZeroCopyInputStreamEE3$_0EENS_13in_fun_resultINSt11conditionalIL_ZNS_14borrowed_rangeIT_EEEDTclsr8__detailE14__ranges_beginclsr3stdE7declvalIRSM_EEEENS_8danglingEE4typeET1_EEOSM_SS_T0_ /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/ranges_algo.h:197:9 #10 0x557785dd52c3 in doris::FragmentMgr::apply_filterv2(doris::PPublishFilterRequestV2 const*, butil::IOBufAsZeroCopyInputStream*) /root/doris/be/src/runtime/fragment_mgr.cpp:1295:13 apache#11 0x5577865a517e in doris::PInternalService::apply_filterv2(google::protobuf::RpcController*, doris::PPublishFilterRequestV2 const*, doris::PPublishFilterResponse*, google::protobuf::Closure*)::$_0::operator()() const /root/doris/be/src/service/internal_service.cpp:1444:48 apache#12 0x5577865a517e in void std::__invoke_impl<void, doris::PInternalService::apply_filterv2(google::protobuf::RpcController*, doris::PPublishFilterRequestV2 const*, doris::PPublishFilterResponse*, google::protobuf::Closure*)::$_0&>(std::__invoke_other, doris::PInternalService::apply_filterv2(google::protobuf::RpcController*, doris::PPublishFilterRequestV2 const*, doris::PPublishFilterResponse*, google::protobuf::Closure*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 apache#13 0x5577865a517e in std::enable_if<is_invocable_r_v<void, doris::PInternalService::apply_filterv2(google::protobuf::RpcController*, doris::PPublishFilterRequestV2 const*, doris::PPublishFilterResponse*, google::protobuf::Closure*)::$_0&>, void>::type std::__invoke_r<void, doris::PInternalService::apply_filterv2(google::protobuf::RpcController*, doris::PPublishFilterRequestV2 const*, doris::PPublishFilterResponse*, google::protobuf::Closure*)::$_0&>(doris::PInternalService::apply_filterv2(google::protobuf::RpcController*, doris::PPublishFilterRequestV2 const*, doris::PPublishFilterResponse*, google::protobuf::Closure*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 apache#14 0x5577865a517e in std::_Function_handler<void (), doris::PInternalService::apply_filterv2(google::protobuf::RpcController*, doris::PPublishFilterRequestV2 const*, doris::PPublishFilterResponse*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 apache#15 0x5577865f07c1 in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 apache#16 0x5577865f07c1 in doris::WorkThreadPool<false>::work_thread(int) /root/doris/be/src/util/work_thread_pool.hpp:158:17 apache#17 0x5577bddb233f in execute_native_thread_routine /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:82:18 previously allocated by thread T781 (brpc_light) here: #0 0x557781e1bfad in operator new(unsigned long) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x33951fad) (BuildId: 45c9f8cc52f91b28) #1 0x5577863411ae in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<doris::RuntimeFilterWrapper, std::allocator<doris::RuntimeFilterWrapper>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:121:27 #2 0x5577863411ae in std::allocator<std::_Sp_counted_ptr_inplace<doris::RuntimeFilterWrapper, std::allocator<doris::RuntimeFilterWrapper>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:173:32 #3 0x5577863411ae in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<doris::RuntimeFilterWrapper, std::allocator<doris::RuntimeFilterWrapper>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<doris::RuntimeFilterWrapper, std::allocator<doris::RuntimeFilterWrapper>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:460:20 #4 0x5577863411ae in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<doris::RuntimeFilterWrapper, std::allocator<doris::RuntimeFilterWrapper>, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<doris::RuntimeFilterWrapper, std::allocator<doris::RuntimeFilterWrapper>, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<doris::RuntimeFilterWrapper, std::allocator<doris::RuntimeFilterWrapper>, (__gnu_cxx::_Lock_policy)2> >&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocated_ptr.h:97:21 #5 0x5577863411ae in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<doris::RuntimeFilterWrapper, std::allocator<doris::RuntimeFilterWrapper>, doris::RuntimeFilterParams*>(doris::RuntimeFilterWrapper*&, std::_Sp_alloc_shared_tag<std::allocator<doris::RuntimeFilterWrapper> >, doris::RuntimeFilterParams*&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:648:19 #6 0x55778632e555 in std::__shared_ptr<doris::RuntimeFilterWrapper, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<doris::RuntimeFilterWrapper>, doris::RuntimeFilterParams*>(std::_Sp_alloc_shared_tag<std::allocator<doris::RuntimeFilterWrapper> >, doris::RuntimeFilterParams*&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1337:14 #7 0x55778632e555 in std::shared_ptr<doris::RuntimeFilterWrapper>::shared_ptr<std::allocator<doris::RuntimeFilterWrapper>, doris::RuntimeFilterParams*>(std::_Sp_alloc_shared_tag<std::allocator<doris::RuntimeFilterWrapper> >, doris::RuntimeFilterParams*&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:409:4 #8 0x55778632e555 in std::shared_ptr<doris::RuntimeFilterWrapper> std::allocate_shared<doris::RuntimeFilterWrapper, std::allocator<doris::RuntimeFilterWrapper>, doris::RuntimeFilterParams*>(std::allocator<doris::RuntimeFilterWrapper> const&, doris::RuntimeFilterParams*&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:860:14 #9 0x55778632e555 in std::shared_ptr<doris::RuntimeFilterWrapper> std::make_shared<doris::RuntimeFilterWrapper, doris::RuntimeFilterParams*>(doris::RuntimeFilterParams*&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:876:14 #10 0x55778632e555 in doris::RuntimeFilter::_init_with_desc(doris::TRuntimeFilterDesc const*, doris::TQueryOptions const*) /root/doris/be/src/runtime_filter/runtime_filter.cpp:123:16 apache#11 0x5577863794de in doris::RuntimeFilterConsumer::create(doris::RuntimeFilterParamsContext*, doris::TRuntimeFilterDesc const*, int, std::shared_ptr<doris::RuntimeFilterConsumer>*, doris::RuntimeProfile*) /root/doris/be/src/runtime_filter/runtime_filter_consumer.h:46:9 apache#12 0x55778636bb9f in doris::RuntimeFilterMgr::register_consumer_filter(doris::TRuntimeFilterDesc const&, int, std::shared_ptr<doris::RuntimeFilterConsumer>*, doris::RuntimeProfile*) /root/doris/be/src/runtime_filter/runtime_filter_mgr.cpp:80:5 apache#13 0x557786147c10 in doris::RuntimeState::register_consumer_runtime_filter(doris::TRuntimeFilterDesc const&, bool, int, std::shared_ptr<doris::RuntimeFilterConsumer>*, doris::RuntimeProfile*) /root/doris/be/src/runtime/runtime_state.cpp:514:17 apache#14 0x5577a1fbc3e1 in doris::RuntimeFilterConsumerHelper::_register_runtime_filter(bool) /root/doris/be/src/runtime_filter/runtime_filter_consumer_helper.cpp:51:9 ``` ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [x] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [x] Other reason <!-- Add your reason? --> - Behavior changed: - [x] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [x] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
xy720
pushed a commit
that referenced
this pull request
May 15, 2025
) ### What problem does this PR solve? If there is a mark join condition, then even in a right semi join, the columns from the left table involved in the mark join condition will still appear in the intermediate tuple. This PR also completes the missing handling logic for right semi joins with mark join condition. ``` ==2126730==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x503004ee02e0 at pc 0x55f4a7d3f354 bp 0x7f6b6b587e70 sp 0x7f6b6b587e68 READ of size 8 at 0x503004ee02e0 thread T1296 (brpc_light) #0 0x55f4a7d3f353 in std::__shared_ptr<doris::vectorized::IDataType const, (__gnu_cxx::_Lock_policy)2>::__shared_ptr(std::__shared_ptr<doris::vectorized::IDataType const, (__gnu_cxx::_Lock_policy)2> const&) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/shared_ptr_base.h:1522:7 #1 0x55f4a7d3f01e in std::shared_ptr<doris::vectorized::IDataType const>::shared_ptr(std::shared_ptr<doris::vectorized::IDataType const> const&) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/shared_ptr.h:204:7 #2 0x55f4f8aa231c in doris::pipeline::HashJoinProbeOperatorX::prepare(doris::RuntimeState*) /root/doris/be/src/pipeline/exec/hashjoin_probe_operator.cpp:577:33 #3 0x55f4fb1e4a28 in doris::pipeline::Pipeline::prepare(doris::RuntimeState*) /root/doris/be/src/pipeline/pipeline.cpp:89:5 #4 0x55f4fb089d3f in doris::pipeline::PipelineFragmentContext::prepare(doris::TPipelineFragmentParams const&, doris::ThreadPool*) /root/doris/be/src/pipeline/pipeline_fragment_context.cpp:352:9 #5 0x55f4ad615a5c in doris::FragmentMgr::exec_plan_fragment(doris::TPipelineFragmentParams const&, doris::QuerySource, std::function<void (doris::RuntimeState*, doris::Status*)> const&, doris::TPipelineFragmentParamsList const&) /root/doris/be/src/runtime/fragment_mgr.cpp:855:9 #6 0x55f4ad613ca6 in doris::FragmentMgr::exec_plan_fragment(doris::TPipelineFragmentParams const&, doris::QuerySource, doris::TPipelineFragmentParamsList const&) /root/doris/be/src/runtime/fragment_mgr.cpp:634:16 #7 0x55f4ae2e867c in doris::PInternalService::_exec_plan_fragment_impl(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, doris::PFragmentRequestVersion, bool, std::function<void (doris::RuntimeState*, doris::Status*)> const&) /root/doris/be/src/service/internal_service.cpp:613:17 #8 0x55f4ae2e4b67 in doris::PInternalService::_exec_plan_fragment_in_pthread(google::protobuf::RpcController*, doris::PExecPlanFragmentRequest const*, doris::PExecPlanFragmentResult*, google::protobuf::Closure*) /root/doris/be/src/service/internal_service.cpp:343:14 #9 0x55f4ae31d4e1 in doris::PInternalService::exec_plan_fragment_prepare(google::protobuf::RpcController*, doris::PExecPlanFragmentRequest const*, doris::PExecPlanFragmentResult*, google::protobuf::Closure*)::$_0::operator()() const /root/doris/be/src/service/internal_service.cpp:367:9 #10 0x55f4ae31d33e in void std::__invoke_impl<void, doris::PInternalService::exec_plan_fragment_prepare(google::protobuf::RpcController*, doris::PExecPlanFragmentRequest const*, doris::PExecPlanFragmentResult*, google::protobuf::Closure*)::$_0&>(std::__invoke_other, doris::PInternalService::exec_plan_fragment_prepare(google::protobuf::RpcController*, doris::PExecPlanFragmentRequest const*, doris::PExecPlanFragmentResult*, google::protobuf::Closure*)::$_0&) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 apache#11 0x55f4ae31d27e in std::enable_if<is_invocable_r_v<void, doris::PInternalService::exec_plan_fragment_prepare(google::protobuf::RpcController*, doris::PExecPlanFragmentRequest const*, doris::PExecPlanFragmentResult*, google::protobuf::Closure*)::$_0&>, void>::type std::__invoke_r<void, doris::PInternalService::exec_plan_fragment_prepare(google::protobuf::RpcController*, doris::PExecPlanFragmentRequest const*, doris::PExecPlanFragmentResult*, google::protobuf::Closure*)::$_0&>(doris::PInternalService::exec_plan_fragment_prepare(google::protobuf::RpcController*, doris::PExecPlanFragmentRequest const*, doris::PExecPlanFragmentResult*, google::protobuf::Closure*)::$_0&) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 apache#12 0x55f4ae31cf55 in std::_Function_handler<void (), doris::PInternalService::exec_plan_fragment_prepare(google::protobuf::RpcController*, doris::PExecPlanFragmentRequest const*, doris::PExecPlanFragmentResult*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 apache#13 0x55f4a74175af in std::function<void ()>::operator()() const /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 apache#14 0x55f4ae3c21c4 in doris::WorkThreadPool<false>::work_thread(int) /root/doris/be/src/util/work_thread_pool.hpp:158:17 apache#15 0x55f4ae3c4cc8 in void std::__invoke_impl<void, void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&>(std::__invoke_memfun_deref, void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:74:14 apache#16 0x55f4ae3c4a92 in std::__invoke_result<void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&>::type std::__invoke<void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&>(void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96:14 apache#17 0x55f4ae3c49f8 in decltype(std::__invoke((*this)._M_pmf, std::forward<doris::WorkThreadPool<false>*&>(fp), std::forward<int&>(fp))) std::_Mem_fn_base<void (doris::WorkThreadPool<false>::*)(int), true>::operator()<doris::WorkThreadPool<false>*&, int&>(doris::WorkThreadPool<false>*&, int&) const /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:170:11 apache#18 0x55f4ae3c4942 in void std::__invoke_impl<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&>(std::__invoke_other, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 apache#19 0x55f4ae3c4752 in std::enable_if<is_invocable_r_v<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&>, void>::type std::__invoke_r<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&>(std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2 apache#20 0x55f4ae3c4664 in void std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>::__call<void, 0ul, 1ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:654:11 apache#21 0x55f4ae3c4395 in void std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>::operator()<>() /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:713:17 apache#22 0x55f4ae3c428e in void std::__invoke_impl<void, std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>(std::__invoke_other, std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>&&) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 apache#23 0x55f4ae3c41ce in std::__invoke_result<std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>::type std::__invoke<std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>(std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>&&) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96:14 apache#24 0x55f4ae3c417b in void std::thread::_Invoker<std::tuple<std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>>::_M_invoke<0ul>(std::_Index_tuple<0ul>) /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_thread.h:292:13 apache#25 0x55f4ae3c40f6 in std::thread::_Invoker<std::tuple<std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>>::operator()() /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_thread.h:299:11 apache#26 0x55f4ae3c3f34 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>>>::_M_run() /root/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_thread.h:244:13 apache#27 0x55f4fea05d2e in execute_native_thread_routine pthread_atfork.c apache#28 0x55f4a7162e0a in asan_thread_start(void*) crtstuff.c apache#29 0x7f7149c421c9 in start_thread (/lib64/libpthread.so.0+0x81c9) (BuildId: 7c4add5c7a885e6ff4ce17867d6a2286e4420eec) apache#30 0x7f714a6318d2 in clone (/lib64/libc.so.6+0x398d2) (BuildId: 4ee3325955e3b55b6805f33959b7cb77745ad625) ```
xy720
pushed a commit
that referenced
this pull request
May 26, 2025
…string to a struct. (apache#50851) ``` select cast('{"a":1,"b":"1","c":"1","d":"1"}' as struct<a:int, b:int>); ==3393513==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x506001118e20 at pc 0x55808f91364a bp 0x7f9ec8816af0 sp 0x7f9ec8816ae8 READ of size 8 at 0x506001118e20 thread T1298 (brpc_light) #0 0x55808f913649 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_data() const /mnt/disk6/common/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:223:28 #1 0x55808f9386f6 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::data() const /mnt/disk6/common/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.h:2597:16 #2 0x55808fc4e57a in doris::Slice::Slice(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) /mnt/disk12/yanxuecheng/doris/be/src/util/slice.h:72:40 #3 0x5580bebb7ed9 in doris::vectorized::DataTypeStructSerDe::deserialize_one_cell_from_json(doris::vectorized::IColumn&, doris::Slice&, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk12/yanxuecheng/doris/be/src/vec/data_types/serde/data_type_struct_serde.cpp:141:17 #4 0x5580c90cf235 in doris::vectorized::ConvertImplGenericFromString::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) /mnt/disk12/yanxuecheng/doris/be/src/vec/functions/function_cast.h:513:36 #5 0x5580c89607e6 in doris::Status std::__invoke_impl<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk6/common/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 #6 0x5580c89605ca in std::enable_if<is_invocable_r_v<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>, doris::Status>::type std::__invoke_r<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk6/common/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:114:9 #7 0x5580c8960201 in std::_Function_handler<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long), doris::Status (*)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)>::_M_invoke(std::_Any_data const&, doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk6/common/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 #8 0x5580c827ff5c in std::function<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long)>::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long) const /mnt/disk6/common/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 #9 0x5580c88fb7a7 in doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /mnt/disk12/yanxuecheng/doris/be/src/vec/functions/function_cast.h:2148:17 #10 0x5580c88fae32 in doris::Status std::__invoke_impl<doris::Status, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)&, doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)&, doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk6/common/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14 apache#11 0x5580c88fac0a in std::enable_if<is_invocable_r_v<doris::Status, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)&, doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>, doris::Status>::type std::__invoke_r<doris::Status, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)&, doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)&, doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk6/common/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:114:9 apache#12 0x5580c88fa641 in std::_Function_handler<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long), doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)>::_M_invoke(std::_Any_data const&, doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk6/common/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9 apache#13 0x5580c827ff5c in std::function<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long)>::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long) const /mnt/disk6/common/ldb_toolchain_robin/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9 ``` now ``` mysql> select cast('{"a":1,"b":"1","c":"1","d":"1"}' as struct<a:int, b:int>); +-----------------------------------------------------------------+ | cast('{"a":1,"b":"1","c":"1","d":"1"}' as struct<a:int, b:int>) | +-----------------------------------------------------------------+ | NULL | +-----------------------------------------------------------------+
xy720
pushed a commit
that referenced
this pull request
Jun 19, 2025
### What problem does this PR solve?
```
=================================================================
==8328==ERROR: AddressSanitizer: attempting double-free on 0x60600333cf80 in thread T1682:
#0 0x5631ccdaed9d in operator delete(void*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x3caa6d9d) (BuildId: dab4df3df11ecdce)
#1 0x5631ccdde467 in std::__uniq_ptr_impl>::operator=(std::__uniq_ptr_impl>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:167:2
#2 0x5631ccdc942d in std::__uniq_ptr_data, true, true>::operator=(std::__uniq_ptr_data, true, true>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:212:61
#3 0x5631ccdc942d in std::unique_ptr>::operator=(std::unique_ptr>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:371:51
#4 0x5631ccdc942d in doris::Status::operator=(doris::Status&&) /home/zcp/repo_center/doris_master/doris/be/src/common/status.h:400:22
#5 0x56320bfc33ba in doris::pipeline::MaterializationCallback::call() /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/materialization_sink_operator.cpp:88:39
#6 0x56320bfc5cdd in doris::AutoReleaseClosure>::Run() /home/zcp/repo_center/doris_master/doris/be/src/util/brpc_closure.h:98:18
#7 0x5631d03b7b1b in doris::FailureDetectClosure::Run() /home/zcp/repo_center/doris_master/doris/be/src/util/brpc_client_cache.h:83:20
#8 0x56320dce1943 in brpc::Controller::EndRPC(brpc::Controller::CompletionInfo const&) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d9d9943) (BuildId: dab4df3df11ecdce)
#9 0x56320dd0cc4e in brpc::policy::ProcessRpcResponse(brpc::InputMessageBase*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7da04c4e) (BuildId: dab4df3df11ecdce)
#10 0x56320dd03f76 in brpc::ProcessInputMessage(void*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d9fbf76) (BuildId: dab4df3df11ecdce)
apache#11 0x56320dd04920 in brpc::InputMessenger::InputMessageClosure::~InputMessageClosure() (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d9fc920) (BuildId: dab4df3df11ecdce)
apache#12 0x56320dd052ab in brpc::InputMessenger::OnNewMessages(brpc::Socket*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d9fd2ab) (BuildId: dab4df3df11ecdce)
apache#13 0x56320de3437d in brpc::Socket::ProcessEvent(void*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7db2c37d) (BuildId: dab4df3df11ecdce)
apache#14 0x56320dc970a6 in bthread::TaskGroup::task_runner(long) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d98f0a6) (BuildId: dab4df3df11ecdce)
apache#15 0x56320dc83660 in bthread_make_fcontext (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d97b660) (BuildId: dab4df3df11ecdce)
0x60600333cf80 is located 0 bytes inside of 64-byte region [0x60600333cf80,0x60600333cfc0)
freed by thread T1791 here:
#0 0x5631ccdaed9d in operator delete(void*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x3caa6d9d) (BuildId: dab4df3df11ecdce)
previously allocated by thread T1623 here:
#0 0x5631ccdae53d in operator new(unsigned long) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x3caa653d) (BuildId: dab4df3df11ecdce)
Thread T1682 created by T0 here:
#0 0x5631ccd5bcaa in pthread_create (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x3ca53caa) (BuildId: dab4df3df11ecdce)
#1 0x56320dc907be in bthread::TaskControl::add_workers(int) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d9887be) (BuildId: dab4df3df11ecdce)
#2 0x56320dc7f0ac in bthread_setconcurrency (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d9770ac) (BuildId: dab4df3df11ecdce)
#3 0x56320ddfcc79 in brpc::Server::StartInternal(butil::EndPoint const&, brpc::PortRange const&, brpc::ServerOptions const*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7daf4c79) (BuildId: dab4df3df11ecdce)
#4 0x56320ddfecc9 in brpc::Server::Start(butil::EndPoint const&, brpc::ServerOptions const*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7daf6cc9) (BuildId: dab4df3df11ecdce)
#5 0x5631d13708df in doris::BRpcService::start(int, int) /home/zcp/repo_center/doris_master/doris/be/src/service/brpc_service.cpp:94:18
#6 0x5631ccdb73e2 in main /home/zcp/repo_center/doris_master/doris/be/src/service/doris_main.cpp:570:28
#7 0x7faec2201d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
Thread T1791 created by T0 here:
#0 0x5631ccd5bcaa in pthread_create (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x3ca53caa) (BuildId: dab4df3df11ecdce)
#1 0x56320dc907be in bthread::TaskControl::add_workers(int) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d9887be) (BuildId: dab4df3df11ecdce)
#2 0x56320dc7f0ac in bthread_setconcurrency (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d9770ac) (BuildId: dab4df3df11ecdce)
#3 0x56320ddfcc79 in brpc::Server::StartInternal(butil::EndPoint const&, brpc::PortRange const&, brpc::ServerOptions const*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7daf4c79) (BuildId: dab4df3df11ecdce)
#4 0x56320ddfecc9 in brpc::Server::Start(butil::EndPoint const&, brpc::ServerOptions const*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7daf6cc9) (BuildId: dab4df3df11ecdce)
#5 0x5631d13708df in doris::BRpcService::start(int, int) /home/zcp/repo_center/doris_master/doris/be/src/service/brpc_service.cpp:94:18
#6 0x5631ccdb73e2 in main /home/zcp/repo_center/doris_master/doris/be/src/service/doris_main.cpp:570:28
#7 0x7faec2201d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
Thread T1623 created by T0 here:
#0 0x5631ccd5bcaa in pthread_create (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x3ca53caa) (BuildId: dab4df3df11ecdce)
#1 0x56320dc907be in bthread::TaskControl::add_workers(int) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d9887be) (BuildId: dab4df3df11ecdce)
#2 0x56320dc7f0ac in bthread_setconcurrency (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7d9770ac) (BuildId: dab4df3df11ecdce)
#3 0x56320ddfcc79 in brpc::Server::StartInternal(butil::EndPoint const&, brpc::PortRange const&, brpc::ServerOptions const*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7daf4c79) (BuildId: dab4df3df11ecdce)
#4 0x56320ddfecc9 in brpc::Server::Start(butil::EndPoint const&, brpc::ServerOptions const*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x7daf6cc9) (BuildId: dab4df3df11ecdce)
#5 0x5631d13708df in doris::BRpcService::start(int, int) /home/zcp/repo_center/doris_master/doris/be/src/service/brpc_service.cpp:94:18
#6 0x5631ccdb73e2 in main /home/zcp/repo_center/doris_master/doris/be/src/service/doris_main.cpp:570:28
#7 0x7faec2201d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
SUMMARY: AddressSanitizer: double-free (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x3caa6d9d) (BuildId: dab4df3df11ecdce) in operator delete(void*)
```
Add a lock when need change the materialization status
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [x] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [x] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
xy720
pushed a commit
that referenced
this pull request
Jun 19, 2025
…apache#51743) During graceful exit, async result writer will not exit and hang because it will try to pull data block the queue endlessly. AsyncResultWriter thread should exit when fragment mgr wants to exit. Thread 652 (Thread 0x7fd20c1b0700 (LWP 434882) "FragmentMgrAsyn"): #0 futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7fca3442f560, clockid=<optimized out>, expected=0, futex_word=0x61b000c009f8) at ../sysdeps/nptl/futex-internal.h:320 #1 __pthread_cond_wait_common (abstime=0x7fca3442f560, clockid=<optimized out>, mutex=0x61b000c009a8, cond=0x61b000c009d0) at pthread_cond_wait.c:520 #2 __pthread_cond_timedwait (cond=0x61b000c009d0, mutex=0x61b000c009a8, abstime=0x7fca3442f560) at pthread_cond_wait.c:665 #3 0x00005566ce61f350 in __gthread_cond_timedwait (__cond=0x61b000c009d0, __mutex=0x61b000c009a8, __abs_timeout=0x7fca3442f560) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/x86_64-linux-gnu/c++/11/bits/gthr-default.h:872 #4 std::__condvar::wait_until (this=0x61b000c009d0, __m=..., __abs_time=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_mutex.h:162 #5 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=<optimized out>, __lock=..., __atime=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/condition_variable:222 #6 0x00005566ce61f021 in std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=<optimized out>, __lock=..., __atime=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/condition_variable:135 #7 0x000055670b59b0be in std::condition_variable::wait_for<long, std::ratio<1l, 1l> > (this=0x61b000c009d0, __lock=..., __rtime=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/condition_variable:163 #8 doris::vectorized::AsyncResultWriter::process_block (this=<optimized out>, state=0x61e001b3c080, profile=<optimized out>) at /root/doris/be/src/vec/sink/writer/async_result_writer.cpp:149 #9 0x000055670b59eb1f in doris::vectorized::AsyncResultWriter::start_writer(doris::RuntimeState*, doris::RuntimeProfile*)::$_0::operator()() const (this=0x60400146b110) at /root/doris/be/src/vec/sink/writer/async_result_writer.cpp:103 #10 std::__invoke_impl<void, doris::vectorized::AsyncResultWriter::start_writer(doris::RuntimeState*, doris::RuntimeProfile*)::$_0&>(std::__invoke_other, doris::vectorized::AsyncResultWriter::start_writer(doris::RuntimeState*, doris::RuntimeProfile*)::$_0&) (__f=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61 apache#11 std::__invoke_r<void, doris::vectorized::AsyncResultWriter::start_writer(doris::RuntimeState*, doris::RuntimeProfile*)::$_0&>(doris::vectorized::AsyncResultWriter::start_writer(doris::RuntimeState*, doris::RuntimeProfile*)::$_0&) (__fn=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111 apache#12 std::_Function_handler<void (), doris::vectorized::AsyncResultWriter::start_writer(doris::RuntimeState*, doris::RuntimeProfile*)::$_0>::_M_invoke(std::_Any_data const&) (__functor=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291 apache#13 0x00005566d2f2270b in doris::ThreadPool::dispatch_thread (this=0x616000415580) at /root/doris/be/src/util/threadpool.cpp:615 apache#14 0x00005566d2ef973a in doris::Thread::supervise_thread (arg=0x611005af0540) at /root/doris/be/src/util/thread.cpp:468 apache#15 0x00007fd5c2034609 in start_thread (arg=<optimized out>) at pthread_create.c:477 apache#16 0x00007fd5c22e1133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
xy720
pushed a commit
that referenced
this pull request
Jul 23, 2025
### What problem does this PR solve? ### Problem after enable java support, be can not start correctly, it will hang on stack: ``` (gdb) bt #0 0x00007f5fb1e97ce6 in __futex_abstimed_wait_common () from /lib64/libc.so.6 #1 0x00007f5fb1e9a798 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libc.so.6 #2 0x00007f5fb2c98bf3 in os::PlatformEvent::park() () from /usr/lib/jvm/java-17//lib/server/libjvm.so #3 0x00007f5fb2c693a5 in ObjectMonitor::wait(long, bool, JavaThread*) () from /usr/lib/jvm/java-17//lib/server/libjvm.so #4 0x00007f5fb2e8b316 in ObjectSynchronizer::wait(Handle, long, JavaThread*) () from /usr/lib/jvm/java-17//lib/server/libjvm.so #5 0x00007f5fb2934a97 in JVM_MonitorWait () from /usr/lib/jvm/java-17//lib/server/libjvm.so #6 0x00007f5f9de245ba in ?? () #7 0x00007f5f49446158 in ?? () #8 0x00007f5faeb9fa00 in ?? () #9 0x00007ffc37f91178 in ?? () #10 0x00007f5f9de304bd in ?? () apache#11 0x00007ffc37f910a0 in ?? () apache#12 0x0000000000000000 in ?? () ``` jstack of be: ``` "main" #1 prio=5 os_prio=0 cpu=931.38ms elapsed=66.08s tid=0x00007fab8e12c400 nid=0x3e68aa in Object.wait() [0x00007ffdb3d4c000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait([email protected]/Native Method) - waiting on <0x00000000bd183b68> (a java.lang.ProcessImpl) at java.lang.Object.wait([email protected]/Object.java:338) at java.lang.ProcessImpl.waitFor([email protected]/ProcessImpl.java:434) - locked <0x00000000bd183b68> (a java.lang.ProcessImpl) at org.apache.hadoop.util.Shell.runCommand(Shell.java:1061) at org.apache.hadoop.util.Shell.run(Shell.java:957) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1282) at org.apache.hadoop.util.Shell.isSetsidSupported(Shell.java:853) at org.apache.hadoop.util.Shell.<clinit>(Shell.java:838) at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:79) at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1713) at org.apache.hadoop.security.SecurityUtil.setConfigurationInternal(SecurityUtil.java:103) at org.apache.hadoop.security.SecurityUtil.<clinit>(SecurityUtil.java:92) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:312) - locked <0x00000000bd5fd708> (a java.lang.Class for org.apache.hadoop.security.UserGroupInformation) at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:300) - locked <0x00000000bd5fd708> (a java.lang.Class for org.apache.hadoop.security.UserGroupInformation) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:575) at org.apache.hadoop.fs.viewfs.ViewFileSystem.<init>(ViewFileSystem.java:279) at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0([email protected]/Native Method) at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance([email protected]/NativeConstructorAccessorImpl.java:77) at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance([email protected]/DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstanceWithCaller([email protected]/Constructor.java:500) at java.lang.reflect.Constructor.newInstance([email protected]/Constructor.java:481) at java.util.ServiceLoader$ProviderImpl.newInstance([email protected]/ServiceLoader.java:789) at java.util.ServiceLoader$ProviderImpl.get([email protected]/ServiceLoader.java:729) at java.util.ServiceLoader$3.next([email protected]/ServiceLoader.java:1403) at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:3534) - locked <0x00000000826529f0> (a java.lang.Class for org.apache.hadoop.fs.FileSystem) ``` This problem comes from: apache#45287 after this fix, we could enable java support: apache#52412 ### Another Fix Method Add JAVA_OPTS `-Djdk.lang.processReaperUseDefaultStackSize=true` inside be.conf, also can fix this problem: https://bugs.openjdk.org/browse/JDK-8153057 From gemini: 
xy720
pushed a commit
that referenced
this pull request
Jul 23, 2025
…rflow error (apache#53018) ### What problem does this PR solve? Problem Summary: ``` ==2403213==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x506001097220 at pc 0x55837edd7fe9 bp 0x7f7efe2dc830 sp 0x7f7efe2dbff8 READ of size 63 at 0x506001097220 thread T1262 (brpc_light) #0 0x55837edd7fe8 in strlen (/mnt/disk8/zhangsida/doris/output/be/lib/doris_be+0x2c8d7fe8) (BuildId: 5acbe48773972a88) #1 0x5583bd3ff969 in fmt::v7::detail::buffer_appender<char> fmt::v7::detail::write<char, fmt::v7::detail::buffer_appender<char>>(fmt::v7::detail::buffer_appender<char>, char const*) (/mnt/disk8/zhangsida/doris/output/be/lib/doris_be+0x6aeff969) (BuildId: 5acbe48773972a88) #2 0x5583bd41640e in char const* fmt::v7::detail::parse_replacement_field<char, fmt::v7::detail::format_handler<fmt::v7::detail::buffer_appender<char>, char, fmt::v7::basic_format_context<fmt::v7::detail::buffer_appender<char>, char>>&>(char const*, char const*, fmt::v7::detail::format_handler<fmt::v7::detail::buffer_appender<char>, char, fmt::v7::basic_format_context<fmt::v7::detail::buffer_appender<char>, char>>&) (/mnt/disk8/zhangsida/doris/output/be/lib/doris_be+0x6af1640e) (BuildId: 5acbe48773972a88) #3 0x5583bd416c7f in void fmt::v7::detail::vformat_to<char>(fmt::v7::detail::buffer<char>&, fmt::v7::basic_string_view<char>, fmt::v7::basic_format_args<fmt::v7::basic_format_context<fmt::v7::detail::buffer_appender<fmt::v7::type_identity<char>::type>, fmt::v7::type_identity<char>::type>>, fmt::v7::detail::locale_ref) (/mnt/disk8/zhangsida/doris/output/be/lib/doris_be+0x6af16c7f) (BuildId: 5acbe48773972a88) #4 0x5583bd3f74aa in fmt::v7::detail::vformat[abi:cxx11](fmt::v7::basic_string_view<char>, fmt::v7::format_args) (/mnt/disk8/zhangsida/doris/output/be/lib/doris_be+0x6aef74aa) (BuildId: 5acbe48773972a88) #5 0x5583a6b4d3bb in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> fmt::v7::format<char [45], char const*&, char>(char const (&) [45], char const*&) /mnt/disk8/zhangsida/doris/thirdparty/installed/include/fmt/core.h:2076:10 #6 0x5583a6b4d3bb in doris::vectorized::BitmapFromBase64::vector(doris::vectorized::PODArray<unsigned char, 4096ul, doris::Allocator<false, false, false, doris::NoTrackingDefaultMemoryAllocator>, 16ul, 15ul> const&, doris::vectorized::PODArray<unsigned int, 4096ul, doris::Allocator<false, false, false, doris::NoTrackingDefaultMemoryAllocator>, 16ul, 15ul> const&, std::vector<doris::BitmapValue, std::allocator<doris::BitmapValue>>&, doris::vectorized::PODArray<unsigned char, 4096ul, doris::Allocator<false, false, false, doris::NoTrackingDefaultMemoryAllocator>, 16ul, 15ul>&, unsigned long) /mnt/disk8/zhangsida/doris/be/src/vec/functions/function_bitmap.cpp:310:29 #7 0x5583a6b4b93b in doris::vectorized::FunctionBitmapAlwaysNull<doris::vectorized::BitmapFromBase64>::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /mnt/disk8/zhangsida/doris/be/src/vec/functions/function_bitmap.cpp:381:13 #8 0x5583a02062ef in doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /mnt/disk8/zhangsida/doris/be/src/vec/functions/function.h:459:26 #9 0x5583a49d77e3 in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool) const /mnt/disk8/zhangsida/doris/be/src/vec/functions/function.cpp #10 0x5583a49d0bf8 in doris::vectorized::PreparedFunctionImpl::default_implementation_for_constant_arguments(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool, bool*) const /mnt/disk8/zhangsida/doris/be/src/vec/functions/function.cpp:168:5 apache#11 0x5583a49d3324 in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool) const /mnt/disk8/zhangsida/doris/be/src/vec/functions/function.cpp:237:5 ```
xy720
pushed a commit
that referenced
this pull request
Jul 23, 2025
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
1.fix LRU queue crash use after free
2.fix extra LRU queue info when 'need_to_move' flag unset
3.use concurrent queueu to record queueu change info for thread safety
```
ERROR: AddressSanitizer: heap-use-after-free on address 0x603005548c40 at pc 0x55f28e8c4785 bp 0x7f603582e1f0 sp 0x7f603582e1e8
READ of size 8 at 0x603005548c40 thread T201
#0 0x55f28e8c4784 in std::_Head_base<0ul, doris::io::CacheLRULog*, false>::_Head_base<doris::io::CacheLRULog*>(doris::io::CacheLRULog*&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/tuple:190:17
#1 0x55f28e8c4784 in std::_Tuple_impl<0ul, doris::io::CacheLRULog*, std::default_delete<doris::io::CacheLRULog>>::_Tuple_impl(std::_Tuple_impl<0ul, doris::io::CacheLRULog*, std::default_delete<doris::io::CacheLRULog>>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/tuple:292:2
#2 0x55f28e8c4784 in std::tuple<doris::io::CacheLRULog*, std::default_delete<doris::io::CacheLRULog>>::tuple(std::tuple<doris::io::CacheLRULog*, std::default_delete<doris::io::CacheLRULog>>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/tuple:1079:17
#3 0x55f28e8c4784 in std::_uniq_ptr_impl<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>::uniq_ptr_impl(std::_uniq_ptr_impl<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:162:9
#4 0x55f28e8c4784 in std::_uniq_ptr_data<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>, true, true>::uniq_ptr_data(std::_uniq_ptr_data<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>, true, true>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:211:7
#5 0x55f28e8c4784 in std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>::unique_ptr(std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:327:7
#6 0x55f28e8c4784 in doris::io::LRUQueueRecorder::replay_queue_event(doris::io::FileCacheType) /root/doris/be/src/io/cache/lru_queue_recorder.cpp:40:20
#7 0x55f28e82d620 in doris::io::BlockFileCache::run_background_lru_log_replay() /root/doris/be/src/io/cache/block_file_cache.cpp:2242:24
#8 0x55f2cdc2720f in execute_native_thread_routine /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc+-v3/src/c11/../../../../../libstdc-v3/src/c+11/thread.cc:82:18
#9 0x7f61f1842608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
#10 0x7f61f1aef132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
0x603005548c40 is located 16 bytes inside of 24-byte region [0x603005548c30,0x603005548c48)
freed by thread T201 here:
#0 0x55f28e51680d in operator delete(void*) (/home/work/unlimit_teamcity/TeamCity/Agents/20250708205944agent_172.16.0.48_1/work/60183217f6ee2a9c/output/be/lib/doris_be+0x3975a80d) (BuildId: 8b6ba6101e736655)
#1 0x55f28e8c3ce0 in std::__cxx11::list<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>, std::allocator<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>>::pop_front() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_list.h:1198:15
#2 0x55f28e8c3ce0 in doris::io::LRUQueueRecorder::replay_queue_event(doris::io::FileCacheType) /root/doris/be/src/io/cache/lru_queue_recorder.cpp:41:19
#3 0x55f28e82d620 in doris::io::BlockFileCache::run_background_lru_log_replay() /root/doris/be/src/io/cache/block_file_cache.cpp:2242:24
#4 0x55f2cdc2720f in execute_native_thread_routine /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc+-v3/src/c11/../../../../../libstdc-v3/src/c+11/thread.cc:82:18
previously allocated by thread T607 (CumuCompactionT) here:
#0 0x55f28e515fad in operator new(unsigned long) (/home/work/unlimit_teamcity/TeamCity/Agents/20250708205944agent_172.16.0.48_1/work/60183217f6ee2a9c/output/be/lib/doris_be+0x39759fad) (BuildId: 8b6ba6101e736655)
#1 0x55f28e8c660d in __gnu_cxx::new_allocator<std::_List_node<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>>::allocate(unsigned long, void const*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:121:27
#2 0x55f28e8c660d in std::allocator<std::_List_node<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>>::allocate(unsigned long) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:173:32
#3 0x55f28e8c660d in std::allocator_traits<std::allocator<std::_List_node<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>>>::allocate(std::allocator<std::_List_node<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>>&, unsigned long) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:460:20
#4 0x55f28e8c660d in std::__cxx11::_List_base<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>, std::allocator<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>>::_M_get_node() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_list.h:442:16
#5 0x55f28e8c660d in std::List_node<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>* std::_cxx11::list<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>, std::allocator<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>>::_M_create_node<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>(std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_list.h:634:21
#6 0x55f28e8c660d in void std::__cxx11::list<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>, std::allocator<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>>::_M_insert<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>(std::_List_iterator<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>, std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_list.h:1911:18
#7 0x55f28e8c3522 in std::__cxx11::list<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>, std::allocator<std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>>>::push_back(std::unique_ptr<doris::io::CacheLRULog, std::default_delete<doris::io::CacheLRULog>>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_list.h:1217:15
#8 0x55f28e8c3522 in doris::io::LRUQueueRecorder::record_queue_event(doris::io::FileCacheType, doris::io::CacheLRULogType, doris::io::UInt128Wrapper, unsigned long, unsigned long) /root/doris/be/src/io/cache/lru_queue_recorder.cpp:29:15
#9 0x55f28e82f09b in doris::io::BlockFileCache::use_cell(doris::io::BlockFileCache::FileBlockCell const&, std::__cxx11::list<std::shared_ptr<doris::io::FileBlock>, std::allocator<std::shared_ptr<doris::io::FileBlock>>>*, bool, std::lock_guard<std::mutex>&) /root/doris/be/src/io/cache/block_file_cache.cpp:380:20
#10 0x55f28e833d1b in doris::io::BlockFileCache::get_impl[abi:cxx11](doris::io::UInt128Wrapper const&, doris::io::CacheContext const&, doris::io::FileBlock::Range const&, std::lock_guard<std::mutex>&) /root/doris/be/src/io/cache/block_file_cache.cpp:572:13
apache#11 0x55f28e83b4ef in doris::io::BlockFileCache::get_or_set(doris::io::UInt128Wrapper const&, unsigned long, unsigned long, doris::io::CacheContext&) /root/doris/be/src/io/cache/block_file_cache.cpp:762:27
apache#12 0x55f28e7ffcee in doris::io::CachedRemoteFileReader::read_at_impl(unsigned long, doris::Slice, unsigned long*, doris::io::IOContext const*) /root/doris/be/src/io/cache/cached_remote_file_reader.cpp:191:21
apache#13 0x55f28e7f8017 in doris::io::FileReader::read_at(unsigned long, doris::Slice, unsigned long*, doris::io::IOContext const*) /root/doris/be/src/io/fs/file_reader.cpp:34:17
```
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [x] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
---------
Signed-off-by: zhengyu <[email protected]>
xy720
pushed a commit
that referenced
this pull request
Jul 23, 2025
==15185==ERROR: AddressSanitizer: heap-use-after-free on address 0x7da44ced96d0 at pc 0x55795dac5f46 bp 0x7bc26b9a3e50 sp 0x7bc26b9a3e48 READ of size 8 at 0x7da44ced96d0 thread T1357 #0 0x55795dac5f45 in std::_Function_base::_M_empty() const /usr/local/ldb-toolchain-v0.25/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:249:37 #1 0x55795dac5f45 in std::function<doris::Status ()>::operator()() const /usr/local/ldb-toolchain-v0.25/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:591:6 #2 0x55795dac5f45 in doris::cloud::bthread_fork_join(std::vector<std::function<doris::Status ()>, std::allocator<std::function<doris::Status ()>>> const&, int)::$_0::operator()() const /root/doris/be/src/cloud/cloud_meta_mgr.cpp:106:23 #3 0x55795dac5f45 in void std::_invoke_impl<void, doris::cloud::bthread_fork_join(std::vector<std::function<doris::Status ()>, std::allocator<std::function<doris::Status ()>>> const&, int)::$_0&>(std::_invoke_other, doris::cloud::bthread_fork_join(std::vector<std::function<doris::Status ()>, std::allocator<std::function<doris::Status ()>>> const&, int)::$_0&) /usr/local/ldb-toolchain-v0.25/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 #4 0x55795dac5f45 in std::enable_if<is_invocable_r_v<void, doris::cloud::bthread_fork_join(std::vector<std::function<doris::Status ()>, std::allocator<std::function<doris::Status ()>>> const&, int)::$0&>, void>::type std::_invoke_r<void, doris::cloud::bthread_fork_join(std::vector<std::function<doris::Status ()>, std::allocator<std::function<doris::Status ()>>> const&, int)::$_0&>(doris::cloud::bthread_fork_join(std::vector<std::function<doris::Status ()>, std::allocator<std::function<doris::Status ()>>> const&, int)::$_0&) /usr/local/ldb-toolchain-v0.25/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:113:2 #5 0x55795dac5f45 in std::_Function_handler<void (), doris::cloud::bthread_fork_join(std::vector<std::function<doris::Status ()>, std::allocator<std::function<doris::Status ()>>> const&, int)::$_0>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.25/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 #6 0x55795da78d1d in std::function<void ()>::operator()() const /usr/local/ldb-toolchain-v0.25/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 #7 0x55795da78d1d in doris::cloud::run_bthread_work(void*) /root/doris/be/src/cloud/cloud_meta_mgr.cpp:74:5 #8 0x55795f3da334 in bthread::TaskGroup::task_runner(long) (/home/work/unlimit_teamcity/TeamCity/Agents/20250718121603agent_172.16.0.202_1/work/60183217f6ee2a9c/output/be/lib/doris_be+0x50a4e334) #9 0x55795f3ccc80 in bthread_make_fcontext (/home/work/unlimit_teamcity/TeamCity/Agents/20250718121603agent_172.16.0.202_1/work/60183217f6ee2a9c/output/be/lib/doris_be+0x50a40c80) 0x7da44ced96d0 is located 2640 bytes inside of 2688-byte region [0x7da44ced8c80,0x7da44ced9700) freed by thread T1431 here: #0 0x5579380e1f12 in operator delete(void*, unsigned long) (/home/work/unlimit_teamcity/TeamCity/Agents/20250718121603agent_172.16.0.202_1/work/60183217f6ee2a9c/output/be/lib/doris_be+0x29755f12) previously allocated by thread T795 (brpc_light) here: #0 0x5579380e12ad in operator new(unsigned long) (/home/work/unlimit_teamcity/TeamCity/Agents/20250718121603agent_172.16.0.202_1/work/60183217f6ee2a9c/output/be/lib/doris_be+0x297552ad) #1 0x55795c7b57ab in std::__new_allocator<std::function<doris::Status ()>>::allocate(unsigned long, void const*) /usr/local/ldb-toolchain-v0.25/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/new_allocator.h:151:27 #2 0x55795c7b57ab in std::allocator<std::function<doris::Status ()>>::allocate(unsigned long) /usr/local/ldb-toolchain-v0.25/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/allocator.h:203:32 #3 0x55795c7b57ab in std::allocator_traits<std::allocator<std::function<doris::Status ()>>>::allocate(std::allocator<std::function<doris::Status ()>>&, unsigned long) /usr/local/ldb-toolchain-v0.25/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/alloc_traits.h:614:20 #4 0x55795c7b57ab in std::_Vector_base<std::function<doris::Status ()>, std::allocator<std::function<doris::Status ()>>>::_M_allocate(unsigned long) /usr/local/ldb-toolchain-v0.25/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_vector.h:387:20 #5 0x55795c7b57ab in std::vector<std::function<doris::Status ()>, std::allocator<std::function<doris::Status ()>>>::reserve(unsigned long) /usr/local/ldb-toolchain-v0.25/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/vector.tcc:79:22 #6 0x55795c7787f6 in doris::pipeline::OlapScanLocalState::_sync_cloud_tablets(doris::RuntimeState*) /root/doris/be/src/pipeline/exec/olap_scan_operator.cpp:466:20 #7 0x55795c777e7d in doris::pipeline::OlapScanLocalState::init(doris::RuntimeState*, doris::pipeline::LocalStateInfo&) /root/doris/be/src/pipeline/exec/olap_scan_operator.cpp:51:5 #8 0x55795b459bf9 in doris::pipeline::OperatorX<doris::pipeline::OlapScanLocalState>::setup_local_state(doris::RuntimeState*, doris::pipeline::LocalStateInfo&) /root/doris/be/src/pipeline/exec/operator.cpp:486:5 #9 0x55795d9c9a36 in doris::pipeline::PipelineTask::prepare(std::vector<doris::TScanRangeParams, std::allocator<doris::TScanRangeParams>> const&, int, doris::TDataSink const&) /root/doris/be/src/pipeline/pipeline_task.cpp:118:9
xy720
pushed a commit
that referenced
this pull request
Jul 27, 2025
…#53713) ### What problem does this PR solve? Problem Summary: memcpy(dst, &result, sizeof(result)); when use memcpy, it's size if sizeof(result), so use int64 maybe overflow of dst ``` ==3524968==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f18404e1d73 at pc 0x559e2c162b01 bp 0x7f18439e5dc0 sp 0x7f18439e5db8 WRITE of size 8 at 0x7f18404e1d73 thread T1265 (brpc_light) #0 0x559e2c162b00 in bool doris::vectorized::parse_ipv4<char const, doris::vectorized::parse_ipv6(char const*, char const*, unsigned char*)::'lambda'()>(char const*&, doris::vectorized::parse_ipv6(char const*, char const*, unsigned char*)::'lambda'(), unsigned char*, long) /mnt/disk8/zhangsida/doris/be/src/vec/common/format_ip.h:165:5 #1 0x559e2c161eb3 in bool doris::vectorized::parse_ipv6<char const, doris::vectorized::parse_ipv6(char const*, char const*, unsigned char*)::'lambda'()>(char const*&, doris::vectorized::parse_ipv6(char const*, char const*, unsigned char*)::'lambda'(), unsigned char*, int) /mnt/disk8/zhangsida/doris/be/src/vec/common/format_ip.h:416:18 #2 0x559e2c160c44 in doris::vectorized::parse_ipv6(char const*, char const*, unsigned char*) /mnt/disk8/zhangsida/doris/be/src/vec/common/format_ip.h:467:9 #3 0x559e2c160c44 in doris::vectorized::parse_ipv6_whole(char const*, char const*, unsigned char*) /mnt/disk8/zhangsida/doris/be/src/vec/common/format_ip.h:475:12 #4 0x559e2c160c44 in doris::IPv6Value::from_string(unsigned __int128&, char const*, unsigned long) /mnt/disk8/zhangsida/doris/be/src/vec/runtime/ipv6_value.h:71:16 #5 0x559e4fdb05f3 in doris::vectorized::FunctionToIP<(doris::vectorized::IPConvertExceptionMode)0, (doris::PrimitiveType)37>::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /mnt/disk8/zhangsida/doris/be/src/vec/functions/function_ip.h:1180:21 #6 0x559e4c233b1e in doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /mnt/disk8/zhangsida/doris/be/src/vec/functions/function.h:447:26 #7 0x559e4eebcef3 in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool) const /mnt/disk8/zhangsida/doris/be/src/vec/functions/function.cpp #8 0x559e4eeb68c4 in doris::vectorized::PreparedFunctionImpl::default_implementation_for_constant_arguments(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool, bool*) const /mnt/disk8/zhangsida/doris/be/src/vec/functions/function.cpp:168:5 #9 0x559e4eeb8fc4 in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long, bool) const /mnt/disk8/zhangsida/doris/be/src/vec/functions/function.cpp:237:5 ```
xy720
pushed a commit
that referenced
this pull request
Jul 30, 2025
### What problem does this PR solve? set NDEBUG_SANITIZER on asan mode ``` AddressSanitizer:DEADLYSIGNAL 18:05:08 ================================================================= 18:05:08 ==17515==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x563cea4a925f bp 0x7ffedd655da0 sp 0x7ffedd655d90 T0) 18:05:08 ==17515==The signal is caused by a READ memory access. 18:05:08 ==17515==Hint: this fault was caused by a dereference of a high value address (see register values below). Disassemble the provided pc to learn which register was used. 18:05:17 #0 0x563cea4a925f in absl::lts_20250512::container_internal::btree_node<absl::lts_20250512::container_internal::map_params<S2CellId, S2ShapeIndexCell*, std::less<S2CellId>, std::allocator<std::pair<S2CellId const, S2ShapeIndexCell*>>, 256, false>>::parent() const /var/local/thirdparty/installed/include/absl/container/internal/btree.h:714:39 18:05:17 #1 0x563cea4a9216 in absl::lts_20250512::container_internal::btree_node<absl::lts_20250512::container_internal::map_params<S2CellId, S2ShapeIndexCell*, std::less<S2CellId>, std::allocator<std::pair<S2CellId const, S2ShapeIndexCell*>>, 256, false>>::is_root() const /var/local/thirdparty/installed/include/absl/container/internal/btree.h:718:33 18:05:17 #2 0x563cea4a9fae in absl::lts_20250512::container_internal::btree_node<absl::lts_20250512::container_internal::map_params<S2CellId, S2ShapeIndexCell*, std::less<S2CellId>, std::allocator<std::pair<S2CellId const, S2ShapeIndexCell*>>, 256, false>>::get_root_generation() const /var/local/thirdparty/installed/include/absl/container/internal/btree.h:729:19 18:05:17 #3 0x563cea4a9b86 in absl::lts_20250512::container_internal::btree_node<absl::lts_20250512::container_internal::map_params<S2CellId, S2ShapeIndexCell*, std::less<S2CellId>, std::allocator<std::pair<S2CellId const, S2ShapeIndexCell*>>, 256, false>>::generation() const /var/local/thirdparty/installed/include/absl/container/internal/btree.h:735:41 18:05:17 #4 0x563cea4a90bb in void absl::lts_20250512::container_internal::btree_iterator_generation_info_enabled::assert_valid_generation<absl::lts_20250512::container_internal::btree_node<absl::lts_20250512::container_internal::map_params<S2CellId, S2ShapeIndexCell*, std::less<S2CellId>, std::allocator<std::pair<S2CellId const, S2ShapeIndexCell*>>, 256, false>>>(absl::lts_20250512::container_internal::btree_node<absl::lts_20250512::container_internal::map_params<S2CellId, S2ShapeIndexCell*, std::less<S2CellId>, std::allocator<std::pair<S2CellId const, S2ShapeIndexCell*>>, 256, false>> const*) const /var/local/thirdparty/installed/include/absl/container/internal/btree.h:1077:34 18:05:17 #5 0x563cea4b9771 in absl::lts_20250512::container_internal::btree_iterator<absl::lts_20250512::container_internal::btree_node<absl::lts_20250512::container_internal::map_params<S2CellId, S2ShapeIndexCell*, std::less<S2CellId>, std::allocator<std::pair<S2CellId const, S2ShapeIndexCell*>>, 256, false>> const, std::pair<S2CellId const, S2ShapeIndexCell*> const&, std::pair<S2CellId const, S2ShapeIndexCell*> const*>::increment() /var/local/thirdparty/installed/include/absl/container/internal/btree.h:1296:5 18:05:17 #6 0x563cea4b94fa in absl::lts_20250512::container_internal::btree_iterator<absl::lts_20250512::container_internal::btree_node<absl::lts_20250512::container_internal::map_params<S2CellId, S2ShapeIndexCell*, std::less<S2CellId>, std::allocator<std::pair<S2CellId const, S2ShapeIndexCell*>>, 256, false>> const, std::pair<S2CellId const, S2ShapeIndexCell*> const&, std::pair<S2CellId const, S2ShapeIndexCell*> const*>::operator++() /var/local/thirdparty/installed/include/absl/container/internal/btree.h:1215:5 18:05:17 #7 0x563cea4a7dbd in MutableS2ShapeIndex::Iterator::Next() /var/local/thirdparty/installed/include/s2/mutable_s2shape_index.h:720:3 18:05:17 #8 0x563d0901dada (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4b466ada) 18:05:17 #9 0x563d0901eeac in s2shapeutil::FindSelfIntersection(S2ShapeIndex const&, S2Error*) (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4b467eac) 18:05:17 #10 0x563d08fa4fbf in S2Loop::IsValid() const (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4b3edfbf) 18:05:17 apache#11 0x563cea498899 in doris::to_s2loop(doris::GeoCoordinateList const&, std::unique_ptr<S2Loop, std::default_delete<S2Loop>>*) /root/doris/be/src/geo/geo_types.cpp:340:19 18:05:17 apache#12 0x563cea4891cf in doris::to_s2polygon(doris::GeoCoordinateListList const&, std::unique_ptr<S2Polygon, std::default_delete<S2Polygon>>*) /root/doris/be/src/geo/geo_types.cpp:374:20 18:05:17 apache#13 0x563cea488ee4 in doris::GeoPolygon::from_coords(doris::GeoCoordinateListList const&) /root/doris/be/src/geo/geo_types.cpp:786:12 18:05:17 apache#14 0x563cea46d9be in wkt_parse(WktParseContext*) /root/doris/be/src/geo/wkt_yacc.y:130:38 18:05:17 apache#15 0x563cea45fefc in doris::WktParse::parse_wkt(char const*, unsigned long, doris::GeoShape**) /root/doris/be/src/geo/wkt_parse.cpp:37:16 18:05:17 apache#16 0x563cea4814c2 in doris::GeoShape::from_wkt(char const*, unsigned long, doris::GeoParseStatus*) /root/doris/be/src/geo/geo_types.cpp:407:15 18:05:17 apache#17 0x563cda7df4ed in doris::GeoTypesTest_point_intersects_Test::TestBody() /root/doris/be/test/geo/geo_types_test.cpp:108:13 18:05:17 apache#18 0x563d07bf5023 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4a03e023) 18:05:17 apache#19 0x563d07be4245 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4a02d245) 18:05:17 apache#20 0x563d07bbed66 in testing::Test::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4a007d66) 18:05:17 apache#21 0x563d07bbfa4c in testing::TestInfo::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4a008a4c) 18:05:17 apache#22 0x563d07bc01df in testing::TestSuite::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4a0091df) 18:05:17 apache#23 0x563d07bd199b in testing::internal::UnitTestImpl::RunAllTests() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4a01a99b) 18:05:17 apache#24 0x563d07bf6e53 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4a03fe53) 18:05:17 apache#25 0x563d07be6465 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4a02f465) 18:05:17 apache#26 0x563d07bd152a in testing::UnitTest::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x4a01a52a) 18:05:17 apache#27 0x563cdd9d3322 in RUN_ALL_TESTS() /var/local/thirdparty/installed/include/gtest/gtest.h:2490:46 18:05:17 apache#28 0x563cdd9c689e in main /root/doris/be/test/testutil/run_all_tests.cpp:111:19 18:05:17 apache#29 0x7f73ae7a4554 in __libc_start_main (/lib64/libc.so.6+0x22554) (BuildId: 1a8fb61bb4614a483833d5334202ab50edda2a25) 18:05:17 apache#30 0x563cd9b0e029 in _start (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x1bf57029) ```
xy720
pushed a commit
that referenced
this pull request
Jul 30, 2025
) ### What problem does this PR solve? In branch-2.0, we already refractor these codes in pr: apache#18590 Deadlock stack: 1、While load, we alloc MemTracker and need lock TrackerGroup.group_lock ``` NodeChannel::NodeChannel _node_channel_tracker = std::make_shared<MemTracker> MemTracker::bind_parent std::lock_guard<std::mutex> l(mem_tracker_pool[_parent_group_num].group_lock); _tracker_group_it = mem_tracker_pool[_parent_group_num].trackers.insert( mem_tracker_pool[_parent_group_num].trackers.end(), this); ``` 2、but while we try to call std::list::insert, we need alloc std::_List_node，here new_hook (in file tcmalloc_hook.h) is triggered, then we lock the same TrackerGroup.group_lock, make it deadlock ``` new_hook doris::ThreadMemTrackerMgr::consume doris::ThreadMemTrackerMgr::flush_untracked_mem<true, true> doris::ThreadMemTrackerMgr::exceeded doris::MemTrackerLimiter::print_log_usage doris::MemTracker::make_group_snapshot std::lock_guard<std::mutex> l(mem_tracker_pool[group_num].group_lock); ``` Full stack info: ``` (gdb) bt #0 0x00007f219772454d in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f219771fe9b in _L_lock_883 () from /lib64/libpthread.so.0 #2 0x00007f219771fd68 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x0000563ca7a8a824 in __gthread_mutex_lock (__mutex=0x563cb18cbc58) at /var/local/ldb-toolchain/include/x86_64-linux-gnu/c++/11/bits/gthr-default.h:749 #4 std::mutex::lock (this=0x563cb18cbc58) at /var/local/ldb-toolchain/include/c++/11/bits/std_mutex.h:100 #5 std::lock_guard<std::mutex>::lock_guard (__m=..., this=<synthetic pointer>) at /var/local/ldb-toolchain/include/c++/11/bits/std_mutex.h:229 #6 doris::MemTracker::make_group_snapshot (snapshots=0x7f1f7fe06ea0, group_num=<optimized out>, parent_label=...) at /data/TCHouse-D-1.2/be/src/runtime/memory/mem_tracker.cpp:115 #7 0x0000563ca7a7eeb4 in doris::MemTrackerLimiter::print_log_usage (this=0x5640ab7956c0, msg=...) at /data/TCHouse-D-1.2/be/src/runtime/memory/mem_tracker_limiter.cpp:198 #8 0x0000563ca7a8d1e4 in doris::ThreadMemTrackerMgr::exceeded (this=this@entry=0x563ce6a2c820, size=1048584) at /data/TCHouse-D-1.2/be/src/runtime/memory/thread_mem_tracker_mgr.cpp:59 #9 0x0000563ca78cfeb4 in doris::ThreadMemTrackerMgr::flush_untracked_mem<true, true> (this=0x563ce6a2c820) at /data/TCHouse-D-1.2/be/src/runtime/memory/thread_mem_tracker_mgr.h:223 #10 doris::ThreadMemTrackerMgr::consume (size=<optimized out>, this=0x563ce6a2c820) at /data/TCHouse-D-1.2/be/src/runtime/memory/thread_mem_tracker_mgr.h:188 apache#11 doris::ThreadMemTrackerMgr::consume (size=<optimized out>, this=0x563ce6a2c820) at /data/TCHouse-D-1.2/be/src/runtime/memory/thread_mem_tracker_mgr.h:178 apache#12 new_hook (ptr=<optimized out>, size=24) at /data/TCHouse-D-1.2/be/src/runtime/memory/tcmalloc_hook.h:39 apache#13 0x0000563caf3dfa78 in MallocHook::InvokeNewHookSlow (p=p@entry=0x56422d713b40, s=s@entry=24) at src/malloc_hook.cc:498 apache#14 0x0000563caf55f2c1 in MallocHook::InvokeNewHook (s=24, p=0x56422d713b40) at src/malloc_hook-inl.h:127 apache#15 tcmalloc::do_allocate_full<tcmalloc::cpp_throw_oom> (size=size@entry=24) at src/tcmalloc.cc:1805 apache#16 tcmalloc::allocate_full_cpp_throw_oom (size=size@entry=24) at src/tcmalloc.cc:1815 apache#17 0x0000563caf55f429 in tcmalloc::dispatch_allocate_full<tcmalloc::cpp_throw_oom> (size=24) at src/tcmalloc.cc:1822 apache#18 0x0000563ca7a8ab9a in __gnu_cxx::new_allocator<std::_List_node<doris::MemTracker*> >::allocate (__n=1, this=0x563cb18cbc40) at /var/local/ldb-toolchain/include/c++/11/ext/new_allocator.h:103 apache#19 std::allocator_traits<std::allocator<std::_List_node<doris::MemTracker*> > >::allocate (__n=1, __a=...) at /var/local/ldb-toolchain/include/c++/11/bits/alloc_traits.h:460 apache#20 std::__cxx11::_List_base<doris::MemTracker*, std::allocator<doris::MemTracker*> >::_M_get_node (this=0x563cb18cbc40) at /var/local/ldb-toolchain/include/c++/11/bits/stl_list.h:442 apache#21 std::__cxx11::list<doris::MemTracker*, std::allocator<doris::MemTracker*> >::_M_create_node<doris::MemTracker*> (this=0x563cb18cbc40) at /var/local/ldb-toolchain/include/c++/11/bits/stl_list.h:634 apache#22 std::__cxx11::list<doris::MemTracker*, std::allocator<doris::MemTracker*> >::emplace<doris::MemTracker*> (__position=..., this=0x563cb18cbc40) at /var/local/ldb-toolchain/include/c++/11/bits/list.tcc:92 apache#23 std::__cxx11::list<doris::MemTracker*, std::allocator<doris::MemTracker*> >::insert (__x=<optimized out>, __position=..., this=0x563cb18cbc40) at /var/local/ldb-toolchain/include/c++/11/bits/stl_list.h:1309 apache#24 doris::MemTracker::bind_parent (this=0x563eff46ad10, parent=<optimized out>) at /data/TCHouse-D-1.2/be/src/runtime/memory/mem_tracker.cpp:79 apache#25 0x0000563ca7a8b608 in doris::MemTracker::MemTracker (this=this@entry=0x563eff46ad10, label=..., parent=parent@entry=0x0) at /data/TCHouse-D-1.2/be/src/runtime/memory/mem_tracker.cpp:66 apache#26 0x0000563ca7967467 in __gnu_cxx::new_allocator<doris::MemTracker>::construct<doris::MemTracker, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (__p=0x563eff46ad10, this=<optimized out>) at /var/local/ldb-toolchain/include/c++/11/ext/new_allocator.h:154 apache#27 std::allocator_traits<std::allocator<doris::MemTracker> >::construct<doris::MemTracker, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (__p=0x563eff46ad10, __a=...) at /var/local/ldb-toolchain/include/c++/11/bits/alloc_traits.h:512 apache#28 std::_Sp_counted_ptr_inplace<doris::MemTracker, std::allocator<doris::MemTracker>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (__a=..., this=0x563eff46ad00) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr_base.h:519 apache#29 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<doris::MemTracker, std::allocator<doris::MemTracker>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (__a=..., __p=<optimized out>, this=<optimized out>) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr_base.h:650 apache#30 std::__shared_ptr<doris::MemTracker, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<doris::MemTracker>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (__tag=..., this=<optimized out>) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr_base.h:1337 apache#31 std::shared_ptr<doris::MemTracker>::shared_ptr<std::allocator<doris::MemTracker>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (__tag=..., this=<optimized out>) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr.h:409 apache#32 std::allocate_shared<doris::MemTracker, std::allocator<doris::MemTracker>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (__a=...) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr.h:861 apache#33 std::make_shared<doris::MemTracker, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > () at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr.h:877 apache#34 doris::stream_load::NodeChannel::NodeChannel (this=this@entry=0x563d0ca7e110, parent=<optimized out>, index_channel=index_channel@entry=0x563d4b43f200, node_id=<optimized out>) at /data/TCHouse-D-1.2/be/src/exec/tablet_sink.cpp:49 apache#35 0x0000563cac74a82d in doris::stream_load::VNodeChannel::VNodeChannel (this=this@entry=0x563d0ca7e110, parent=<optimized out>, index_channel=index_channel@entry=0x563d4b43f200, node_id=<optimized out>) at /data/TCHouse-D-1.2/be/src/vec/sink/vtablet_sink.cpp:37 apache#36 0x0000563ca7967eaa in __gnu_cxx::new_allocator<doris::stream_load::VNodeChannel>::construct<doris::stream_load::VNodeChannel, doris::stream_load::OlapTableSink*&, doris::stream_load::IndexChannel*, long&> (__p=0x563d0ca7e110, this=<optimized out>) at /var/local/ldb-toolchain/include/c++/11/ext/new_allocator.h:154 apache#37 std::allocator_traits<std::allocator<doris::stream_load::VNodeChannel> >::construct<doris::stream_load::VNodeChannel, doris::stream_load::OlapTableSink*&, doris::stream_load::IndexChannel*, long&> (__p=0x563d0ca7e110, __a=...) at /var/local/ldb-toolchain/include/c++/11/bits/alloc_traits.h:512 apache#38 std::_Sp_counted_ptr_inplace<doris::stream_load::VNodeChannel, std::allocator<doris::stream_load::VNodeChannel>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<doris::stream_load::OlapTableSink*&, doris::stream_load::IndexChannel*, long&> (__a=..., this=0x563d0ca7e100) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr_base.h:519 apache#39 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<doris::stream_load::VNodeChannel, std::allocator<doris::stream_load::VNodeChannel>, doris::stream_load::OlapTableSink*&, doris::stream_load::IndexChannel*, long&> (__a=..., __p=<optimized out>, this=<optimized out>) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr_base.h:650 apache#40 std::__shared_ptr<doris::stream_load::VNodeChannel, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<doris::stream_load::VNodeChannel>, doris::stream_load::OlapTableSink*&, doris::stream_load::IndexChannel*, long&> (__tag=..., this=<optimized out>) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr_base.h:1337 apache#41 std::shared_ptr<doris::stream_load::VNodeChannel>::shared_ptr<std::allocator<doris::stream_load::VNodeChannel>, doris::stream_load::OlapTableSink*&, doris::stream_load::IndexChannel*, long&> (__tag=..., this=<optimized out>) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr.h:409 apache#42 std::allocate_shared<doris::stream_load::VNodeChannel, std::allocator<doris::stream_load::VNodeChannel>, doris::stream_load::OlapTableSink*&, doris::stream_load::IndexChannel*, long&> (__a=...) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr.h:861 apache#43 std::make_shared<doris::stream_load::VNodeChannel, doris::stream_load::OlapTableSink*&, doris::stream_load::IndexChannel*, long&> () at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr.h:877 apache#44 doris::stream_load::IndexChannel::init (this=this@entry=0x563d4b43f200, state=state@entry=0x563e027a3500, tablets=...) at /data/TCHouse-D-1.2/be/src/exec/tablet_sink.cpp:705 apache#45 0x0000563ca79698d8 in doris::stream_load::OlapTableSink::prepare (this=this@entry=0x5644b9efe880, state=state@entry=0x563e027a3500) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr_base.h:1290 apache#46 0x0000563cac74db65 in doris::stream_load::VOlapTableSink::prepare (this=0x5644b9efe880, state=0x563e027a3500) at /data/TCHouse-D-1.2/be/src/vec/sink/vtablet_sink.cpp:450 apache#47 0x0000563ca7946c21 in doris::PlanFragmentExecutor::prepare (this=this@entry=0x563f55efd280, request=..., fragments_ctx=<optimized out>) at /var/local/ldb-toolchain/include/c++/11/bits/unique_ptr.h:173 apache#48 0x0000563ca791f17c in doris::FragmentExecState::prepare (this=this@entry=0x563f55efd200, params=...) --Type <RET> for more, q to quit, c to continue without paging-- at /data/TCHouse-D-1.2/be/src/runtime/fragment_mgr.cpp:238 apache#49 0x0000563ca7926629 in doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams const&, std::function<void (doris::PlanFragmentExecutor*)>) (this=this@entry=0x563cb6e93400, params=..., cb=...) at /data/TCHouse-D-1.2/be/src/runtime/fragment_mgr.cpp:720 apache#50 0x0000563ca7928dab in doris::FragmentMgr::exec_plan_fragment (this=0x563cb6e93400, params=...) at /data/TCHouse-D-1.2/be/src/runtime/fragment_mgr.cpp:564 apache#51 0x0000563ca7aaf507 in doris::PInternalServiceImpl::_exec_plan_fragment_impl (this=this@entry=0x563cb577b880, ser_request=..., version=<optimized out>, compact=<optimized out>) at /data/TCHouse-D-1.2/be/src/service/internal_service.cpp:480 apache#52 0x0000563ca7aaf703 in doris::PInternalServiceImpl::_exec_plan_fragment_in_pthread (this=0x563cb577b880, controller=<optimized out>, request=0x563d6c5bb9b0, response=0x564203b92ce0, done=0x563d885a60c0) at /data/TCHouse-D-1.2/be/src/service/internal_service.cpp:254 apache#53 0x0000563ca78d6dbd in std::function<void ()>::operator()() const (this=0x7f1f7fe090c8) at /var/local/ldb-toolchain/include/c++/11/bits/std_function.h:560 apache#54 doris::PriorityThreadPool::work_thread (this=0x563cb577ba38, thread_id=<optimized out>) at /data/TCHouse-D-1.2/be/src/util/priority_thread_pool.hpp:145 apache#55 0x0000563caf526b00 in std::execute_native_thread_routine (__p=0x563cbfc81d10) at ../../../../../libstdc++-v3/src/c++11/thread.cc:82 apache#56 0x00007f219771dea5 in start_thread () from /lib64/libpthread.so.0 apache#57 0x00007f2197a309fd in clone () from /lib64/libc.so.6 ``` ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
xy720
pushed a commit
that referenced
this pull request
Aug 17, 2025
…O0 (apache#54672) ### What problem does this PR solve? Because on the pipeline we use O1 for compilation, but some UBSan errors only appear locally when compiling with O0, for example: ``` 0x7ba502eb3a01: note: pointer points here 02 00 00 00 30 52 8c 02 95 7b 00 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ^ #0 0x55cbe9f9acd2 in doris::Slice::compare(doris::Slice const&) const /mnt/disk8/yanxuecheng/tmp-doris/be/src/util/slice.h #1 0x55cbede3405c in doris::FieldTypeTraits<(doris::FieldType)13>::cmp(void const*, void const*) /mnt/disk8/yanxuecheng/tmp-doris/be/src/olap/types.h:1347:25 #2 0x55cbe9e98dae in doris::ScalarTypeInfo::cmp(void const*, void const*) const /mnt/disk8/yanxuecheng/tmp-doris/be/src/olap/types.h:104:74 #3 0x55cbeee73cf0 in int doris::Field::compare_cell<doris::RowCursorCell, doris::RowCursorCell>(doris::RowCursorCell const&, doris::RowCursorCell const&) const /mnt/disk8/yanxuecheng/tmp-doris/be/src/olap/field.h:120:41 #4 0x55cbeee6a030 in doris::compare_row_key(doris::RowCursor const&, doris::RowCursor const&) /mnt/disk8/yanxuecheng/tmp-doris/be/src/olap/tablet_reader.h:75:47 #5 0x55cbeee43237 in doris::TabletReader::_capture_rs_readers(doris::TabletReader::ReaderParams const&) /mnt/disk8/yanxuecheng/tmp-doris/be/src/olap/tablet_reader.cpp:188:17 #6 0x55cc1d5f7985 in doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/olap/block_reader.cpp:106:16 #7 0x55cc1d5fd004 in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/olap/block_reader.cpp:226:19 #8 0x55cc15e09f53 in doris::vectorized::OlapScanner::open(doris::RuntimeState*) /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/exec/scan/olap_scanner.cpp:260:32 #9 0x55cc15d7d89d in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/exec/scan/scanner_scheduler.cpp:182:5 #10 0x55cc15d88ce6 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/exec/scan/scanner_scheduler.cpp:96:17 apache#11 0x55cc15d884b1 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/exec/scan/scanner_scheduler.cpp:95:27 apache#12 0x55cc15d8829c in bool std::__invoke_impl<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&) /mnt/disk6/common/ldb_toolchain_taipan/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 apache#13 0x55cc15d8819c in std::enable_if<is_invocable_r_v<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&) /mnt/disk6/common/ldb_toolchain_taipan/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 apache#14 0x55cc15d87c94 in std::_Function_handler<bool (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /mnt/disk6/common/ldb_toolchain_taipan/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 apache#15 0x55cbe9632efd in std::function<bool ()>::operator()() const /mnt/disk6/common/ldb_toolchain_taipan/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 apache#16 0x55cc15d87314 in doris::vectorized::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l>>) /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/exec/scan/scanner_scheduler.cpp:407:25 ```
xy720
pushed a commit
that referenced
this pull request
Aug 18, 2025
…4737) Related PR: apache#54598 before 54598, logf is using impl from toolchain, after 54598, logf is using impl from doris glibc-compatibility Analysis from gpt5: ``` the 0x1ff << 23 part is evaluated in type int. 0x1ff = 511 decimal. Left-shifting it 23 bits gives 0x3ff800000 in math, which is way bigger than INT_MAX (on 32-bit signed int, any shift producing a bit in the sign position or beyond is undefined behavior in C). Even if ix and tmp are larger types, that constant expression is first computed as an int before any promotions, triggering the UB warning from UBSan. ``` Fix the problem below ```text #0 0x5632a618a17 in logf / root/doris/be/src/glibc-compatibility/musl/logf. c: 56:25 #1 0x5632a59fec26 in std:: log(float) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/cmath: 334:12 #2 0x5632a59fcee3 in lucene:: index: :MultiLevelSkipListWriter::MultiLevelSkipListWriter(int, int, int) /root/doris/contrib/clucene/src/core/CLucene/index/SkipListWriter.cpp:70:56 #3 0x5632a59fe497 in lucene:: index: :DefaultSkipListWriter: :DefaultSkipListWriter(int, int, int, lucene::store::IndexOutput*, lucene::store: :IndexOutput*) /root/doris/contrib/clucene/src/core/CLucene/index/SkipListWriter. cpp:175:11 + echo 'cp -r /mnt/ssd01/pipline/0penSourceDoris/clusterEnv/PO//Cluster0/fe/bin /home/work/pipline/backup_center/54276_0b92341b39685a8528a1d769885cefe95197e7fb_p0/fe/' + cp -r /mnt/ssd01/pipline/0penSourceDoris/clusterEnv/PO//Cluster0/fe/conf /home/work/pipline/backup_center/54276_0b92341b39685a8528a1d769885cefe95197e7fb_p0/fe/ #4 0x5632a5affdb in lucene:: index:: SDocumentsWriter<char>: :writeSegment(std: :vector<std::__cxx11::basic_string<char, std: :char_traits<char>, std: :allocator<char> >, std: :allocator<std::_cxx11::basic_string<char, std:: char_traits<chars #5 0x5632a5af009e in lucene:: index::DocumentsWriter<char>:: flush(bool) /root/doris/contrib/clucene/src/core/Lucene/index/DocumentWriter. cpp: 1340:9 #6 0x5632a5a41842 in lucene:: index:: IndexWriter:: doFlush(bool) /root/doris/contrib/clucene/src/core/CLucene/index/IndexWriter. cpp:2262:58 #7 0x5632a5a1f59b in lucene:: index:: IndexWriter:: flush(bool, bool) / root/doris/contrib/clucene/src/core/CLucene/index/IndexWriter.cpp:2154:9 #8 0x5632a5a1dd09 in lucene:: index:: IndexWriter:: closeInternal(bool) /root/doris/contrib/clucene/src/core/CLucene/index/IndexWriter. cpp: 546:9 #9 0x5632a5a1d16e in lucene:: index:: IndexWriter:: close(bool) / root/doris/contrib/clucene/src/core/CLucene/index/IndexWriter.cpp: 519:9 #10 0x563282c28c76 in _ZN5doris10segment_v213finally_closeISt10unique_ptrIN6lucene5index11IndexWriterESt14default_deleteIS5_EEQ8HasCloseIT_EEEVRS9_RNS0_12ErrorContextE /root/doris/be/src/olap/rowset/segment_v2/inverted_index_common.h:56 apache#11 0x563282c3b9c2 in doris::segment_v2: :InvertedIndexColumnWriter<(doris::FieldType)17>::finish) /root/doris/be/src/olap/rowset/segment_v2/inverted index_writer. cpp: 687:9 apache#12 0x563282b9fble in doris:: segment_v2::ArrayColumnWriter::write_inverted_index() /root/doris/be/src/olap/rowset/segment_v2/column_writer.cpp: 940:41 apache#13 0x56328248ec in doris:: segment_v2::VerticalSegmentWriter::_write_inverted_index() /root/doris/be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp: 1296:9 + echo cp -r /mnt/ssd01/pipline/0penSourceDoris/clusterEnv/P0//Cluster0/fe/log /home/work/pipline/backup_center/54276_0b92341b39685a8528a1d769885cefe95197e7fb_p0/fe/' apache#14 0x5632824d120 in doris::segment_v2::VerticalSegmentWriter::finalize_columns_indexunsigned long*) / root/doris/be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp: 1204:5 + cp -r /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0//Cluster0/fe/log /home/work/pipline/backup_center/54276_0b92341b39685a8528a1d769885cefe95197e7fb_p0/fe/ apache#15 0x56328251046 in doris: :segment_v2: :VerticalSegmentWriter:: finalize(unsigned long*, unsigned long*) / root/doris/be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp: 1253:5 apache#16 0x563282519a5a in doris: SegmentFlusher: :_flush_segment_writer(std: :unique_ptr<doris::segment_v2::VerticalSegmentWriter, std: :default_delete<doris::segment_v2: :VerticalSegmentWriter> >&, std: :shared_ptr<doris:: TabletSchema>, long*) apache#17 0x56328251642f in doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*) /root/doris/be/src/olap/rowset/segment_creator.cpp:77:9 apache#18 0x563282521151 in doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*) /root/doris/be/src/olap/rowset/segment_creator.cpp: 389:5 apache#19 0x56329fe59dc4 in doris::BetaRowsetWriterV2:: flush_memtable(doris::vectorized::Block*, int, long*) /root/doris/be/src/olap/rowset/beta_rowset_writer_v2.cpp: 109:9 apache#20 0x56328215778 in doris:: FlushToken: :_do_flush_memtable(doris::MemTable*, int, long*) / root/doris/be/src/olap/memtable_flush_executor.cpp:210:9 apache#21 apache#22 0x56328217962 in doris:: FlushToken: :_flush_memtable(std: :shared_ptr<doris::MemTable>, int, long) / root/doris/be/src/olap/memtable_flush_executor. cpp:265:16 0x5632821f1f7d in doris::MemtableFlushTask:: run() /root/doris/be/src/olap/memtable_flush_executor.cpp:63:20 apache#23 0x56328437d33 in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:614:24 apache#24 0x56328435аабс in std:: function<void ()›::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 apache#25 0x56328435aac in doris::Thread:: supervise_thread(void*) / root/doris/be/src/util/thread. cpp: 461:5 apache#26 0x56327fc4fd26 in asan_thread_start(void*) (/mnt/ssd01/pipline/0penSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x28da4d26) apache#27 0x7fdd62967608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c: 477:8 apache#28 0x7fdd6287a132 in _clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S: 95 ```
xy720
pushed a commit
that referenced
this pull request
Sep 4, 2025
TaskQueue should be released after all theads stop.
```
=================================================================
==6632==ERROR: AddressSanitizer: heap-use-after-free on address 0x61700518d440 at pc 0x5616fa5c2a87 bp 0x7f7635b709b0 sp 0x7f7635b709a8
READ of size 8 at 0x61700518d440 thread T5188 (Pipe_test_follo)
#0 0x5616fa5c2a86 in std::__atomic_base<unsigned long>::load(std::memory_order) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x5616fa5c2a86 in std::__atomic_base<unsigned long>::operator unsigned long() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:341:16
#2 0x5616fa5c2a86 in doris::pipeline::PriorityTaskQueue::_try_take_unprotected(bool) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/task_queue.cpp:58:9
#3 0x5616fa5c2e16 in doris::pipeline::PriorityTaskQueue::take(unsigned int) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/task_queue.cpp:110:16
#4 0x5616fa5c4b06 in doris::pipeline::MultiCoreTaskQueue::take(int) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/task_queue.cpp:170:51
#5 0x5616fa5d1919 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/task_scheduler.cpp:267:35
#6 0x5616d1e169f8 in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/threadpool.cpp:544:24
#7 0x5616d1df3e38 in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
#8 0x5616d1df3e38 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/thread.cpp:498:5
#9 0x7f83a4c35ac2 in start_thread nptl/pthread_create.c:442:8
#10 0x7f83a4cc784f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
0x61700518d440 is located 704 bytes inside of 728-byte region [0x61700518d180,0x61700518d458)
freed by thread T2166 here:
#0 0x5616cea38d9d in operator delete(void*) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x16dc0d9d) (BuildId: 09b7b6856fee4f7f)
#1 0x5616fa5cae77 in std::default_delete<doris::pipeline::PriorityTaskQueue>::operator()(doris::pipeline::PriorityTaskQueue*) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2
#2 0x5616fa5cae77 in std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>::~unique_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4
#3 0x5616fa5cae77 in void std::destroy_at<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>>(std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
#4 0x5616fa5cae77 in void std::_Destroy<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>>(std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7
#5 0x5616fa5cae77 in void std::_Destroy_aux<false>::__destroy<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>*>(std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>*, std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6
#6 0x5616fa5cae77 in void std::_Destroy<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>*>(std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>*, std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7
#7 0x5616fa5cae77 in void std::_Destroy<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>*, std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>>(std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>*, std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>*, std::allocator<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>>&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7
#8 0x5616fa5cae77 in std::vector<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>, std::allocator<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>>>::~vector() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680:2
#9 0x5616fa5c466d in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
#10 0x5616fa5c466d in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:11
apache#11 0x5616fa5c466d in std::__shared_ptr<std::vector<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>, std::allocator<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>>>, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31
apache#12 0x5616fa5c466d in void std::atomic_store<std::vector<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>, std::allocator<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>>>>(std::shared_ptr<std::vector<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>, std::allocator<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>>>>*, std::shared_ptr<std::vector<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>, std::allocator<std::unique_ptr<doris::pipeline::PriorityTaskQueue, std::default_delete<doris::pipeline::PriorityTaskQueue>>>>>) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_atomic.h:149:7
apache#13 0x5616fa5c466d in doris::pipeline::MultiCoreTaskQueue::close() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/task_queue.cpp:151:5
apache#14 0x5616fa5ce9f4 in doris::pipeline::TaskScheduler::stop() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/task_scheduler.cpp:437:26
apache#15 0x5616d19b7d36 in doris::WorkloadGroup::try_stop_schedulers() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/workload_group/workload_group.cpp:654:22
apache#16 0x5616d19c6042 in doris::WorkloadGroupMgr::delete_workload_group_by_ids(std::set<unsigned long, std::less<unsigned long>, std::allocator<unsigned long>>) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/workload_group/workload_group_manager.cpp:107:13
apache#17 0x5616d1a7f6a1 in doris::WorkloadGroupListener::handle_topic_info(std::vector<doris::TopicInfo, std::allocator<doris::TopicInfo>> const&) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/agent/workload_group_listener.cpp:82:38
apache#18 0x5616d1a7dd7e in doris::TopicSubscriber::handle_topic_info(doris::TPublishTopicRequest const&) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/agent/topic_subscriber.cpp:45:35
apache#19 0x5616d2002932 in doris::BackendServiceProcessor::process_publish_topic_info(int, apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, void*) /home/zcp/repo_center/doris_branch-2.1/doris/gensrc/build/gen_cpp/BackendService.cpp:6577:13
apache#20 0x5616d1fd6f11 in doris::BackendServiceProcessor::dispatchCall(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, int, void*) /home/zcp/repo_center/doris_branch-2.1/doris/gensrc/build/gen_cpp/BackendService.cpp:5416:3
apache#21 0x5616d2039186 in apache::thrift::TDispatchProcessor::process(std::shared_ptr<apache::thrift::protocol::TProtocol>, std::shared_ptr<apache::thrift::protocol::TProtocol>, void*) /home/zcp/repo_center/doris_branch-2.1/doris/thirdparty/installed/include/thrift/TDispatchProcessor.h:121:12
apache#22 0x5616faee19f7 in apache::thrift::server::TConnectedClient::run() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x432699f7) (BuildId: 09b7b6856fee4f7f)
apache#23 0x5616faee2cf6 in apache::thrift::server::TThreadedServer::TConnectedClientRunner::run() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326acf6) (BuildId: 09b7b6856fee4f7f)
apache#24 0x5616faee830d in apache::thrift::concurrency::Thread::threadMain(std::shared_ptr<apache::thrift::concurrency::Thread>) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4327030d) (BuildId: 09b7b6856fee4f7f)
apache#25 0x5616faee81b3 in void std::__invoke_impl<void, void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>(std::__invoke_other, void (*&&)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>&&) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x432701b3) (BuildId: 09b7b6856fee4f7f)
apache#26 0x5616faee8106 in std::__invoke_result<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>::type std::__invoke<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>(void (*&&)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>&&) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x43270106) (BuildId: 09b7b6856fee4f7f)
apache#27 0x5616faee8076 in void std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x43270076) (BuildId: 09b7b6856fee4f7f)
apache#28 0x5616faee7ffb in std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>>::operator()() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326fffb) (BuildId: 09b7b6856fee4f7f)
apache#29 0x5616faee7f9f in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>>>::_M_run() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326ff9f) (BuildId: 09b7b6856fee4f7f)
apache#30 0x5616fd80568f in execute_native_thread_routine /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:82:18
previously allocated by thread T3392 here:
#0 0x5616cea3853d in operator new(unsigned long) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x16dc053d) (BuildId: 09b7b6856fee4f7f)
#1 0x5616fa5c3e29 in std::_MakeUniq<doris::pipeline::PriorityTaskQueue>::__single_object std::make_unique<doris::pipeline::PriorityTaskQueue>() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:962:30
#2 0x5616fa5c3e29 in doris::pipeline::MultiCoreTaskQueue::MultiCoreTaskQueue(int) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/task_queue.cpp:139:39
#3 0x5616d19b276d in decltype(::new((void*)(0)) doris::pipeline::MultiCoreTaskQueue(std::declval<int&>())) std::construct_at<doris::pipeline::MultiCoreTaskQueue, int&>(doris::pipeline::MultiCoreTaskQueue*, int&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:97:39
#4 0x5616d19b276d in void std::allocator_traits<std::allocator<doris::pipeline::MultiCoreTaskQueue>>::construct<doris::pipeline::MultiCoreTaskQueue, int&>(std::allocator<doris::pipeline::MultiCoreTaskQueue>&, doris::pipeline::MultiCoreTaskQueue*, int&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:514:4
#5 0x5616d19b276d in std::_Sp_counted_ptr_inplace<doris::pipeline::MultiCoreTaskQueue, std::allocator<doris::pipeline::MultiCoreTaskQueue>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<int&>(std::allocator<doris::pipeline::MultiCoreTaskQueue>, int&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:519:4
#6 0x5616d19b276d in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<doris::pipeline::MultiCoreTaskQueue, std::allocator<doris::pipeline::MultiCoreTaskQueue>, int&>(doris::pipeline::MultiCoreTaskQueue*&, std::_Sp_alloc_shared_tag<std::allocator<doris::pipeline::MultiCoreTaskQueue>>, int&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:651:6
#7 0x5616d19b276d in std::__shared_ptr<doris::pipeline::MultiCoreTaskQueue, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<doris::pipeline::MultiCoreTaskQueue>, int&>(std::_Sp_alloc_shared_tag<std::allocator<doris::pipeline::MultiCoreTaskQueue>>, int&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1337:14
#8 0x5616d19b276d in std::shared_ptr<doris::pipeline::MultiCoreTaskQueue>::shared_ptr<std::allocator<doris::pipeline::MultiCoreTaskQueue>, int&>(std::_Sp_alloc_shared_tag<std::allocator<doris::pipeline::MultiCoreTaskQueue>>, int&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:409:4
#9 0x5616d19b276d in std::shared_ptr<doris::pipeline::MultiCoreTaskQueue> std::allocate_shared<doris::pipeline::MultiCoreTaskQueue, std::allocator<doris::pipeline::MultiCoreTaskQueue>, int&>(std::allocator<doris::pipeline::MultiCoreTaskQueue> const&, int&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:860:14
#10 0x5616d19b276d in std::shared_ptr<doris::pipeline::MultiCoreTaskQueue> std::make_shared<doris::pipeline::MultiCoreTaskQueue, int&>(int&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:876:14
apache#11 0x5616d19b276d in doris::WorkloadGroup::upsert_thread_pool_no_lock(doris::WorkloadGroupInfo*, std::shared_ptr<doris::CgroupCpuCtl>, doris::ExecEnv*) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/workload_group/workload_group.cpp:462:27
apache#12 0x5616d19b5991 in doris::WorkloadGroup::upsert_task_scheduler(doris::WorkloadGroupInfo*, doris::ExecEnv*) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/workload_group/workload_group.cpp:572:9
apache#13 0x5616d1a7ecd8 in doris::WorkloadGroupListener::handle_topic_info(std::vector<doris::TopicInfo, std::allocator<doris::TopicInfo>> const&) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/agent/workload_group_listener.cpp:60:13
apache#14 0x5616d1a7dd7e in doris::TopicSubscriber::handle_topic_info(doris::TPublishTopicRequest const&) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/agent/topic_subscriber.cpp:45:35
apache#15 0x5616d2002932 in doris::BackendServiceProcessor::process_publish_topic_info(int, apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, void*) /home/zcp/repo_center/doris_branch-2.1/doris/gensrc/build/gen_cpp/BackendService.cpp:6577:13
apache#16 0x5616d1fd6f11 in doris::BackendServiceProcessor::dispatchCall(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, int, void*) /home/zcp/repo_center/doris_branch-2.1/doris/gensrc/build/gen_cpp/BackendService.cpp:5416:3
apache#17 0x5616d2039186 in apache::thrift::TDispatchProcessor::process(std::shared_ptr<apache::thrift::protocol::TProtocol>, std::shared_ptr<apache::thrift::protocol::TProtocol>, void*) /home/zcp/repo_center/doris_branch-2.1/doris/thirdparty/installed/include/thrift/TDispatchProcessor.h:121:12
apache#18 0x5616faee19f7 in apache::thrift::server::TConnectedClient::run() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x432699f7) (BuildId: 09b7b6856fee4f7f)
apache#19 0x5616faee2cf6 in apache::thrift::server::TThreadedServer::TConnectedClientRunner::run() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326acf6) (BuildId: 09b7b6856fee4f7f)
apache#20 0x5616faee830d in apache::thrift::concurrency::Thread::threadMain(std::shared_ptr<apache::thrift::concurrency::Thread>) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4327030d) (BuildId: 09b7b6856fee4f7f)
apache#21 0x5616faee81b3 in void std::__invoke_impl<void, void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>(std::__invoke_other, void (*&&)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>&&) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x432701b3) (BuildId: 09b7b6856fee4f7f)
apache#22 0x5616faee8106 in std::__invoke_result<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>::type std::__invoke<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>(void (*&&)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>&&) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x43270106) (BuildId: 09b7b6856fee4f7f)
apache#23 0x5616faee8076 in void std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x43270076) (BuildId: 09b7b6856fee4f7f)
apache#24 0x5616faee7ffb in std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>>::operator()() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326fffb) (BuildId: 09b7b6856fee4f7f)
apache#25 0x5616faee7f9f in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>>>::_M_run() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326ff9f) (BuildId: 09b7b6856fee4f7f)
apache#26 0x5616fd80568f in execute_native_thread_routine /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:82:18
Thread T5188 (Pipe_test_follo) created by T3392 here:
#0 0x5616ce9e5caa in pthread_create (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x16d6dcaa) (BuildId: 09b7b6856fee4f7f)
#1 0x5616d1df2cc9 in doris::Thread::start_thread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::function<void ()> const&, unsigned long, scoped_refptr<doris::Thread>*) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/thread.cpp:449:15
#2 0x5616d1e1d979 in doris::Status doris::Thread::create<void (doris::ThreadPool::*)(), doris::ThreadPool*>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, void (doris::ThreadPool::* const&)(), doris::ThreadPool* const&, scoped_refptr<doris::Thread>*) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/thread.h:56:16
#3 0x5616d1e1423d in doris::ThreadPool::create_thread() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/threadpool.cpp:612:12
#4 0x5616d1e13e43 in doris::ThreadPool::init() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/threadpool.cpp:266:25
#5 0x5616cec31535 in doris::Status doris::ThreadPoolBuilder::build<doris::ThreadPool>(std::unique_ptr<doris::ThreadPool, std::default_delete<doris::ThreadPool>>*) const /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/threadpool.h:121:13
#6 0x5616fa5cef22 in doris::pipeline::TaskScheduler::start() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/task_scheduler.cpp:208:5
#7 0x5616d19b2a9b in doris::WorkloadGroup::upsert_thread_pool_no_lock(doris::WorkloadGroupInfo*, std::shared_ptr<doris::CgroupCpuCtl>, doris::ExecEnv*) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/workload_group/workload_group.cpp:467:47
#8 0x5616d19b5991 in doris::WorkloadGroup::upsert_task_scheduler(doris::WorkloadGroupInfo*, doris::ExecEnv*) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/workload_group/workload_group.cpp:572:9
#9 0x5616d1a7ecd8 in doris::WorkloadGroupListener::handle_topic_info(std::vector<doris::TopicInfo, std::allocator<doris::TopicInfo>> const&) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/agent/workload_group_listener.cpp:60:13
#10 0x5616d1a7dd7e in doris::TopicSubscriber::handle_topic_info(doris::TPublishTopicRequest const&) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/agent/topic_subscriber.cpp:45:35
apache#11 0x5616d2002932 in doris::BackendServiceProcessor::process_publish_topic_info(int, apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, void*) /home/zcp/repo_center/doris_branch-2.1/doris/gensrc/build/gen_cpp/BackendService.cpp:6577:13
apache#12 0x5616d1fd6f11 in doris::BackendServiceProcessor::dispatchCall(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, int, void*) /home/zcp/repo_center/doris_branch-2.1/doris/gensrc/build/gen_cpp/BackendService.cpp:5416:3
apache#13 0x5616d2039186 in apache::thrift::TDispatchProcessor::process(std::shared_ptr<apache::thrift::protocol::TProtocol>, std::shared_ptr<apache::thrift::protocol::TProtocol>, void*) /home/zcp/repo_center/doris_branch-2.1/doris/thirdparty/installed/include/thrift/TDispatchProcessor.h:121:12
apache#14 0x5616faee19f7 in apache::thrift::server::TConnectedClient::run() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x432699f7) (BuildId: 09b7b6856fee4f7f)
apache#15 0x5616faee2cf6 in apache::thrift::server::TThreadedServer::TConnectedClientRunner::run() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326acf6) (BuildId: 09b7b6856fee4f7f)
apache#16 0x5616faee830d in apache::thrift::concurrency::Thread::threadMain(std::shared_ptr<apache::thrift::concurrency::Thread>) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4327030d) (BuildId: 09b7b6856fee4f7f)
apache#17 0x5616faee81b3 in void std::__invoke_impl<void, void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>(std::__invoke_other, void (*&&)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>&&) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x432701b3) (BuildId: 09b7b6856fee4f7f)
apache#18 0x5616faee8106 in std::__invoke_result<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>::type std::__invoke<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>(void (*&&)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>&&) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x43270106) (BuildId: 09b7b6856fee4f7f)
apache#19 0x5616faee8076 in void std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x43270076) (BuildId: 09b7b6856fee4f7f)
apache#20 0x5616faee7ffb in std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>>::operator()() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326fffb) (BuildId: 09b7b6856fee4f7f)
apache#21 0x5616faee7f9f in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<apache::thrift::concurrency::Thread>), std::shared_ptr<apache::thrift::concurrency::Thread>>>>::_M_run() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326ff9f) (BuildId: 09b7b6856fee4f7f)
apache#22 0x5616fd80568f in execute_native_thread_routine /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:82:18
Thread T3392 created by T933 here:
#0 0x5616ce9e5caa in pthread_create (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x16d6dcaa) (BuildId: 09b7b6856fee4f7f)
#1 0x5616fd8057b5 in __gthread_create /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:663:35
#2 0x5616fd8057b5 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State>>, void (*)()) /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:147:37
#3 0x5616faee667c in apache::thrift::concurrency::Thread::start() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326e67c) (BuildId: 09b7b6856fee4f7f)
#4 0x5616faee2a26 in apache::thrift::server::TThreadedServer::onClientConnected(std::shared_ptr<apache::thrift::server::TConnectedClient> const&) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326aa26) (BuildId: 09b7b6856fee4f7f)
#5 0x5616faedebc4 in apache::thrift::server::TServerFramework::newlyConnectedClient(std::shared_ptr<apache::thrift::server::TConnectedClient> const&) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x43266bc4) (BuildId: 09b7b6856fee4f7f)
#6 0x5616faede25f in apache::thrift::server::TServerFramework::serve() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326625f) (BuildId: 09b7b6856fee4f7f)
#7 0x5616faee2791 in apache::thrift::server::TThreadedServer::serve() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326a791) (BuildId: 09b7b6856fee4f7f)
#8 0x5616d1e31419 in doris::ThriftServer::ThriftServerEventProcessor::supervise() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/thrift_server.cpp:206:34
#9 0x5616fd80568f in execute_native_thread_routine /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:82:18
Thread T933 created by T0 here:
#0 0x5616ce9e5caa in pthread_create (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x16d6dcaa) (BuildId: 09b7b6856fee4f7f)
#1 0x5616fd8057b5 in __gthread_create /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:663:35
#2 0x5616fd8057b5 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State>>, void (*)()) /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:147:37
#3 0x5616d1e37788 in doris::ThriftServer::start() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/thrift_server.cpp:421:5
#4 0x5616cea3f174 in main /home/zcp/repo_center/doris_branch-2.1/doris/be/src/service/doris_main.cpp:536:25
#5 0x7f83a4bcad8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
Thread T2166 created by T933 here:
#0 0x5616ce9e5caa in pthread_create (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x16d6dcaa) (BuildId: 09b7b6856fee4f7f)
#1 0x5616fd8057b5 in __gthread_create /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:663:35
#2 0x5616fd8057b5 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State>>, void (*)()) /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:147:37
#3 0x5616faee667c in apache::thrift::concurrency::Thread::start() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326e67c) (BuildId: 09b7b6856fee4f7f)
#4 0x5616faee2a26 in apache::thrift::server::TThreadedServer::onClientConnected(std::shared_ptr<apache::thrift::server::TConnectedClient> const&) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326aa26) (BuildId: 09b7b6856fee4f7f)
#5 0x5616faedebc4 in apache::thrift::server::TServerFramework::newlyConnectedClient(std::shared_ptr<apache::thrift::server::TConnectedClient> const&) (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x43266bc4) (BuildId: 09b7b6856fee4f7f)
#6 0x5616faede25f in apache::thrift::server::TServerFramework::serve() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326625f) (BuildId: 09b7b6856fee4f7f)
#7 0x5616faee2791 in apache::thrift::server::TThreadedServer::serve() (/mnt/hdd01/ci/doris-deploy-branch-2.1-local/be/lib/doris_be+0x4326a791) (BuildId: 09b7b6856fee4f7f)
#8 0x5616d1e31419 in doris::ThriftServer::ThriftServerEventProcessor::supervise() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/thrift_server.cpp:206:34
#9 0x5616fd80568f in execute_native_thread_routine /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:82:18
SUMMARY: AddressSanitizer: heap-use-after-free /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9 in std::__atomic_base<unsigned long>::load(std::memory_order) const
Shadow bytes around the buggy address:
0x61700518d180: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x61700518d200: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x61700518d280: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x61700518d300: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x61700518d380: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x61700518d400: fd fd fd fd fd fd fd fd[fd]fd fd fa fa fa fa fa
0x61700518d480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x61700518d500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x61700518d580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x61700518d600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x61700518d680: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==6632==ABORTING
```
xy720
pushed a commit
that referenced
this pull request
Sep 7, 2025
…pache#55346) ### What problem does this PR solve? Related PR: apache#53729 Problem Summary: runtime error: member access within null pointer of type 'FileReaderStats' in `TracingFileReader` ``` /home/zcp/repo_center/doris_branch-3.1/doris/be/src/io/fs/tracing_file_reader.h:34:9: runtime error: member access within null pointer of type 'FileReaderStats' #0 0x55c028481ace in doris::io::TracingFileReader::read_at_impl(unsigned long, doris::Slice, unsigned long*, doris::io::IOContext const*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/io/fs/tracing_file_reader.h:34:9 #1 0x55c00922a1f7 in doris::io::FileReader::read_at(unsigned long, doris::Slice, unsigned long*, doris::io::IOContext const*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/io/fs/file_reader.cpp:34:17 #2 0x55c028ae8a18 in doris::vectorized::parse_thrift_footer(std::shared_ptr<doris::io::FileReader>, doris::vectorized::FileMetaData**, unsigned long*, doris::io::IOContext*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/vec/exec/format/parquet/parquet_thrift_util.h:44:5 #3 0x55c028ae8a18 in doris::vectorized::ParquetReader::_open_file() /home/zcp/repo_center/doris_branch-3.1/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:234:21 #4 0x55c028aeb69c in doris::vectorized::ParquetReader::init_reader(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>> const&, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::variant<doris::ColumnValueRange<(doris::PrimitiveType)3>, doris::ColumnValueRange<(doris::PrimitiveType)4>, doris::ColumnValueRange<(doris::PrimitiveType)5>, doris::ColumnValueRange<(doris::PrimitiveType)6>, doris::ColumnValueRange<(doris::PrimitiveType)7>, doris::ColumnValueRange<(doris::PrimitiveType)36>, doris::ColumnValueRange<(doris::PrimitiveType)37>, doris::ColumnValueRange<(doris::PrimitiveType)15>, doris::ColumnValueRange<(doris::PrimitiveType)10>, doris::ColumnValueRange<(doris::PrimitiveType)23>, doris::ColumnValueRange<(doris::PrimitiveType)11>, doris::ColumnValueRange<(doris::PrimitiveType)25>, doris::ColumnValueRange<(doris::PrimitiveType)12>, doris::ColumnValueRange<(doris::PrimitiveType)26>, doris::ColumnValueRange<(doris::PrimitiveType)20>, doris::ColumnValueRange<(doris::PrimitiveType)2>, doris::ColumnValueRange<(doris::PrimitiveType)19>, doris::ColumnValueRange<(doris::PrimitiveType)28>, doris::ColumnValueRange<(doris::PrimitiveType)29>, doris::ColumnValueRange<(doris::PrimitiveType)30>, doris::ColumnValueRange<(doris::PrimitiveType)35>>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const, std::variant<doris::ColumnValueRange<(doris::PrimitiveType)3>, doris::ColumnValueRange<(doris::PrimitiveType)4>, doris::ColumnValueRange<(doris::PrimitiveType)5>, doris::ColumnValueRange<(doris::PrimitiveType)6>, doris::ColumnValueRange<(doris::PrimitiveType)7>, doris::ColumnValueRange<(doris::PrimitiveType)36>, doris::ColumnValueRange<(doris::PrimitiveType)37>, doris::ColumnValueRange<(doris::PrimitiveType)15>, doris::ColumnValueRange<(doris::PrimitiveType)10>, doris::ColumnValueRange<(doris::PrimitiveType)23>, doris::ColumnValueRange<(doris::PrimitiveType)11>, doris::ColumnValueRange<(doris::PrimitiveType)25>, doris::ColumnValueRange<(doris::PrimitiveType)12>, doris::ColumnValueRange<(doris::PrimitiveType)26>, doris::ColumnValueRange<(doris::PrimitiveType)20>, doris::ColumnValueRange<(doris::PrimitiveType)2>, doris::ColumnValueRange<(doris::PrimitiveType)19>, doris::ColumnValueRange<(doris::PrimitiveType)28>, doris::ColumnValueRange<(doris::PrimitiveType)29>, doris::ColumnValueRange<(doris::PrimitiveType)30>, doris::ColumnValueRange<(doris::PrimitiveType)35>>>>> const*, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext>>> const&, doris::TupleDescriptor const*, doris::RowDescriptor const*, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, int, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const, int>>> const*, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext>>> const*, std::unordered_map<int, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext>>>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext>>>>>> const*, std::shared_ptr<doris::vectorized::TableSchemaChangeHelper::Node>, bool) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:318:5 #5 0x55c00cceda85 in doris::PushBrokerReader::_get_next_reader() /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/push_handler.cpp:671:39 #6 0x55c00cce72df in doris::PushBrokerReader::next(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/push_handler.cpp:434:9 #7 0x55c00cce27b3 in doris::PushHandler::_convert_v2(std::shared_ptr<doris::Tablet>, std::shared_ptr<doris::Rowset>*, std::shared_ptr<doris::TabletSchema>, doris::PushType) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/push_handler.cpp:293:31 #8 0x55c00ccdd92d in doris::PushHandler::_do_streaming_ingestion(std::shared_ptr<doris::Tablet>, doris::TPushReq const&, doris::PushType, std::vector<doris::TTabletInfo, std::allocator<doris::TTabletInfo>>*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/push_handler.cpp:199:11 #9 0x55c00ccd96f3 in doris::PushHandler::process_streaming_ingestion(std::shared_ptr<doris::Tablet>, doris::TPushReq const&, doris::PushType, std::vector<doris::TTabletInfo, std::allocator<doris::TTabletInfo>>*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/push_handler.cpp:94:11 #10 0x55c00cccff9c in doris::EngineBatchLoadTask::_push(doris::TPushReq const&, std::vector<doris::TTabletInfo, std::allocator<doris::TTabletInfo>>*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/task/engine_batch_load_task.cpp:293:28 apache#11 0x55c00cccbbc3 in doris::EngineBatchLoadTask::_process() /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/task/engine_batch_load_task.cpp:249:18 apache#12 0x55c00ccc6851 in doris::EngineBatchLoadTask::execute() /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/task/engine_batch_load_task.cpp:83:22 apache#13 0x55c0090293eb in doris::push_callback(doris::StorageEngine&, doris::TAgentTaskRequest const&) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/agent/task_worker_pool.cpp:1880:31 apache#14 0x55c008ff6554 in doris::PriorTaskWorkerPool::normal_loop() /home/zcp/repo_center/doris_branch-3.1/doris/be/src/agent/task_worker_pool.cpp:687:9 apache#15 0x55c00db29237 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/util/thread.cpp:498:5 apache#16 0x7fa0651f2ac2 in start_thread nptl/pthread_create.c:442:8 apache#17 0x7fa06528484f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 ```
xy720
pushed a commit
that referenced
this pull request
Oct 16, 2025
#9 0x000055e72fad6219 in google::LogMessageFatal::~LogMessageFatal() () #10 0x000055e72a3f01fc in doris::ThreadMemTrackerMgr::limiter_mem_tracker_sptr (this=0x7f38233fa000) at /mnt/disk2/doris/be/src/runtime/memory/thread_mem_tracker_mgr.h:112 apache#11 0x000055e72b33efa0 in doris::SwitchThreadMemTrackerLimiter::SwitchThreadMemTrackerLimiter (this=0x7f3796375df7, mem_tracker=...) at /mnt/disk2/doris/be/src/runtime/thread_context.cpp:88 apache#12 0x000055e72a457f0b in doris::io::FileBufferBuilder::build (this=0x7f3796375e30, buf=0x7f3823375438) at /mnt/disk2/doris/be/src/io/fs/s3_file_bufferpool.cpp:243 apache#13 0x000055e72a44f448 in doris::io::S3FileWriter::_build_upload_buffer (this=<optimized out>) at /mnt/disk2/doris/be/src/io/fs/s3_file_writer.cpp:196 apache#14 0x000055e72a44cd71 in doris::io::S3FileWriter::appendv (this=0x7f3823375300, data=0x7f3823392000, data_cnt=200) at /mnt/disk2/doris/be/src/io/fs/s3_file_writer.cpp:248 apache#15 0x000055e72a355f02 in MicrobenchS3FileWriter::appendv (this=0x7f3823375300, slices=slices@entry=0x7f3823392000, slices_size=slices_size@entry=200, write_bvar=...) at /mnt/disk2/doris/be/src/io/tools/file_cache_microbench.cpp:303 apache#16 0x000055e72a3557d4 in JobManager::execute_write_tasks(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, Job&, JobConfig const&)::{lambda()#1}::operator()() const (this=0x7f3819eb1d38) at /mnt/disk2/doris/be/src/io/tools/file_cache_microbench.cpp:1090 Signed-off-by: zhengyu <[email protected]>
xy720
pushed a commit
that referenced
this pull request
Oct 16, 2025
…armup` due to capture by reference (apache#56395) ### What problem does this PR solve? introduced in apache#54611 ``` (gdb) bt #0 __GI___pthread_sigmask (how=2, newmask=<optimized out>, oldmask=0x0) at ./nptl/pthread_sigmask.c:43 #1 0x00007f7aa6b5d71e in PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #2 0x00007f7aa6b5e206 in JVM_handle_linux_signal () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #3 <signal handler called> #4 0x000055e20050d443 in std::_Hashtable<doris::RowsetId, std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo>, std::allocator<std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo> >, std::__detail::_Select1st, std::equal_to<doris::RowsetId>, std::hash<doris::RowsetId>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_before_node (this=0x7f79e69acde0, __bkt=4186920012728959759, __k=..., __code=11200260987994981938) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/hashtable.h:2205 #5 std::_Hashtable<doris::RowsetId, std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo>, std::allocator<std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo> >, std::__detail::_Select1st, std::equal_to<doris::RowsetId>, std::hash<doris::RowsetId>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_locate (this=this@entry=0x7f79e69acde0, __k=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/hashtable.h:2283 #6 0x000055e200505a8d in std::_Hashtable<doris::RowsetId, std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo>, std::allocator<std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo> >, std::__detail::_Select1st, std::equal_to<doris::RowsetId>, std::hash<doris::RowsetId>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::find (this=0x7f79e69acde0, __k=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/hashtable.h:1929 #7 std::unordered_map<doris::RowsetId, doris::CloudTablet::RowsetWarmUpInfo, std::hash<doris::RowsetId>, std::equal_to<doris::RowsetId>, std::allocator<std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo> > >::contains (this=0x7f79e69acde0, __x=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/unordered_map.h:999 #8 doris::CloudTablet::complete_rowset_segment_warmup (this=<optimized out>, rowset_id=..., status=..., segment_num=1, inverted_idx_num=0) at /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_tablet.cpp:1692 #9 0x000055e2005338ba in doris::CloudWarmUpManager::handle_jobs()::$_0::operator()(doris::Status) const (this=0x7f79e8bebad0, st=...) at /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_warm_up_manager.cpp:243 #10 std::__invoke_impl<void, doris::CloudWarmUpManager::handle_jobs()::$_0&, doris::Status>(std::__invoke_other, doris::CloudWarmUpManager::handle_jobs()::$_0&, doris::Status&&) (__f=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63 apache#11 std::__invoke_r<void, doris::CloudWarmUpManager::handle_jobs()::$_0&, doris::Status>(doris::CloudWarmUpManager::handle_jobs()::$_0&, doris::Status&&) (__fn=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:113 apache#12 std::_Function_handler<void (doris::Status), doris::CloudWarmUpManager::handle_jobs()::$_0>::_M_invoke(std::_Any_data const&, doris::Status&&) (__functor=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292 apache#13 0x000055e200533310 in std::function<void (doris::Status)>::operator()(doris::Status) const (this=0x7f795a3ff1d0, __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#14 doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0::operator()(doris::Status) const ( this=0x7f79f0710b60, st=...) at /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_warm_up_manager.cpp:149 apache#15 std::__invoke_impl<void, doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0&, doris::Status>(std::__invoke_other, doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0&, doris::Status&&) ( __f=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63 apache#16 std::__invoke_r<void, doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0&, doris::Status>(doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0&, doris::Status&&) (__fn=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:113 apache#17 std::_Function_handler<void (doris::Status), doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0>::_M_invoke(std::_Any_data const&, doris::Status&&) (__functor=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292 apache#18 0x000055e200469cb1 in std::function<void (doris::Status)>::operator()(doris::Status) const (this=0x7f796941cee0, __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#19 doris::io::FileCacheBlockDownloader::download_segment_file (this=<optimized out>, meta=...) at /home/zcp/repo_center/doris_master/doris/be/src/io/cache/block_file_cache_downloader.cpp:297 apache#20 0x000055e1fbfcbe85 in doris::ThreadPool::dispatch_thread (this=0x7f7995a1da00) at /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:614 apache#21 0x000055e1fbfc0fac in std::function<void ()>::operator()() const (this=0x7f796941cee0) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#22 doris::Thread::supervise_thread (arg=0x7f79f13de010) at /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:460 apache#23 0x00007f7aa5bf8ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 apache#24 0x00007f7aa5c8a850 in __closefrom_fallback (from=1685031872, dirfd_fallback=<optimized out>) at ../sysdeps/unix/sysv/linux/closefrom_fallback.c:45 apache#25 0x0000000000000000 in ?? () ``` Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
xy720
pushed a commit
that referenced
this pull request
Oct 16, 2025
```
/home/zcp/repo_center/doris_master/doris/be/src/util/counts.h:127:85: runtime error: signed integer overflow: 147483648 - -2147483648 cannot be represented in type 'int'
#0 0x558c5a9d58cd in doris::Counts::terminate(double) /home/zcp/repo_center/doris_master/doris/be/src/util/counts.h:127:85
#1 0x558c5ab0ce99 in doris::vectorized::PercentileState<(doris::PrimitiveType)5>::insert_result_into(doris::vectorized::IColumn&) const /home/zcp/repo_center/doris_master/doris/be/src/vec/aggregate_functions/aggregate_function_percentile.h:430:49
#2 0x558c5ab0812f in doris::vectorized::AggregateFunctionPercentileArray<(doris::PrimitiveType)5>::insert_result_into(char const*, doris::vectorized::IColumn&) const /home/zcp/repo_center/doris_master/doris/be/src/vec/aggregate_functions/aggregate_function_percentile.h:557:59
#3 0x558c5ab086a8 in doris::vectorized::IAggregateFunctionHelper>::insert_result_into_vec(std::vector> const&, unsigned long, doris::vectorized::IColumn&, unsigned long) const /home/zcp/repo_center/doris_master/doris/be/src/vec/aggregate_functions/aggregate_function.h:393:22
#4 0x558c68af8445 in _ZZN5doris8pipeline13AggLocalState31_get_with_serialized_key_resultEPNS_12RuntimeStateEPNS_10vectorized5BlockEPbENK3$_1clINS4_15MethodKeysFixedI9PHHashMapIN4wide7integerILm256EjEEPc9HashCRC32ISE_EEEEEEvRT_ /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/aggregation_source_operator.cpp:287:67
#5 0x558c68af8445 in void std::__invoke_impl, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>&>(std::__invoke_other, doris::vectorized::Overload&&, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14
#6 0x558c68af8445 in std::__invoke_result, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>&>::type std::__invoke, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>&>(doris::vectorized::Overload&&, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:98:14
#7 0x558c68af8445 in std::__detail::__variant::__gen_vtable_impl (*)(doris::vectorized::Overload&&, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&)>, std::integer_sequence>::__visit_invoke(doris::vectorized::Overload&&, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/variant:1055:11
#8 0x558c68a85790 in decltype(auto) std::__do_visit, doris::vectorized::Overload, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&>(doris::vectorized::Overload&&, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/variant:1858:15
#9 0x558c68a85790 in std::invoke_result, std::__conditional>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&>>::type>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&>()))>::type>::type&, std::variant_alternative<0ul, std::remove_reference>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&>()))>::type>::type&&>>::type std::visit, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&>(doris::vectorized::Overload&&, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/variant:1954:13
#10 0x558c68a85790 in doris::pipeline::AggLocalState::_get_with_serialized_key_result(doris::RuntimeState*, doris::vectorized::Block*, bool*) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/aggregation_source_operator.cpp:251:5
apache#11 0x558c68b3e3e7 in doris::pipeline::AggLocalState::init(doris::RuntimeState*, doris::pipeline::LocalStateInfo&)::$_2::operator()(doris::RuntimeState*, doris::vectorized::Block*, bool*) const /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/aggregation_source_operator.cpp:75:24
apache#12 0x558c68b3e3e7 in doris::Status std::__invoke_impl(std::__invoke_other, doris::pipeline::AggLocalState::init(doris::RuntimeState*, doris::pipeline::LocalStateInfo&)::$_2&, doris::RuntimeState*&&, doris::vectorized::Block*&&, bool*&&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14
apache#13 0x558c68b3e3e7 in std::enable_if, doris::Status>::type std::__invoke_r(doris::pipeline::AggLocalState::init(doris::RuntimeState*, doris::pipeline::LocalStateInfo&)::$_2&, doris::RuntimeState*&&, doris::vectorized::Block*&&, bool*&&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9
apache#14 0x558c68b3e3e7 in std::_Function_handler::_M_invoke(std::_Any_data const&, doris::RuntimeState*&&, doris::vectorized::Block*&&, bool*&&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9
apache#15 0x558c68a8b819 in std::function::operator()(doris::RuntimeState*, doris::vectorized::Block*, bool*) const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9
apache#16 0x558c68a8b819 in doris::pipeline::AggSourceOperatorX::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/aggregation_source_operator.cpp:448:5
apache#17 0x558c68f652cb in doris::pipeline::PartitionedAggSourceOperatorX::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/partitioned_aggregation_source_operator.cpp:167:36
apache#18 0x558c676b1ddd in doris::pipeline::OperatorXBase::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/operator.cpp:391:18
apache#19 0x558c69b0493d in doris::pipeline::PipelineTask::execute(bool*) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/pipeline_task.cpp:521:13
apache#20 0x558c69b52916 in doris::pipeline::TaskScheduler::_do_work(int) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/task_scheduler.cpp:147:9
apache#21 0x558c54fa66f2 in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:614:24
apache#22 0x558c54f83766 in std::function::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9
apache#23 0x558c54f83766 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:460:5
apache#24 0x558c507cbd26 in asan_thread_start(void*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x1f57fd26)
apache#25 0x7fb97f744ac2 in start_thread nptl/pthread_create.c:442:8
apache#26 0x7fb97f7d684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/zcp/repo_center/doris_master/doris/be/src/util/counts.h:127:85
```
xy720
pushed a commit
that referenced
this pull request
Oct 16, 2025
### What problem does this PR solve? introduced by apache#56405 2 changes here: 1. print local runtime filter mgr info inside each task 2. make a pair for tasks with its corresponding runtime state. ==198869==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7ba3927ac338 at pc 0x559d81e109b9 bp 0x7b7a5b8ad410 sp 0x7b7a5b8ad408 12:38:06 READ of size 8 at 0x7ba3927ac338 thread T1845 (EvHttpServer [w) 12:38:06 #0 0x559d81e109b8 in std::__uniq_ptr_impl<doris::RuntimeState, std::default_delete<doris::RuntimeState> >::_M_ptr() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/unique_ptr.h:193:51 12:38:06 #1 0x559d81e109b8 in std::unique_ptr<doris::RuntimeState, std::default_delete<doris::RuntimeState> >::get() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/unique_ptr.h:473:21 12:38:06 #2 0x559d81e109b8 in std::unique_ptr<doris::RuntimeState, std::default_delete<doris::RuntimeState> >::operator->() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/unique_ptr.h:466:9 12:38:06 #3 0x559d81e109b8 in doris::pipeline::PipelineFragmentContext::debug_string[abi:cxx11]() /root/doris/be/src/pipeline/pipeline_fragment_context.cpp:1941:28 12:38:06 #4 0x559d6d65aa2f in doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0::operator()(phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&) const /root/doris/be/src/runtime/fragment_mgr.cpp:807:36 12:38:06 + echo 'cp -r /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0//Cluster0/fe/bin /home/work/pipline/backup_center/56368_04d746711764283b1cbafe40b1f849e10a5bcb6e_p0/fe/' 12:38:06 #5 0x559d6d65aa2f in doris::Status std::__invoke_impl<doris::Status, doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&>(std::__invoke_other, doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 12:38:06 + cp -r /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0//Cluster0/fe/conf /home/work/pipline/backup_center/56368_04d746711764283b1cbafe40b1f849e10a5bcb6e_p0/fe/ 12:38:06 #6 0x559d6d65aa2f in std::enable_if<is_invocable_r_v<doris::Status, doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&>, doris::Status>::type std::__invoke_r<doris::Status, doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&>(doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 12:38:06 #7 0x559d6d65aa2f in std::_Function_handler<doris::Status (phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&), doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0>::_M_invoke(std::_Any_data const&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 12:38:06 #8 0x559d6d665d37 in std::function<doris::Status (phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&)>::operator()(phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&) const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 12:38:06 #9 0x559d6d665d37 in doris::ConcurrentContextMap<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, doris::pipeline::PipelineFragmentContext>::apply(std::function<doris::Status (phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&)>&&) /root/doris/be/src/runtime/fragment_mgr.h:94:31 12:38:06 #10 0x559d6d647f93 in doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long) /root/doris/be/src/runtime/fragment_mgr.cpp:792:23 12:38:06 apache#11 0x559d7068942b in doris::LongPipelineTaskAction::handle(doris::HttpRequest*) /root/doris/be/src/http/action/pipeline_task_action.cpp:54:69 12:38:06 apache#12 0x559d82c776c6 (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x385416c6) 12:38:06 apache#13 0x559d82c5775f in bufferevent_run_readcb_ /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/bufferevent.c:251:3 12:38:06 apache#14 0x559d82c79802 in bufferevent_trigger_nolock_ /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/bufferevent-internal.h:411:3 12:38:06 apache#15 0x559d82c79802 in bufferevent_readcb /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/bufferevent_sock.c:214:2 12:38:06 apache#16 0x559d82c60724 in event_persist_closure /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/event.c:1623:9 12:38:06 apache#17 0x559d82c60724 in event_process_active_single_queue /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/event.c:1682:4 12:38:06 apache#18 0x559d82c60d76 in event_process_active /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/event.c:1783:9 12:38:06 + echo 'cp -r /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0//Cluster0/fe/log /home/work/pipline/backup_center/56368_04d746711764283b1cbafe40b1f849e10a5bcb6e_p0/fe/' 12:38:06 apache#19 0x559d82c6355f in event_base_loop.constprop.0 /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/event.c:2006:12 12:38:06 apache#20 0x559d7072b08c in doris::EvHttpServer::start()::$_0::operator()() const /root/doris/be/src/http/ev_http_server.cpp:139:13 12:38:06 + cp -r /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0//Cluster0/fe/log /home/work/pipline/backup_center/56368_04d746711764283b1cbafe40b1f849e10a5bcb6e_p0/fe/ 12:38:06 apache#21 0x559d7072b08c in void std::__invoke_impl<void, doris::EvHttpServer::start()::$_0&>(std::__invoke_other, doris::EvHttpServer::start()::$_0&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 12:38:06 apache#22 0x559d7072b08c in std::enable_if<is_invocable_r_v<void, doris::EvHttpServer::start()::$_0&>, void>::type std::__invoke_r<void, doris::EvHttpServer::start()::$_0&>(doris::EvHttpServer::start()::$_0&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:113:2 12:38:06 apache#23 0x559d7072b08c in std::_Function_handler<void (), doris::EvHttpServer::start()::$_0>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 12:38:06 apache#24 0x559d6e121772 in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:614:24 12:38:06 apache#25 0x559d6e0fe7e6 in std::function<void ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 12:38:06 apache#26 0x559d6e0fe7e6 in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:460:5 12:38:06 apache#27 0x559d6984ed26 in asan_thread_start(void*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x1f118d26) 12:38:06 apache#28 0x7f83909ee608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8 12:38:06 apache#29 0x7f8390901132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95 ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
xy720
pushed a commit
that referenced
this pull request
Oct 16, 2025
…sMgr::report_runtime_query_statistics (apache#56744) try to avoid asan check fail: ```cpp OpenJDK 64-Bit Server VM warning: Option CriticalJNINatives was deprecated in version 16.0 and will likely be removed in a future release. Listening for transport dt_socket at address: 8871 WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. start BE in local mode thrift error, reason=write() send(): Broken pipe================================================================= AddressSanitizer: CHECK failed: asan_thread.cpp:369 "((ptr[0] == kCurrentStackFrameMagic)) != (0)" (0x0, 0x0) (tid=13525) #0 0x557a503468e1 in __asan::CheckUnwind() (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2a08e1) #1 0x557a50361182 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2bb182) #2 0x557a5034a97b in __asan::AsanThread::GetStackFrameAccessByAddr(unsigned long, __asan::AsanThread::StackFrameAccess*) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2a497b) #3 0x557a502a4a30 in __asan::AddressDescription::AddressDescription(unsigned long, unsigned long, bool) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f1fea30) #4 0x557a502a67a3 in __asan::ErrorGeneric::ErrorGeneric(unsigned int, unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2007a3) #5 0x557a50343878 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f29d878) #6 0x557a50344f75 in __asan_report_load8 (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f29ef75) #7 0x557a5039791e in std::__cxx11::basic_string, std::allocator>::_M_create(unsigned long&, unsigned long) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.tcc:148:11 #8 0x557a503984db in void std::__cxx11::basic_string, std::allocator>::_M_construct(char const*, unsigned long) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.tcc:291:12 #9 0x557a503984db in std::__cxx11::basic_string, std::allocator>::basic_string(std::__cxx11::basic_string, std::allocator> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.h:617:2 #10 0x557a55b4d77d in std::pair, std::allocator> const, doris::TQueryStatistics>::pair(std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_pair.h:312:17 apache#11 0x557a55b4d77d in _ZSt12construct_atISt4pairIKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN5doris16TQueryStatisticsEEJRKSA_EQaant20is_unbounded_array_vIT_ErqXgsnwcvPvLi0E_SD_pispclsr3stdE7declvalIT0_EEEEEPSD_SG_DpOSF_ /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_construct.h:110:22 apache#12 0x557a55b4d77d in void std::allocator_traits, std::allocator> const, doris::TQueryStatistics>>>>::construct, std::allocator> const, doris::TQueryStatistics>, std::pair, std::allocator> const, doris::TQueryStatistics> const&>(std::allocator, std::allocator> const, doris::TQueryStatistics>>>&, std::pair, std::allocator> const, doris::TQueryStatistics>*, std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/alloc_traits.h:676:4 apache#13 0x557a55b4d77d in void std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_construct_node, std::allocator> const, doris::TQueryStatistics> const&>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:1237:8 apache#14 0x557a55b4cfe3 in std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node::operator(), std::allocator> const, doris::TQueryStatistics> const&>(std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h apache#15 0x557a55b4cbe6 in std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_clone_node, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:1287:8 apache#16 0x557a55b4c6f6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2546:20 apache#17 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3 apache#18 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3 apache#19 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3 apache#20 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3 apache#21 0x557a55b4c27a in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:1523:6 apache#22 0x557a55740d4f in std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::operator=(std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2453:18 apache#23 0x557a55740d4f in std::map, std::allocator>, doris::TQueryStatistics, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::operator=(std::map, std::allocator>, doris::TQueryStatistics, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_map.h:354:37 apache#24 0x557a55740d4f in doris::TReportWorkloadRuntimeStatusParams::__set_query_statistics_map(std::map, std::allocator>, doris::TQueryStatistics, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&) /home/zcp/repo_center/doris_branch-4.0/doris/gensrc/build/gen_cpp/FrontendService_types.cpp:4391:30 apache#25 0x557a5447e13a in doris::RuntimeQueryStatisticsMgr::report_runtime_query_statistics() /home/zcp/repo_center/doris_branch-4.0/doris/be/src/runtime/runtime_query_statistics_mgr.cpp:403:31 apache#26 0x557a504f1192 in doris::Daemon::report_runtime_query_statistics_thread() /home/zcp/repo_center/doris_branch-4.0/doris/be/src/common/daemon.cpp:457:65 apache#27 0x557a54bf2eb6 in std::function::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 apache#28 0x557a54bf2eb6 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-4.0/doris/be/src/util/thread.cpp:460:5 apache#29 0x557a50336d26 in asan_thread_start(void*) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f290d26) apache#30 0x7f12a399eac2 in start_thread nptl/pthread_create.c:442:8 apache#31 0x7f12a3a3084f in __closefrom_fallback misc/../sysdeps/unix/sysv/linux/closefrom_fallback.c:45:19 172.20.59.34 last coredump sql: last SQL query not found ```
xy720
pushed a commit
that referenced
this pull request
Oct 16, 2025
…e#56935) ``` (gdb) bt #0 doris::cloud::FdbTxnKv::get_partition_boundaries (this=0x7f11ac6e3950, boundaries=0x7f1130017f20) at /home/zcp/repo_center/doris_branch-4.0/doris/cloud/src/meta-store/txn_kv.cpp:212 #1 0x000055e494de9276 in doris::cloud::export_fdb_kv_ranges_details (kv=0x7f11ac6e3950) at /home/zcp/repo_center/doris_branch-4.0/doris/cloud/src/common/metric.cpp:358 #2 doris::cloud::FdbMetricExporter::export_fdb_metrics (txn_kv=0x7f11ac6e3950) at /home/zcp/repo_center/doris_branch-4.0/doris/cloud/src/common/metric.cpp:393 #3 0x000055e494dec77d in doris::cloud::FdbMetricExporter::start()::$_0::operator()() const (this=<optimized out>) at /home/zcp/repo_center/doris_branch-4.0/doris/cloud/src/common/metric.cpp:415 #4 std::__invoke_impl<void, doris::cloud::FdbMetricExporter::start()::$_0>(std::__invoke_other, doris::cloud::FdbMetricExporter::start()::$_0&&) (__f=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63 #5 std::__invoke<doris::cloud::FdbMetricExporter::start()::$_0>(doris::cloud::FdbMetricExporter::start()::$_0&&) (__fn=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:98 #6 std::thread::_Invoker<std::tuple<doris::cloud::FdbMetricExporter::start()::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_thread.h:303 #7 std::thread::_Invoker<std::tuple<doris::cloud::FdbMetricExporter::start()::$_0> >::operator()() (this=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_thread.h:310 #8 std::thread::_State_impl<std::thread::_Invoker<std::tuple<doris::cloud::FdbMetricExporter::start()::$_0> > >::_M_run() (this=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_thread.h:255 #9 0x000055e495e547b0 in execute_native_thread_routine () #10 0x00007f11b1dfeac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 apache#11 0x00007f11b1e90850 in __closefrom_fallback (from=-1266337808, dirfd_fallback=<optimized out>) at ../sysdeps/unix/sysv/linux/closefrom_fallback.c:45 apache#12 0x0000000000000000 in ?? () ```
xy720
pushed a commit
that referenced
this pull request
Oct 16, 2025
…56755) ### What problem does this PR solve? ```text #0 __GI___pthread_sigmask (how=2, newmask=<optimized out>, oldmask=0x0) at ./nptl/pthread_sigmask.c:43 #1 0x00007f91fd6c471e in PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #2 0x00007f91fd6c5206 in JVM_handle_linux_signal () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #3 <signal handler called> #4 __gnu_cxx::__exchange_and_add (__mem=0xe, __val=-1) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/ext/atomicity.h:68 #5 __gnu_cxx::__exchange_and_add_dispatch (__mem=0xe, __val=-1) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/ext/atomicity.h:103 #6 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_weak_release (this=0x2) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:211 #7 std::__weak_count<(__gnu_cxx::_Lock_policy)2>::~__weak_count (this=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1168 #8 std::__weak_ptr<doris::QueryContext, (__gnu_cxx::_Lock_policy)2>::~__weak_ptr (this=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:2003 #9 doris::QueryTaskController::revoke_memory()::$_1::~$_1() (this=<optimized out>) at /root/doris/be/src/runtime/workload_management/query_task_controller.cpp:168 #10 std::_Function_base::_Base_manager<doris::QueryTaskController::revoke_memory()::$_1>::_M_destroy(std::_Any_data&, std::integral_constant<bool, false>) (__victim=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:177 apache#11 std::_Function_base::_Base_manager<doris::QueryTaskController::revoke_memory()::$_1>::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (__dest=..., __op=std::__destroy_functor, __source=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:205 apache#12 std::_Function_handler<void (doris::pipeline::SpillContext*), doris::QueryTaskController::revoke_memory()::$_1>::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (__dest=..., __source=..., __op=std::__destroy_functor) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:284 apache#13 0x000055f37fba60c5 in std::_Function_base::~_Function_base (this=0x7f8dbb0c9fe0) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:246 apache#14 doris::pipeline::SpillContext::~SpillContext (this=0x7f91536f0150) at /root/doris/be/src/pipeline/exec/spill_utils.h:57 apache#15 0x000055f384259e4b in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x7f91536f0140) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:345 apache#16 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::operator= (this=0x7f8d61302518, __r=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1088 apache#17 std::__shared_ptr<doris::pipeline::SpillContext, (__gnu_cxx::_Lock_policy)2>::operator= (this=0x7f8d61302510) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1530 apache#18 std::shared_ptr<doris::pipeline::SpillContext>::operator= (this=0x7f8d61302510) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr.h:413 apache#19 doris::pipeline::PipelineTask::revoke_memory (this=0x7f8d61302490, spill_context=...) at /root/doris/be/src/pipeline/pipeline_task.cpp:812 apache#20 0x000055f37fba3c2d in doris::QueryTaskController::revoke_memory (this=<optimized out>) at /root/doris/be/src/runtime/workload_management/query_task_controller.cpp:185 apache#21 0x000055f37fb99d08 in doris::WorkloadGroupMgr::handle_single_query_ (this=<optimized out>, requestor=..., size_to_reserve=size_to_reserve@entry=1024000, time_in_queue=time_in_queue@entry=27, paused_reason=...) at /root/doris/be/src/runtime/workload_group/workload_group_manager.cpp:820 apache#22 0x000055f37fb981d0 in doris::WorkloadGroupMgr::handle_paused_queries (this=0x7f9141d8a800) at /root/doris/be/src/runtime/workload_group/workload_group_manager.cpp:381 apache#23 0x000055f37eb97297 in doris::Daemon::memory_maintenance_thread (this=0x7ffe63cad730) at /root/doris/be/src/common/daemon.cpp:354 apache#24 0x000055f37fd812fc in std::function<void ()>::operator()() const (this=0x7f8dbb0c9fe0) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#25 doris::Thread::supervise_thread (arg=0x7f914ba25e10) at /root/doris/be/src/util/thread.cpp:460 apache#26 0x00007f91fc75fac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 apache#27 0x00007f91fc7f1850 in __closefrom_fallback (from=1674236160, dirfd_fallback=<optimized out>) at ../sysdeps/unix/sysv/linux/closefrom_fallback.c:45 apache#28 0x0000000000000000 in ?? () ``` Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
xy720
pushed a commit
that referenced
this pull request
Oct 23, 2025
…icate (apache#57100) ### What problem does this PR solve? ```text #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140649357411904) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=140649357411904) at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=140649357411904, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 #3 0x00007feca2a42476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #4 0x00007feca2a287f3 in __GI_abort () at ./stdlib/abort.c:79 #5 0x000055cd63341b52 in __gnu_cxx::__verbose_terminate_handler() () #6 0x000055cd6333fe26 in __cxxabiv1::__terminate(void (*)()) () #7 0x000055cd6333fe79 in std::terminate() () #8 0x000055cd6333ffc3 in __cxa_throw () #9 0x000055cd6339556a in std::__throw_out_of_range(char const*) () #10 0x000055cd5a9a7a68 in __gnu_cxx::__stoa<double, double, char> (__str=0x7fe842726fc0 "9.", '9' <repeats 15 times>, "1e-309", __idx=0x0, __convf=<optimized out>, __name=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/ext/string_conversions.h:90 apache#11 std::__cxx11::stod (__str=..., __idx=0x0) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.h:4490 apache#12 doris::IntegerPredicateCreator<(doris::PrimitiveType)9, (doris::PredicateType)1, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::convert (condition=...) at /root/doris_branch-4.0/doris/be/src/olap/predicate_creator.h:68 apache#13 doris::IntegerPredicateCreator<(doris::PrimitiveType)9, (doris::PredicateType)1, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::create (this=<optimized out>, column=..., index=1, conditions=..., opposite=false, arena=...) at /root/doris_branch-4.0/doris/be/src/olap/predicate_creator.h:59 apache#14 0x000055cd5a3dd63b in doris::create_predicate<(doris::PredicateType)1, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (column=..., index=1, conditions=..., opposite=false, arena=...) at /root/doris_branch-4.0/doris/be/src/olap/predicate_creator.h:270 apache#15 doris::create_comparison_predicate<(doris::PredicateType)1> (column=..., index=1, condition=..., opposite=false, arena=...) at /root/doris_branch-4.0/doris/be/src/olap/predicate_creator.h:278 apache#16 0x000055cd5a3dd435 in doris::parse_to_predicate (column=..., index=1, condition=..., arena=..., opposite=false) at /root/doris_branch-4.0/doris/be/src/olap/predicate_creator.h:326 apache#17 0x000055cd5afc2759 in doris::TabletReader::_init_conditions_param (this=this@entry=0x7fe842e1a800, read_params=...) at /root/doris_branch-4.0/doris/be/src/olap/tablet_reader.cpp:549 apache#18 0x000055cd5afc1022 in doris::TabletReader::_init_params (this=this@entry=0x7fe842e1a800, read_params=...) at /root/doris_branch-4.0/doris/be/src/olap/tablet_reader.cpp:304 apache#19 0x000055cd5afbfe00 in doris::TabletReader::init (this=0x7fe842e1a800, read_params=...) at /root/doris_branch-4.0/doris/be/src/olap/tablet_reader.cpp:127 apache#20 0x000055cd5f019d0f in doris::vectorized::BlockReader::init (this=0x4c1b, read_params=...) at /root/doris_branch-4.0/doris/be/src/vec/olap/block_reader.cpp:206 apache#21 0x000055cd5f79a3f4 in doris::vectorized::OlapScanner::open (this=0x7febe2c16010, state=<optimized out>) at /root/doris_branch-4.0/doris/be/src/vec/exec/scan/olap_scanner.cpp:278 apache#22 0x000055cd5e05303a in doris::vectorized::ScannerScheduler::_scanner_scan (ctx=..., scan_task=...) at /root/doris_branch-4.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:182 apache#23 0x000055cd5e0549e5 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const (this=<optimized out>) at /root/doris_branch-4.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:96 apache#24 doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}::operator()() const (this=0x7febe321dac0) at /root/doris_branch-4.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:95 apache#25 std::__invoke_impl<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&) (__f=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63 apache#26 std::__invoke_r<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&) (__fn=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116 apache#27 std::_Function_handler<bool (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) ( __functor=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292 apache#28 0x000055cd5b4ab5d5 in doris::ThreadPool::dispatch_thread (this=0x7fec35e01800) at /root/doris_branch-4.0/doris/be/src/util/threadpool.cpp:614 apache#29 0x000055cd5b4a06fc in std::function<void ()>::operator()() const (this=0x4a17) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#30 doris::Thread::supervise_thread (arg=0x7fec35872110) at /root/doris_branch-4.0/doris/be/src/util/thread.cpp:460 apache#31 0x00007feca2a94ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 apache#32 0x00007feca2b26850 in __closefrom_fallback (from=-823527648, dirfd_fallback=<optimized out>) at ../sysdeps/unix/sysv/linux/closefrom_fallback.c:45 apache#33 0x0000000000000000 in ?? () ``` Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.