Slow shutdown: mutation cannot be cancelled in buildOrderedSetInplace #51586
Description
Postprocessed stacktraces: https://pastila.nl/?0006a09d/09e0f587c41a2895002171df28213f5c
A mutation could not be canceled on server shutdown for 2 minutes:
248080:2023.06.29 06:37:22.282449 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
248103:2023.06.29 06:37:22.288072 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
248119:2023.06.29 06:37:22.293608 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
248155:2023.06.29 06:37:22.298986 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
248179:2023.06.29 06:37:22.304319 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
248207:2023.06.29 06:37:22.309078 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
248445:2023.06.29 06:37:22.333395 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Trace> CreatingSetsTransform: Building set, key: __set_13174138895228284904_638944423351839049
248448:2023.06.29 06:37:22.333471 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Trace> CreatingSetsTransform: Creating set, key: __set_13174138895228284904_638944423351839049
272450:2023.06.29 06:37:46.202029 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Debug> MemoryTracker: Current memory usage background process to apply mutate/merge in table: 1.00 GiB.
291259:2023.06.29 06:38:12.070370 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Debug> MemoryTracker: Current memory usage background process to apply mutate/merge in table: 2.00 GiB.
304148:2023.06.29 06:38:41.152316 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Debug> CreatingSetsTransform: Created Set with 200000000 entries from 200000000 rows in 78.819460723 sec.
304149:2023.06.29 06:38:41.152484 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Test> PipelineExecutor: Thread finished. Total time: 78.832093113 sec. Execution time: 78.4752538 sec. Processing time: 0.334895134 sec. Wait time: 0.021944179 sec.
304150:2023.06.29 06:38:41.156670 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Trace> CreatingSetsTransform: Building set, key: __set_4281589654969730874_11673350554031416278
304151:2023.06.29 06:38:41.156749 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Trace> CreatingSetsTransform: Creating set, key: __set_4281589654969730874_11673350554031416278
304240:2023.06.29 06:39:01.111334 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Debug> MemoryTracker: Current memory usage background process to apply mutate/merge in table: 3.00 GiB.
304249:2023.06.29 06:39:24.726556 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Debug> MemoryTracker: Current memory usage background process to apply mutate/merge in table: 4.00 GiB.
304250:2023.06.29 06:39:24.726747 [ 291103 ] {9b18327a-c6e8-4c57-95f6-266c685faa51::all_1_4_1_9} <Debug> MemoryTracker: Current memory usage (total): 5.54 GiB.
Stacktrace:
2023-06-29 06:39:50 Thread 248 (Thread 0x7f891cdf6700 (LWP 291103)):
2023-06-29 06:39:50 #0 HashTable<unsigned int, HashTableCell<unsigned int, HashCRC32<unsigned int>, HashTableNoState>, HashCRC32<unsigned int>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true> >::resize (this=0x7f885058e380, for_num_elems=0, for_buf_size=0) at /build/src/Common/HashTable/HashTable.h:591
2023-06-29 06:39:50 #1 0x000000002d60a740 in HashTable<unsigned int, HashTableCell<unsigned int, HashCRC32<unsigned int>, HashTableNoState>, HashCRC32<unsigned int>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true> >::emplaceNonZeroImpl<unsigned int&> (this=0x7f885058e380, place_value=146882091, key_holder=@0x7f891cde1b74: 1342177286, it=@0x7f891cde1bb0: 0x40f2f01698ac, inserted=@0x7f891cde1baf: true, hash_value=2025930283) at /build/src/Common/HashTable/HashTable.h:989
2023-06-29 06:39:50 #2 HashTable<unsigned int, HashTableCell<unsigned int, HashCRC32<unsigned int>, HashTableNoState>, HashCRC32<unsigned int>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true> >::emplaceNonZero<unsigned int&> (this=0x7f885058e380, key_holder=@0x7f891cde1b74: 1342177286, it=@0x7f891cde1bb0: 0x40f2f01698ac, inserted=@0x7f891cde1baf: true, hash_value=2025930283) at /build/src/Common/HashTable/HashTable.h:1017
2023-06-29 06:39:50 #3 HashTable<unsigned int, HashTableCell<unsigned int, HashCRC32<unsigned int>, HashTableNoState>, HashCRC32<unsigned int>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true> >::emplace<unsigned int&> (this=0x7f885058e380, key_holder=@0x7f891cde1b74: 1342177286, it=@0x7f891cde1bb0: 0x40f2f01698ac, inserted=@0x7f891cde1baf: true, hash_value=2025930283) at /build/src/Common/HashTable/HashTable.h:1096
2023-06-29 06:39:50 #4 HashTable<unsigned int, HashTableCell<unsigned int, HashCRC32<unsigned int>, HashTableNoState>, HashCRC32<unsigned int>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true> >::emplace<unsigned int&> (this=0x7f885058e380, key_holder=@0x7f891cde1b74: 1342177286, it=@0x7f891cde1bb0: 0x40f2f01698ac, inserted=@0x7f891cde1baf: true) at /build/src/Common/HashTable/HashTable.h:1087
2023-06-29 06:39:50 #5 DB::ColumnsHashing::columns_hashing_impl::HashMethodBase<DB::ColumnsHashing::HashMethodOneNumber<unsigned int, void, unsigned int, true, false, false>, unsigned int, void, true, false, false>::emplaceImpl<HashSetTable<unsigned int, HashTableCell<unsigned int, HashCRC32<unsigned int>, HashTableNoState>, HashCRC32<unsigned int>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true> >, unsigned int> (this=0x7f891cde1b20, key_holder=@0x7f891cde1b74: 1342177286, data=...) at /build/src/Common/ColumnsHashingImpl.h:251
2023-06-29 06:39:50 #6 DB::ColumnsHashing::columns_hashing_impl::HashMethodBase<DB::ColumnsHashing::HashMethodOneNumber<unsigned int, void, unsigned int, true, false, false>, unsigned int, void, true, false, false>::emplaceKey<HashSetTable<unsigned int, HashTableCell<unsigned int, HashCRC32<unsigned int>, HashTableNoState>, HashCRC32<unsigned int>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true> > > (this=0x7f891cde1b20, data=..., row=63869, pool=...) at /build/src/Common/ColumnsHashingImpl.h:170
2023-06-29 06:39:50 #7 DB::Set::insertFromBlockImplCase<DB::SetMethodOneNumber<unsigned int, HashSetTable<unsigned int, HashTableCell<unsigned int, HashCRC32<unsigned int>, HashTableNoState>, HashCRC32<unsigned int>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true> >, true>, false, false> (this=0x7f88ba09cc00, method=..., key_columns=..., rows=65409, variants=..., null_map=0x0, out_filter=0x0) at /build/src/Interpreters/Set.cpp:98
2023-06-29 06:39:50 #8 0x000000002d600eb5 in DB::Set::insertFromBlockImpl<DB::SetMethodOneNumber<unsigned int, HashSetTable<unsigned int, HashTableCell<unsigned int, HashCRC32<unsigned int>, HashTableNoState>, HashCRC32<unsigned int>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true> >, true> > (this=0x7f88ba09cc00, method=..., key_columns=..., rows=65409, variants=..., null_map=0x0, out_filter=0x0) at /build/src/Interpreters/Set.cpp:67
2023-06-29 06:39:50 #9 0x000000002d5f8e20 in DB::Set::insertFromColumns (this=0x7f88ba09cc00, columns=..., holder=...) at /build/src/Interpreters/Set.cpp:249
2023-06-29 06:39:50 #10 0x000000002d5f87b6 in DB::Set::insertFromColumns (this=0x7f88ba09cc00, columns=...) at /build/src/Interpreters/Set.cpp:200
2023-06-29 06:39:50 #11 0x000000002d5f86d4 in DB::Set::insertFromBlock (this=0x7f88ba09cc00, columns=...) at /build/src/Interpreters/Set.cpp:188
2023-06-29 06:39:50 #12 0x000000002f24fc24 in DB::CreatingSetsTransform::consume (this=0x7f88d8998498, chunk=...) at /build/src/Processors/Transforms/CreatingSetsTransform.cpp:146
2023-06-29 06:39:50 #13 0x000000002edab695 in DB::IAccumulatingTransform::work (this=0x7f88d8998498) at /build/src/Processors/IAccumulatingTransform.cpp:97
2023-06-29 06:39:50 #14 0x000000002f24c5ba in DB::CreatingSetsTransform::work (this=0x7f88d8998498) at /build/src/Processors/Transforms/CreatingSetsTransform.cpp:51
2023-06-29 06:39:50 #15 0x000000002ede9f43 in DB::executeJob (node=0x7f885d483000, read_progress_callback=0x7f88308f9080) at /build/src/Processors/Executors/ExecutionThreadContext.cpp:47
2023-06-29 06:39:50 #16 0x000000002ede9c80 in DB::ExecutionThreadContext::executeTask (this=0x7f88f0551660) at /build/src/Processors/Executors/ExecutionThreadContext.cpp:92
2023-06-29 06:39:50 #17 0x000000002edce8a1 in DB::PipelineExecutor::executeStepImpl (this=0x7f891cde3018, thread_num=0, yield_flag=0x0) at /build/src/Processors/Executors/PipelineExecutor.cpp:255
2023-06-29 06:39:50 #18 0x000000002edcebb7 in DB::PipelineExecutor::executeSingleThread (this=0x7f891cde3018, thread_num=0) at /build/src/Processors/Executors/PipelineExecutor.cpp:221
2023-06-29 06:39:50 #19 0x000000002edcdbc7 in DB::PipelineExecutor::executeImpl (this=0x7f891cde3018, num_threads=1) at /build/src/Processors/Executors/PipelineExecutor.cpp:379
2023-06-29 06:39:50 #20 0x000000002edcd1a0 in DB::PipelineExecutor::execute (this=0x7f891cde3018, num_threads=1) at /build/src/Processors/Executors/PipelineExecutor.cpp:113
2023-06-29 06:39:50 #21 0x000000002edcb262 in DB::CompletedPipelineExecutor::execute (this=0x7f891cde3278) at /build/src/Processors/Executors/CompletedPipelineExecutor.cpp:105
2023-06-29 06:39:50 #22 0x000000002d47bcb9 in DB::FutureSetFromSubquery::buildOrderedSetInplace (this=0x7f88bb233208, context=...) at /build/src/Interpreters/PreparedSets.cpp:199
2023-06-29 06:39:50 #23 0x000000002e5a09d6 in DB::KeyCondition::tryPrepareSetIndex (this=0x7f891cde5318, func=..., out=..., out_key_column_num=@0x7f891cde3e50: 0) at /build/src/Storages/MergeTree/KeyCondition.cpp:1248
2023-06-29 06:39:50 #24 0x000000002e5a20bc in DB::KeyCondition::extractAtomFromTree (this=0x7f891cde5318, node=..., out=...) at /build/src/Storages/MergeTree/KeyCondition.cpp:1529
2023-06-29 06:39:50 #25 0x000000002e5b0488 in DB::KeyCondition::KeyCondition(std::__1::shared_ptr<DB::ActionsDAG>, std::__1::shared_ptr<DB::Context const>, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::ExpressionActions> const&, std::__1::unordered_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >, bool, bool)::$_0::operator()(DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&) const (this=0x7f891cde44b8, node=..., out=...) at /build/src/Storages/MergeTree/KeyCondition.cpp:842
2023-06-29 06:39:50 #26 0x000000002e5b0445 in std::__1::__invoke[abi:v15000]<DB::KeyCondition::KeyCondition(std::__1::shared_ptr<DB::ActionsDAG>, std::__1::shared_ptr<DB::Context const>, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::ExpressionActions> const&, std::__1::unordered_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >, bool, bool)::$_0&, DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&> (__f=..., __args=..., __args=...) at /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394
2023-06-29 06:39:50 #27 0x000000002e5b0405 in std::__1::__invoke_void_return_wrapper<bool, false>::__call<DB::KeyCondition::KeyCondition(std::__1::shared_ptr<DB::ActionsDAG>, std::__1::shared_ptr<DB::Context const>, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::ExpressionActions> const&, std::__1::unordered_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >, bool, bool)::$_0&, DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&>(DB::KeyCondition::KeyCondition(std::__1::shared_ptr<DB::ActionsDAG>, std::__1::shared_ptr<DB::Context const>, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::ExpressionActions> const&, std::__1::unordered_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >, bool, bool)::$_0&, DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&) (__args=..., __args=..., __args=...) at /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:470
2023-06-29 06:39:50 #28 0x000000002e5b03c5 in std::__1::__function::__default_alloc_func<DB::KeyCondition::KeyCondition(std::__1::shared_ptr<DB::ActionsDAG>, std::__1::shared_ptr<DB::Context const>, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::ExpressionActions> const&, std::__1::unordered_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >, bool, bool)::$_0, bool (DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&)>::operator()[abi:v15000](DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&) (this=0x7f891cde44b8, __arg=..., __arg=...) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:235
2023-06-29 06:39:50 #29 0x000000002e5b038d in std::__1::__function::__policy_invoker<bool (DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&)>::__call_impl<std::__1::__function::__default_alloc_func<DB::KeyCondition::__default_alloc_func(std::__1::shared_ptr<DB::ActionsDAG>, std::__1::shared_ptr<DB::Context const>, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::ExpressionActions> const&, std::__1::unordered_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >, bool, bool)::$_0, bool (DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&)> >(std::__1::__function::__policy_storage const*, DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&) (__buf=0x7f891cde44b8, __args=..., __args=...) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:716
2023-06-29 06:39:50 #30 0x000000002e5d6a26 in std::__1::__function::__policy_func<bool (DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&)>::operator()[abi:v15000](DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&) const (this=0x7f891cde44b8, __args=..., __args=...) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:848
2023-06-29 06:39:50 #31 0x000000002e5d6865 in std::__1::function<bool (DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&)>::operator()(DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&) const (this=0x7f891cde44b8, __arg=..., __arg=...) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187
2023-06-29 06:39:50 #32 0x000000002e5d6578 in DB::RPNBuilder<DB::KeyCondition::RPNElement>::traverseTree (this=0x7f891cde44e8, node=...) at /build/src/Storages/MergeTree/RPNBuilder.h:250
2023-06-29 06:39:50 #33 0x000000002e5d64bf in DB::RPNBuilder<DB::KeyCondition::RPNElement>::traverseTree (this=0x7f891cde44e8, node=...) at /build/src/Storages/MergeTree/RPNBuilder.h:237
2023-06-29 06:39:50 #34 0x000000002e5c83ea in DB::RPNBuilder<DB::KeyCondition::RPNElement>::RPNBuilder(DB::ActionsDAG::Node const*, std::__1::shared_ptr<DB::Context const>, std::__1::function<bool (DB::RPNBuilderTreeNode const&, DB::KeyCondition::RPNElement&)> const&) (this=0x7f891cde44e8, filter_actions_dag_node=0x7f88308f8f50, query_context_=..., extract_atom_from_tree_function_=...) at /build/src/Storages/MergeTree/RPNBuilder.h:205
2023-06-29 06:39:50 #35 0x000000002e59da59 in DB::KeyCondition::KeyCondition (this=0x7f891cde5318, filter_dag=..., context=..., key_column_names=..., key_expr_=..., array_joined_column_names_=..., single_point_=false, strict_=false) at /build/src/Storages/MergeTree/KeyCondition.cpp:840
2023-06-29 06:39:50 #36 0x000000002f53b973 in DB::buildIndexes (indexes=..., filter_actions_dag=..., data=..., context=..., query_info=..., metadata_snapshot=...) at /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1202
2023-06-29 06:39:50 #37 0x000000002f53b235 in DB::ReadFromMergeTree::onAddFilterFinish (this=0x7f8830861c00) at /build/src/Processors/QueryPlan/ReadFromMergeTree.cpp:1307
2023-06-29 06:39:50 #38 0x000000002f5fb477 in DB::QueryPlanOptimizations::optimizePrimaryKeyCondition (stack=...) at /build/src/Processors/QueryPlan/Optimizations/optimizePrimaryKeyCondition.cpp:32
2023-06-29 06:39:50 #39 0x000000002f5cc320 in DB::QueryPlanOptimizations::optimizeTreeSecondPass (optimization_settings=..., root=..., nodes=...) at /build/src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:168
2023-06-29 06:39:50 #40 0x000000002f51489d in DB::QueryPlan::optimize (this=0x7f891cde5e40, optimization_settings=...) at /build/src/Processors/QueryPlan/QueryPlan.cpp:484
2023-06-29 06:39:50 #41 0x000000002f5143b7 in DB::QueryPlan::buildQueryPipeline (this=0x7f891cde5e40, optimization_settings=..., build_pipeline_settings=...) at /build/src/Processors/QueryPlan/QueryPlan.cpp:161
2023-06-29 06:39:50 #42 0x000000002d2cd44b in DB::InterpreterSelectQuery::execute (this=0x7f891cde6730) at /build/src/Interpreters/InterpreterSelectQuery.cpp:932
2023-06-29 06:39:50 #43 0x000000002d45231b in DB::isStorageTouchedByMutations (storage=..., source_part=..., metadata_snapshot=..., commands=..., context=...) at /build/src/Interpreters/MutationsInterpreter.cpp:190
2023-06-29 06:39:50 #44 0x000000002e9b498b in DB::MutateTask::prepare (this=0x7f8838246228) at /build/src/Storages/MergeTree/MutateTask.cpp:1780
2023-06-29 06:39:50 #45 0x000000002e9b4187 in DB::MutateTask::execute (this=0x7f8838246228) at /build/src/Storages/MergeTree/MutateTask.cpp:1688
2023-06-29 06:39:50 #46 0x000000002eb51611 in DB::MutatePlainMergeTreeTask::executeStep (this=0x7f88c796c7d8) at /build/src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:90
2023-06-29 06:39:50 #47 0x000000002e62988b in DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine (this=0x7f892701e518, item=...) at /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:271
2023-06-29 06:39:50 #48 0x000000002e62a2ce in DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction (this=0x7f892701e518) at /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:336
2023-06-29 06:39:50 #49 0x000000002e632d98 in DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::__1::basic_string_view<char, std::__1::char_traits<char> >)::{lambda()#1}::operator()() const (this=0x7f891cde8d10) at /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56
2023-06-29 06:39:50 #50 0x000000002e632d75 in std::__1::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::__1::basic_string_view<char, std::__1::char_traits<char> >)::{lambda()#1}&> (__f=...) at /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394
2023-06-29 06:39:50 #51 0x000000002e632d55 in std::__1::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::__1::basic_string_view<char, std::__1::char_traits<char> >)::{lambda()#1}&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::__1::basic_string_view<char, std::__1::char_traits<char> >)::{lambda()#1}&) (__args=...) at /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:479
2023-06-29 06:39:50 #52 0x000000002e632d35 in std::__1::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::__1::basic_string_view<char, std::__1::char_traits<char> >)::{lambda()#1}, void ()>::operator()[abi:v15000]() (this=0x7f891cde8d10) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:235
2023-06-29 06:39:50 #53 0x000000002e632cfd in std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::__1::basic_string_view<char, std::__1::char_traits<char> >)::{lambda()#1}, void ()> >(std::__1::__function::__policy_storage const*) (__buf=0x7f891cde8d10) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:716
2023-06-29 06:39:50 #54 0x000000002449d936 in std::__1::__function::__policy_func<void ()>::operator()[abi:v15000]() const (this=0x7f891cde8d10) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:848
2023-06-29 06:39:50 #55 0x000000002449ce35 in std::__1::function<void ()>::operator()() const (this=0x7f891cde8d10) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187
2023-06-29 06:39:50 #56 0x00000000245a7a03 in ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::worker (this=0x7f892701e640, thread_it=...) at /build/src/Common/ThreadPool.cpp:416
2023-06-29 06:39:50 #57 0x00000000245b12e4 in ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}::operator()() const (this=0x7f891cde9120) at /build/src/Common/ThreadPool.cpp:180
2023-06-29 06:39:50 #58 0x00000000245b12b5 in std::__1::__invoke[abi:v15000]<ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}&> (__f=...) at /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394
2023-06-29 06:39:50 #59 0x00000000245b1299 in std::__1::__apply_tuple_impl[abi:v15000]<ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}&, std::__1::tuple<>&>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}&, std::__1::tuple<>&, std::__1::__tuple_indices<>) (__f=..., __t=...) at /build/contrib/llvm-project/libcxx/include/tuple:1789
2023-06-29 06:39:50 #60 0x00000000245b11fd in std::__1::apply[abi:v15000]<ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}&, std::__1::tuple<>&>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}&, std::__1::tuple<>&) (__f=..., __t=...) at /build/contrib/llvm-project/libcxx/include/tuple:1798
2023-06-29 06:39:50 #61 0x00000000245b1100 in ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}&&)::{lambda()#1}::operator()() (this=0x7f8927065ff0) at /build/src/Common/ThreadPool.h:229
2023-06-29 06:39:50 #62 0x00000000245b1075 in std::__1::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}&&)::{lambda()#1}&> (__f=...) at /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394
2023-06-29 06:39:50 #63 0x00000000245b1055 in std::__1::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}&&)::{lambda()#1}&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}&&)::{lambda()#1}&) (__args=...) at /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:479
2023-06-29 06:39:50 #64 0x00000000245b1035 in std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}&&)::{lambda()#1}, void ()>::operator()[abi:v15000]() (this=0x7f8927065ff0) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:235
2023-06-29 06:39:50 #65 0x00000000245b1000 in std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}&&)::{lambda()#1}, void ()> >(std::__1::__function::__policy_storage const*) (__buf=0x7f891cde93d0) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:716
2023-06-29 06:39:50 #66 0x000000002449d936 in std::__1::__function::__policy_func<void ()>::operator()[abi:v15000]() const (this=0x7f891cde93d0) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:848
2023-06-29 06:39:50 #67 0x000000002449ce35 in std::__1::function<void ()>::operator()() const (this=0x7f891cde93d0) at /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187
2023-06-29 06:39:50 #68 0x00000000245a44c3 in ThreadPoolImpl<std::__1::thread>::worker (this=0x7f8ad2e22380, thread_it=...) at /build/src/Common/ThreadPool.cpp:416
2023-06-29 06:39:50 #69 0x00000000245ab6e4 in ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}::operator()() const (this=0x7f8927054288) at /build/src/Common/ThreadPool.cpp:180
2023-06-29 06:39:50 #70 0x00000000245ab695 in std::__1::__invoke[abi:v15000]<ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}> (__f=...) at /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394
2023-06-29 06:39:50 #71 0x00000000245ab5bd in std::__1::__thread_execute[abi:v15000]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}>&, std::__1::__tuple_indices<>) (__t=...) at /build/contrib/llvm-project/libcxx/include/thread:284
2023-06-29 06:39:50 #72 0x00000000245ab0a2 in std::__1::__thread_proxy[abi:v15000]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}> >(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::{lambda()#2}>) (__vp=0x7f8927054280) at /build/contrib/llvm-project/libcxx/include/thread:295
2023-06-29 06:39:50 #73 0x00007f8ad3dcd609 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
2023-06-29 06:39:50 #74 0x00007f8ad3cf2133 in clone () from /lib/x86_64-linux-gnu/libc.so.6