Added a test for race conditions. by alexey-milovidov · Pull Request #6514 · ClickHouse/ClickHouse

alexey-milovidov · 2019-08-16T01:29:23Z

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

For changelog. Remove if this is non-significant change.

Category (leave one):

Bugfix

Short description (up to few sentences):
Fixed wrong code in mutations that may lead to memory corruption. Fixed segfault with read of address 0x14c0 that may happed due to concurrent DROP TABLE and SELECT from system.parts or system.parts_columns. Fixed race condition in preparation of mutation queries. Fixed deadlock caused by OPTIMIZE of Replicated tables and concurrent modification operations like ALTERs.

alexey-milovidov · 2019-08-19T20:31:14Z

Fixed

==================
WARNING: ThreadSanitizer: data race (pid=257651)
  Read of size 8 at 0x7b2c00130d10 by thread T76:
    #0 boost::multi_index::detail::sequenced_index<boost::multi_index::detail::nth_layer<1, DB::ColumnDescription, boost::multi_index::indexed_by<boost::multi_index::sequenced<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_unique<boost::multi_index::member<DB::ColumnDescription, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, &(DB::ColumnDescription::name)>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::__1::allocator<DB::ColumnDescription> >, boost::mpl::vector0<mpl_::na> >::begin() const /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/boost/boost/multi_index/sequenced_index.hpp:223:58 (clickhouse+0xc61cb6a)
    #1 DB::ColumnsDescription::getAllPhysical() const /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Storages/ColumnsDescription.cpp:301 (clickhouse+0xc61cb6a)
    #2 DB::MutationsInterpreter::prepare(bool) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/MutationsInterpreter.cpp:170:50 (clickhouse+0xc410d64)
    #3 DB::MutationsInterpreter::validate() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/MutationsInterpreter.cpp:463:5 (clickhouse+0xc41924a)
    #4 DB::InterpreterAlterQuery::execute() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/InterpreterAlterQuery.cpp:65:65 (clickhouse+0xc1075a4)
    #5 DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/executeQuery.cpp:247:32 (clickhouse+0xc4e1a8e)
    #6 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/executeQuery.cpp:487:38 (clickhouse+0xc4e104a)
    #7 DB::TCPHandler::runImpl() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/TCPHandler.cpp:210:24 (clickhouse+0x6c1d086)
    #8 DB::TCPHandler::run() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/TCPHandler.cpp:1083:9 (clickhouse+0x6c2a777)
    #9 Poco::Net::TCPServerConnection::start() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse+0xcfcd322)
    #10 Poco::Net::TCPServerDispatcher::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerDispatcher.cpp:114:20 (clickhouse+0xcfcdbba)
    #11 Poco::PooledThread::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:214:14 (clickhouse+0xd95f4d1)
    #12 Poco::(anonymous namespace)::RunnableHolder::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread.cpp:43:11 (clickhouse+0xd95d97f)
    #13 Poco::ThreadImpl::runnableEntry(void*) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:139:27 (clickhouse+0xd95c0ea)
    #14 decltype(std::__1::forward<void* (*)(void*)>(fp)(std::__1::forward<Poco::ThreadImpl*>(fp0))) std::__1::__invoke<void* (*)(void*), Poco::ThreadImpl*>(void* (*&&)(void*), Poco::ThreadImpl*&&) /home/milovidov/ClickHouse/contrib/libcxx/include/type_traits:4410:1 (clickhouse+0xd95e3aa)
    #15 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*>&, std::__1::__tuple_indices<2ul>) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:341 (clickhouse+0xd95e3aa)
    #16 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*> >(void*) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:351 (clickhouse+0xd95e3aa)

  Previous write of size 8 at 0x7b2c00130d10 by thread T75:
    #0 operator new(unsigned long) /home/milovidov/ClickHouse/ci/workspace/llvm/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:64 (clickhouse+0x6b27c77)
    #1 std::__1::__libcpp_allocate(unsigned long, unsigned long) /home/milovidov/ClickHouse/contrib/libcxx/include/new:238:10 (clickhouse+0xbd3940c)
    #2 std::__1::allocator<boost::multi_index::detail::sequenced_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::null_augment_policy, boost::multi_index::detail::index_node_base<DB::ColumnDescription, std::__1::allocator<DB::ColumnDescription> > > > >::allocate(unsigned long, void const*) /home/milovidov/ClickHouse/contrib/libcxx/include/memory:1813 (clickhouse+0xbd3940c)
    #3 std::__1::allocator_traits<std::__1::allocator<boost::multi_index::detail::sequenced_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::null_augment_policy, boost::multi_index::detail::index_node_base<DB::ColumnDescription, std::__1::allocator<DB::ColumnDescription> > > > > >::allocate(std::__1::allocator<boost::multi_index::detail::sequenced_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::null_augment_policy, boost::multi_index::detail::index_node_base<DB::ColumnDescription, std::__1::allocator<DB::ColumnDescription> > > > >&, unsigned long) /home/milovidov/ClickHouse/contrib/libcxx/include/memory:1546 (clickhouse+0xbd3940c)
    #4 boost::multi_index::multi_index_container<DB::ColumnDescription, boost::multi_index::indexed_by<boost::multi_index::sequenced<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_unique<boost::multi_index::member<DB::ColumnDescription, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, &(DB::ColumnDescription::name)>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::__1::allocator<DB::ColumnDescription> >::allocate_node() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/boost/boost/multi_index_container.hpp:541 (clickhouse+0xbd3940c)
    #5 boost::multi_index::detail::header_holder<boost::multi_index::detail::sequenced_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::null_augment_policy, boost::multi_index::detail::index_node_base<DB::ColumnDescription, std::__1::allocator<DB::ColumnDescription> > > >*, boost::multi_index::multi_index_container<DB::ColumnDescription, boost::multi_index::indexed_by<boost::multi_index::sequenced<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_unique<boost::multi_index::member<DB::ColumnDescription, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, &(DB::ColumnDescription::name)>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::__1::allocator<DB::ColumnDescription> > >::header_holder() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/boost/boost/multi_index/detail/header_holder.hpp:35 (clickhouse+0xbd3940c)
    #6 boost::multi_index::multi_index_container<DB::ColumnDescription, boost::multi_index::indexed_by<boost::multi_index::sequenced<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_unique<boost::multi_index::member<DB::ColumnDescription, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, &(DB::ColumnDescription::name)>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::__1::allocator<DB::ColumnDescription> >::multi_index_container(boost::multi_index::multi_index_container<DB::ColumnDescription, boost::multi_index::indexed_by<boost::multi_index::sequenced<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_unique<boost::multi_index::member<DB::ColumnDescription, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, &(DB::ColumnDescription::name)>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::__1::allocator<DB::ColumnDescription> > const&) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/boost/boost/multi_index_container.hpp:275 (clickhouse+0xbd3940c)
    #7 DB::ColumnsDescription::ColumnsDescription(DB::ColumnsDescription const&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Storages/ColumnsDescription.h:49:7 (clickhouse+0xc7da8b6)
    #8 DB::MergeTreeData::setPrimaryKeyIndicesAndColumns(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::IAST> const&, DB::ColumnsDescription const&, DB::IndicesDescription const&, bool) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Storages/MergeTree/MergeTreeData.cpp:386 (clickhouse+0xc7da8b6)
    #9 DB::StorageMergeTree::alter(DB::AlterCommands const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context const&, DB::TableStructureWriteLockHolder&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Storages/StorageMergeTree.cpp:290:5 (clickhouse+0xc68d1f4)
    #10 DB::InterpreterAlterQuery::execute() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/InterpreterAlterQuery.cpp:79:16 (clickhouse+0xc107775)
    #11 DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/executeQuery.cpp:247:32 (clickhouse+0xc4e1a8e)
    #12 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/executeQuery.cpp:487:38 (clickhouse+0xc4e104a)
    #13 DB::TCPHandler::runImpl() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/TCPHandler.cpp:210:24 (clickhouse+0x6c1d086)
    #14 DB::TCPHandler::run() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/TCPHandler.cpp:1083:9 (clickhouse+0x6c2a777)
    #15 Poco::Net::TCPServerConnection::start() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse+0xcfcd322)
    #16 Poco::Net::TCPServerDispatcher::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerDispatcher.cpp:114:20 (clickhouse+0xcfcdbba)
    #17 Poco::PooledThread::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:214:14 (clickhouse+0xd95f4d1)
    #18 Poco::(anonymous namespace)::RunnableHolder::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread.cpp:43:11 (clickhouse+0xd95d97f)
    #19 Poco::ThreadImpl::runnableEntry(void*) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:139:27 (clickhouse+0xd95c0ea)
    #20 decltype(std::__1::forward<void* (*)(void*)>(fp)(std::__1::forward<Poco::ThreadImpl*>(fp0))) std::__1::__invoke<void* (*)(void*), Poco::ThreadImpl*>(void* (*&&)(void*), Poco::ThreadImpl*&&) /home/milovidov/ClickHouse/contrib/libcxx/include/type_traits:4410:1 (clickhouse+0xd95e3aa)
    #21 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*>&, std::__1::__tuple_indices<2ul>) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:341 (clickhouse+0xd95e3aa)
    #22 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*> >(void*) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:351 (clickhouse+0xd95e3aa)

  Location is heap block of size 168 at 0x7b2c00130c70 allocated by thread T75:
    #0 operator new(unsigned long) /home/milovidov/ClickHouse/ci/workspace/llvm/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:64 (clickhouse+0x6b27c77)
    #1 std::__1::__libcpp_allocate(unsigned long, unsigned long) /home/milovidov/ClickHouse/contrib/libcxx/include/new:238:10 (clickhouse+0xbd3940c)
    #2 std::__1::allocator<boost::multi_index::detail::sequenced_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::null_augment_policy, boost::multi_index::detail::index_node_base<DB::ColumnDescription, std::__1::allocator<DB::ColumnDescription> > > > >::allocate(unsigned long, void const*) /home/milovidov/ClickHouse/contrib/libcxx/include/memory:1813 (clickhouse+0xbd3940c)
    #3 std::__1::allocator_traits<std::__1::allocator<boost::multi_index::detail::sequenced_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::null_augment_policy, boost::multi_index::detail::index_node_base<DB::ColumnDescription, std::__1::allocator<DB::ColumnDescription> > > > > >::allocate(std::__1::allocator<boost::multi_index::detail::sequenced_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::null_augment_policy, boost::multi_index::detail::index_node_base<DB::ColumnDescription, std::__1::allocator<DB::ColumnDescription> > > > >&, unsigned long) /home/milovidov/ClickHouse/contrib/libcxx/include/memory:1546 (clickhouse+0xbd3940c)
    #4 boost::multi_index::multi_index_container<DB::ColumnDescription, boost::multi_index::indexed_by<boost::multi_index::sequenced<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_unique<boost::multi_index::member<DB::ColumnDescription, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, &(DB::ColumnDescription::name)>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::__1::allocator<DB::ColumnDescription> >::allocate_node() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/boost/boost/multi_index_container.hpp:541 (clickhouse+0xbd3940c)
2019.08.19 21:31:55.170945 [ 59 ] {8002a7f8-f28d-4861-abed-117d0be3a3b6} <Trace> InterpreterSelectQuery: FetchColumns -> Complete    #5 boost::multi_index::detail::header_holder<boost::multi_index::detail::sequenced_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::null_augment_policy, boost::multi_index::detail::index_node_base<DB::ColumnDescription, std::__1::allocator<DB::ColumnDescription> > > >*, boost::multi_index::multi_index_container<DB::ColumnDescription, boost::multi_index::indexed_by<boost::multi_index::sequenced<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_unique<boost::multi_index::member<DB::ColumnDescription, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, &(DB::ColumnDescription::name)>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::__1::allocator<DB::ColumnDescription> > >::header_holder() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/boost/boost/multi_index/detail/header_holder.hpp:35 (clickhouse+0xbd3940c)

    #6 boost::multi_index::multi_index_container<DB::ColumnDescription, boost::multi_index::indexed_by<boost::multi_index::sequenced<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_unique<boost::multi_index::member<DB::ColumnDescription, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, &(DB::ColumnDescription::name)>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::__1::allocator<DB::ColumnDescription> >::multi_index_container(boost::multi_index::multi_index_container<DB::ColumnDescription, boost::multi_index::indexed_by<boost::multi_index::sequenced<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_unique<boost::multi_index::member<DB::ColumnDescription, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, &(DB::ColumnDescription::name)>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::__1::allocator<DB::ColumnDescription> > const&) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/boost/boost/multi_index_container.hpp:275 (clickhouse+0xbd3940c)
    #7 DB::ColumnsDescription::ColumnsDescription(DB::ColumnsDescription const&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Storages/ColumnsDescription.h:49:7 (clickhouse+0xc7da8b6)
    #8 DB::MergeTreeData::setPrimaryKeyIndicesAndColumns(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::IAST> const&, DB::ColumnsDescription const&, DB::IndicesDescription const&, bool) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Storages/MergeTree/MergeTreeData.cpp:386 (clickhouse+0xc7da8b6)
2019.08.19 21:31:55.171048 [ 57 ] {73d4ab35-4a97-46e6-aa49-807f4241bee0} <Debug> MemoryTracker: Peak memory usage (for query): 0.00 B.    #9 DB::StorageMergeTree::alter(DB::AlterCommands const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context const&, DB::TableStructureWriteLockHolder&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Storages/StorageMergeTree.cpp:290:5 (clickhouse+0xc68d1f4)

    #10 DB::InterpreterAlterQuery::execute() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/InterpreterAlterQuery.cpp:79:16 (clickhouse+0xc107775)
    #11 DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/executeQuery.cpp:247:32 (clickhouse+0xc4e1a8e)
    #12 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/executeQuery.cpp:487:38 (clickhouse+0xc4e104a)
    #13 DB::TCPHandler::runImpl() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/TCPHandler.cpp:210:24 (clickhouse+0x6c1d086)
    #14 DB::TCPHandler::run() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/TCPHandler.cpp:1083:9 (clickhouse+0x6c2a777)
    #15 Poco::Net::TCPServerConnection::start() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse+0xcfcd322)
    #16 Poco::Net::TCPServerDispatcher::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerDispatcher.cpp:114:20 (clickhouse+0xcfcdbba)
    #17 Poco::PooledThread::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:214:14 (clickhouse+0xd95f4d1)
    #18 Poco::(anonymous namespace)::RunnableHolder::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread.cpp:43:11 (clickhouse+0xd95d97f)
    #19 Poco::ThreadImpl::runnableEntry(void*) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:139:27 (clickhouse+0xd95c0ea)
    #20 decltype(std::__1::forward<void* (*)(void*)>(fp)(std::__1::forward<Poco::ThreadImpl*>(fp0))) std::__1::__invoke<void* (*)(void*), Poco::ThreadImpl*>(void* (*&&)(void*), Poco::ThreadImpl*&&) /home/milovidov/ClickHouse/contrib/libcxx/include/type_traits:4410:1 (clickhouse+0xd95e3aa)
    #21 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*>&, std::__1::__tuple_indices<2ul>) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:341 (clickhouse+0xd95e3aa)
    #22 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*> >(void*) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:351 (clickhouse+0xd95e3aa)
2019.08.19 21:31:55.171302 [ 57 ] {73d4ab35-4a97-46e6-aa49-807f4241bee0} <Information> TCPHandler: Processed in 0.043 sec.

  Thread T76 'TCPHandler' (tid=258090, running) created by thread T52 at:
    #0 pthread_create /home/milovidov/ClickHouse/ci/workspace/llvm/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:976 (clickhouse+0x6a84d95)
    #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /home/milovidov/ClickHouse/contrib/libcxx/include/__threading_support:327:10 (clickhouse+0xd95dd57)
    #2 std::__1::thread::thread<void* (&)(void*), Poco::ThreadImpl*, void>(void* (&)(void*), Poco::ThreadImpl*&&) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:367 (clickhouse+0xd95dd57)
    #3 Poco::ThreadImpl::startImpl(Poco::SharedPtr<Poco::Runnable, Poco::ReferenceCounter, Poco::ReleasePolicy<Poco::Runnable> >) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:58:53 (clickhouse+0xd95bb87)
    #4 Poco::Thread::start(Poco::Runnable&) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread.cpp:116:2 (clickhouse+0xd95d32c)
    #5 Poco::PooledThread::start(int) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:88:10 (clickhouse+0xd960e79)
    #6 Poco::ThreadPool::getThread() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:525 (clickhouse+0xd960e79)
    #7 Poco::ThreadPool::startWithPriority(Poco::Thread::Priority, Poco::Runnable&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:429:2 (clickhouse+0xd96113e)
    #8 Poco::Net::TCPServerDispatcher::enqueue(Poco::Net::StreamSocket const&) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerDispatcher.cpp:144:17 (clickhouse+0xcfce120)
    #9 Poco::Net::TCPServer::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServer.cpp:148:21 (clickhouse+0xcfccd15)
    #10 Poco::(anonymous namespace)::RunnableHolder::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread.cpp:43:11 (clickhouse+0xd95d97f)
    #11 Poco::ThreadImpl::runnableEntry(void*) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:139:27 (clickhouse+0xd95c0ea)
    #12 decltype(std::__1::forward<void* (*)(void*)>(fp)(std::__1::forward<Poco::ThreadImpl*>(fp0))) std::__1::__invoke<void* (*)(void*), Poco::ThreadImpl*>(void* (*&&)(void*), Poco::ThreadImpl*&&) /home/milovidov/ClickHouse/contrib/libcxx/include/type_traits:4410:1 (clickhouse+0xd95e3aa)
    #13 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*>&, std::__1::__tuple_indices<2ul>) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:341 (clickhouse+0xd95e3aa)
    #14 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*> >(void*) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:351 (clickhouse+0xd95e3aa)

  Thread T75 'TCPHandler' (tid=258089, running) created by thread T52 at:
    #0 pthread_create /home/milovidov/ClickHouse/ci/workspace/llvm/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:976 (clickhouse+0x6a84d95)
    #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /home/milovidov/ClickHouse/contrib/libcxx/include/__threading_support:327:10 (clickhouse+0xd95dd57)
2019.08.19 21:31:55.171622 [ 57 ] {} <Information> TCPHandler: Done processing connection.    #2 std::__1::thread::thread<void* (&)(void*), Poco::ThreadImpl*, void>(void* (&)(void*), Poco::ThreadImpl*&&) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:367 (clickhouse+0xd95dd57)

    #3 Poco::ThreadImpl::startImpl(Poco::SharedPtr<Poco::Runnable, Poco::ReferenceCounter, Poco::ReleasePolicy<Poco::Runnable> >) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:58:53 (clickhouse+0xd95bb87)
    #4 Poco::Thread::start(Poco::Runnable&) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread.cpp:116:2 (clickhouse+0xd95d32c)
    #5 Poco::PooledThread::start(int) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:88:10 (clickhouse+0xd960e79)
    #6 Poco::ThreadPool::getThread() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:525 (clickhouse+0xd960e79)
    #7 Poco::ThreadPool::startWithPriority(Poco::Thread::Priority, Poco::Runnable&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:429:2 (clickhouse+0xd96113e)
    #8 Poco::Net::TCPServerDispatcher::enqueue(Poco::Net::StreamSocket const&) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerDispatcher.cpp:144:17 (clickhouse+0xcfce120)
    #9 Poco::Net::TCPServer::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServer.cpp:148:21 (clickhouse+0xcfccd15)
    #10 Poco::(anonymous namespace)::RunnableHolder::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread.cpp:43:11 (clickhouse+0xd95d97f)
    #11 Poco::ThreadImpl::runnableEntry(void*) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:139:27 (clickhouse+0xd95c0ea)
    #12 decltype(std::__1::forward<void* (*)(void*)>(fp)(std::__1::forward<Poco::ThreadImpl*>(fp0))) std::__1::__invoke<void* (*)(void*), Poco::ThreadImpl*>(void* (*&&)(void*), Poco::ThreadImpl*&&) /home/milovidov/ClickHouse/contrib/libcxx/include/type_traits:4410:1 (clickhouse+0xd95e3aa)
    #13 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*>&, std::__1::__tuple_indices<2ul>) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:341 (clickhouse+0xd95e3aa)
    #14 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*> >(void*) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:351 (clickhouse+0xd95e3aa)

SUMMARY: ThreadSanitizer: data race /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/boost/boost/multi_index/sequenced_index.hpp:223:58 in boost::multi_index::detail::sequenced_index<boost::multi_index::detail::nth_layer<1, DB::ColumnDescription, boost::multi_index::indexed_by<boost::multi_index::sequenced<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_unique<boost::multi_index::member<DB::ColumnDescription, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, &(DB::ColumnDescription::name)>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::__1::allocator<DB::ColumnDescription> >, boost::mpl::vector0<mpl_::na
==================

…ickHouse into added-test-for-race-conditions

alexey-milovidov · 2019-08-20T00:17:21Z

Something like race condition of system.parts with DROP table:

(gdb) bt
#0  __GI___pthread_mutex_lock (mutex=0x14b0) at ../nptl/pthread_mutex_lock.c:65
#1  0x0000000016c0173b in std::__1::__libcpp_mutex_lock (__m=0x14b0) at ../contrib/libcxx/include/__threading_support:255
#2  std::__1::mutex::lock (this=0x14b0) at ../contrib/libcxx/src/mutex.cpp:30
#3  0x00000000135d1a2c in std::__1::unique_lock<std::__1::mutex>::unique_lock (this=<optimized out>, __m=...) at /home/milovidov/ClickHouse/contrib/libcxx/include/__mutex_base:130
#4  DB::MergeTreeData::lockParts (this=0x0) at ../dbms/src/Storages/MergeTree/MergeTreeData.h:173
#5  DB::MergeTreeData::getDataPartsVector (this=0x0, affordable_states=..., out_states=0x7fff573cb8c0) at ../dbms/src/Storages/MergeTree/MergeTreeData.cpp:2578
#6  0x0000000011dbd3c4 in DB::StoragesInfo::getParts (this=<optimized out>, state=..., has_state_column=false) at ../dbms/src/Storages/System/StorageSystemPartsBase.cpp:48
#7  0x0000000011db30f8 in DB::StorageSystemParts::processNextStorage (this=0x612000004858, columns_=..., info=..., has_state_column=false) at ../dbms/src/Storages/System/StorageSystemParts.cpp:64
#8  0x0000000011dc5f10 in DB::StorageSystemPartsBase::read (this=0x612000004858, column_names=..., query_info=..., context=...) at ../dbms/src/Storages/System/StorageSystemPartsBase.cpp:226
#9  0x00000000126effd6 in DB::InterpreterSelectQuery::executeFetchColumns<DB::InterpreterSelectQuery::Pipeline> (this=<optimized out>, processing_stage=DB::QueryProcessingStage::FetchColumns, pipeline=..., sorting_info=..., prewhere_info=..., 
    columns_to_remove_after_prewhere=...) at ../dbms/src/Interpreters/InterpreterSelectQuery.cpp:1397
#10 0x00000000126cbb8a in DB::InterpreterSelectQuery::executeImpl<DB::InterpreterSelectQuery::Pipeline> (this=0x6200001d1080, pipeline=..., prepared_input=..., dry_run=false) at ../dbms/src/Interpreters/InterpreterSelectQuery.cpp:863
#11 0x0000000012691bef in DB::InterpreterSelectQuery::executeWithMultipleStreams (this=0x6200001d1080) at ../dbms/src/Interpreters/InterpreterSelectQuery.cpp:395
#12 0x0000000012aa9eb2 in DB::InterpreterSelectWithUnionQuery::executeWithMultipleStreams (this=0x620000206080) at ../dbms/src/Interpreters/InterpreterSelectWithUnionQuery.cpp:173
#13 0x0000000012aaa99d in DB::InterpreterSelectWithUnionQuery::execute (this=0x620000206080) at ../dbms/src/Interpreters/InterpreterSelectWithUnionQuery.cpp:190
#14 0x0000000012e92328 in DB::executeQueryImpl (begin=<optimized out>, end=0x7fff573d1260 "", context=..., internal=false, stage=DB::QueryProcessingStage::Complete, has_query_tail=<optimized out>) at ../dbms/src/Interpreters/executeQuery.cpp:247
#15 0x0000000012e9054a in DB::executeQuery (query=..., context=..., internal=false, stage=DB::QueryProcessingStage::Complete, may_have_embedded_data=<optimized out>) at ../dbms/src/Interpreters/executeQuery.cpp:487
#16 0x00000000077a122e in DB::TCPHandler::runImpl (this=0x624000dfc100) at ../dbms/programs/server/TCPHandler.cpp:210
#17 0x00000000077c0b23 in DB::TCPHandler::run (this=0x624000dfc100) at ../dbms/programs/server/TCPHandler.cpp:1083
#18 0x00000000146b273f in Poco::Net::TCPServerConnection::start (this=0x14b0) at ../contrib/poco/Net/src/TCPServerConnection.cpp:43
#19 0x00000000146b33e0 in Poco::Net::TCPServerDispatcher::run (this=<optimized out>) at ../contrib/poco/Net/src/TCPServerDispatcher.cpp:114
#20 0x00000000158f7685 in Poco::PooledThread::run (this=<optimized out>) at ../contrib/poco/Foundation/src/ThreadPool.cpp:214
#21 0x00000000158f1d2f in Poco::ThreadImpl::runnableEntry (pThread=0x617000312738) at ../contrib/poco/Foundation/src/Thread_STD.cpp:139
#22 0x00000000158f5dbe in std::__1::__invoke<void* (*)(void*), Poco::ThreadImpl*> (__f=@0x7fff573cb5e0: 0x41b58ab3, __args=<optimized out>) at /home/milovidov/ClickHouse/contrib/libcxx/include/type_traits:4410
#23 std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*, 2ul> (__t=...) at /home/milovidov/ClickHouse/contrib/libcxx/include/thread:341
#24 std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*> > (__vp=0x6030005894e0) at /home/milovidov/ClickHouse/contrib/libcxx/include/thread:351
#25 0x00007ffff77b16db in start_thread (arg=0x7fff573da700) at pthread_create.c:463
#26 0x00007ffff713c88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

…ickHouse into added-test-for-race-conditions

alexey-milovidov · 2019-08-20T00:55:07Z

Fixed in 99aa5cc

alexey-milovidov · 2019-08-20T01:15:41Z

Deadlock:

┌───────elapsed─┬─query───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─query_id─────────────────────────────┐
│ 789.408521682 │ OPTIMIZE TABLE alter_table_3 FINAL                                                                                                                                                      │ 580eb176-e31e-43b6-8387-51dc92af3b94 │
│  789.30598735 │ INSERT INTO alter_table_3 SELECT rand(1), rand(2), 1 / rand(3), toString(rand(4)), [rand(5), rand(6)], if(rand(7) % 2, NULL, generateUUIDv4()), (rand(8), rand(9)) FROM numbers(100000) │ fef93667-b1a9-485f-a15f-09a9f9d5ae16 │
│ 788.975409483 │ INSERT INTO alter_table_3 SELECT rand(1), rand(2), 1 / rand(3), toString(rand(4)), [rand(5), rand(6)], if(rand(7) % 2, NULL, generateUUIDv4()), (rand(8), rand(9)) FROM numbers(100000) │ 7bc0eeb9-eb08-4ef8-98a8-8506590cba03 │
│ 788.952671382 │ ALTER TABLE alter_table_3 ADD COLUMN `h` String                                                                                                                                         │ c78a66c2-35a8-4af4-8f4c-9a4277a431a4 │
│ 788.846679259 │ INSERT INTO alter_table_3 SELECT rand(1), rand(2), 1 / rand(3), toString(rand(4)), [rand(5), rand(6)], if(rand(7) % 2, NULL, generateUUIDv4()), (rand(8), rand(9)) FROM numbers(100000) │ 3c66e8d4-7324-4e0a-92eb-7666bb9ac31c │
│ 788.823968155 │ OPTIMIZE TABLE alter_table_3 FINAL                                                                                                                                                      │ 7adebb58-83b7-45af-90bc-0fe741fb6f84 │
│ 788.712727326 │ INSERT INTO alter_table_3 SELECT rand(1), rand(2), 1 / rand(3), toString(rand(4)), [rand(5), rand(6)], if(rand(7) % 2, NULL, generateUUIDv4()), (rand(8), rand(9)) FROM numbers(100000) │ fdb409f3-0166-4481-91dc-6d80f3e49094 │
│ 788.596953239 │ DROP TABLE IF EXISTS alter_table_3                                                                                                                                                      │ 74a60103-7622-4beb-a90b-a5bdb7bde1a7 │
│ 788.533581021 │ DROP TABLE IF EXISTS alter_table_3                                                                                                                                                      │ a9db592b-70ab-4dc7-a738-d773cd27fb86 │
│ 788.374052798 │ OPTIMIZE TABLE alter_table_3 FINAL                                                                                                                                                      │ 05ee376d-53a9-44d6-ba13-c6d4cbc9897e │
│  788.15851403 │ ALTER TABLE alter_table_3 ADD COLUMN `h` String                                                                                                                                         │ 4bafea58-723d-47b0-8308-12f8545a5b73 │
│ 787.886709946 │ ALTER TABLE alter_table_3 ADD COLUMN `h` String                                                                                                                                         │ e4edbe07-59b7-4b72-a839-f56c1a38c7d7 │
│ 787.284257847 │ ALTER TABLE alter_table_3 ADD COLUMN `h` String                                                                                                                                         │ 8e7a215e-5fcf-44e1-9fa4-f85ef2a2b645 │
│ 786.582295149 │ DROP TABLE IF EXISTS alter_table_3                                                                                                                                                      │ dc8a150f-95e1-4b73-a4dd-d3f37f1358d9 │
│ 786.112427378 │ DROP TABLE IF EXISTS alter_table_3                                                                                                                                                      │ 02868aee-4ebe-422d-a51a-5e00294f31ef │
│ 785.780651707 │ OPTIMIZE TABLE alter_table_3 FINAL                                                                                                                                                      │ ec8c1bf6-4295-47c8-842c-58c09138137d │
│ 718.944775558 │ DROP TABLE IF EXISTS alter_table_3                                                                                                                                                      │ deaf8c1d-82b3-4521-8c8e-74fc1114ada3 │
│   0.001800084 │ SELECT elapsed, query, query_id FROM system.processes                                                                                                                                   │ 96e3cedb-166e-46dd-a35b-860ec6b6b6e1 │
└───────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────────┘

| | | | | | | | | | | | | 17   0x0000000012679aba in DB::InterpreterOptimizeQuery::execute at ../dbms/src/Interpreters/InterpreterOptimizeQuery.cpp:27
| | | | | | | | | | | | | | 1    0x000000001332878c in DB::StorageReplicatedMergeTree::optimize at ../dbms/src/Storages/StorageReplicatedMergeTree.cpp:3091
| | | | | | | | | | | | | | | 1    0x0000000013330d1f in DB::StorageReplicatedMergeTree::waitForAllReplicasToProcessLogEntry at ../dbms/src/Storages/StorageReplicatedMergeTree.cpp:3795
| | | | | | | | | | | | | | | | 1    0x000000001337d2d4 in DB::StorageReplicatedMergeTree::waitForReplicaToProcessLogEntry at ../dbms/src/Storages/StorageReplicatedMergeTree.cpp:3927
| | | | | | | | | | | | | | | | | 1    0x0000000013f4012d in zkutil::ZooKeeper::waitForDisappear at ../dbms/src/Common/ZooKeeper/ZooKeeper.cpp:659
| | | | | | | | | | | | | | | | | | 1    Poco::Event::wait at ../contrib/poco/Foundation/src/Event.cpp:79
| | | | | | | | | | | | | | | | | | | 1    0x00000000157c2d22 in std::__1::condition_variable::wait<Poco::Event::wait()::$_0>(std::__1::unique_lock<std::__1::mutex>&, Poco::Event::wait()::$_0) at /home/milovidov/ClickHouse/contrib/libcxx/include/__mutex_base:374
| | | | | | | | | | | | | | | | | | | | 1    std::__1::condition_variable::wait at ../contrib/libcxx/src/condition_variable.cpp:43
| | | | | | | | | | | | | | | | | | | | | 1    0x0000000016bae248 in std::__1::__libcpp_condvar_wait at ../contrib/libcxx/include/__threading_support:286
| | | | | | | | | | | | | | | | | | | | | | 1    __pthread_cond_wait at pthread_cond_wait.c:655
| | | | | | | | | | | | | | | | | | | | | | | 1    __pthread_cond_wait_common at pthread_cond_wait.c:502
| | | | | | | | | | | | | | | | | | | | | | | | 1    0x00007f259eb989f3 in futex_wait_cancelable at ../sysdeps/unix/sysv/linux/futex-internal.h:88

| | | | | | | | | | | | | 17   0x00000000126799a1 in DB::InterpreterOptimizeQuery::execute at ../dbms/src/Interpreters/InterpreterOptimizeQuery.cpp:26
| | | | | | | | | | | | | | 3    0x000000001311db21 in DB::IStorage::lockStructureForShare at ../dbms/src/Storages/IStorage.cpp:313
| | | | | | | | | | | | | | | 3    DB::RWLockImpl::getLock at ../dbms/src/Common/RWLock.cpp:126
| | | | | | | | | | | | | | | | 3    0x0000000013128b7c in std::__1::condition_variable::wait<DB::RWLockImpl::getLock(DB::RWLockImpl::Type, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_1>(std::__1::unique_lock<std::__1::mutex>&, DB::RWLockImpl::getLock(DB::RWLockImpl::Type, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_1) at /home/milovidov/ClickHouse/contrib/libcxx/include/__mutex_base:374
| | | | | | | | | | | | | | | | | 3    std::__1::condition_variable::wait at ../contrib/libcxx/src/condition_variable.cpp:43
| | | | | | | | | | | | | | | | | | 3    0x0000000016bae248 in std::__1::__libcpp_condvar_wait at ../contrib/libcxx/include/__threading_support:286
| | | | | | | | | | | | | | | | | | | 3    __pthread_cond_wait at pthread_cond_wait.c:655
| | | | | | | | | | | | | | | | | | | | 3    __pthread_cond_wait_common at pthread_cond_wait.c:502
| | | | | | | | | | | | | | | | | | | | | 3    0x00007f259eb989f3 in futex_wait_cancelable at ../sysdeps/unix/sysv/linux/futex-internal.h:88

| | | | | | | | | | | | | 17   0x000000001262bb93 in DB::InterpreterAlterQuery::execute at ../dbms/src/Interpreters/InterpreterAlterQuery.cpp:78
| | | | | | | | | | | | | | 4    0x000000001311e3cd in DB::IStorage::lockAlterIntention at ../dbms/src/Storages/IStorage.cpp:324
| | | | | | | | | | | | | | | 4    DB::RWLockImpl::getLock at ../dbms/src/Common/RWLock.cpp:126
| | | | | | | | | | | | | | | | 4    0x0000000013128b7c in std::__1::condition_variable::wait<DB::RWLockImpl::getLock(DB::RWLockImpl::Type, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_1>(std::__1::unique_lock<std::__1::mutex>&, DB::RWLockImpl::getLock(DB::RWLockImpl::Type, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_1) at /home/milovidov/ClickHouse/contrib/libcxx/include/__mutex_base:374
| | | | | | | | | | | | | | | | | 4    std::__1::condition_variable::wait at ../contrib/libcxx/src/condition_variable.cpp:43
| | | | | | | | | | | | | | | | | | 4    0x0000000016bae248 in std::__1::__libcpp_condvar_wait at ../contrib/libcxx/include/__threading_support:286
| | | | | | | | | | | | | | | | | | | 4    __pthread_cond_wait at pthread_cond_wait.c:655
| | | | | | | | | | | | | | | | | | | | 4    __pthread_cond_wait_common at pthread_cond_wait.c:502
| | | | | | | | | | | | | | | | | | | | | 4    0x00007f259eb989f3 in futex_wait_cancelable at ../sysdeps/unix/sysv/linux/futex-internal.h:88

| | | | | | | | | | | | | 17   0x000000001264d0ec in DB::InterpreterInsertQuery::execute at ../dbms/src/Interpreters/InterpreterInsertQuery.cpp:102
| | | | | | | | | | | | | | 4    0x000000001311db21 in DB::IStorage::lockStructureForShare at ../dbms/src/Storages/IStorage.cpp:313
| | | | | | | | | | | | | | | 4    DB::RWLockImpl::getLock at ../dbms/src/Common/RWLock.cpp:126
| | | | | | | | | | | | | | | | 4    0x0000000013128b7c in std::__1::condition_variable::wait<DB::RWLockImpl::getLock(DB::RWLockImpl::Type, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_1>(std::__1::unique_lock<std::__1::mutex>&, DB::RWLockImpl::getLock(DB::RWLockImpl::Type, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_1) at /home/milovidov/ClickHouse/contrib/libcxx/include/__mutex_base:374
| | | | | | | | | | | | | | | | | 4    std::__1::condition_variable::wait at ../contrib/libcxx/src/condition_variable.cpp:43
| | | | | | | | | | | | | | | | | | 4    0x0000000016bae248 in std::__1::__libcpp_condvar_wait at ../contrib/libcxx/include/__threading_support:286
| | | | | | | | | | | | | | | | | | | 4    __pthread_cond_wait at pthread_cond_wait.c:655
| | | | | | | | | | | | | | | | | | | | 4    __pthread_cond_wait_common at pthread_cond_wait.c:502
| | | | | | | | | | | | | | | | | | | | | 4    0x00007f259eb989f3 in futex_wait_cancelable at ../sysdeps/unix/sysv/linux/futex-internal.h:88

| | | | | | | | | | | | | 17   0x000000001261b529 in DB::InterpreterDropQuery::execute at ../dbms/src/Interpreters/InterpreterDropQuery.cpp:42
| | | | | | | | | | | | | | 5    0x000000001261c6fc in DB::InterpreterDropQuery::executeToTable at ../dbms/src/Interpreters/InterpreterDropQuery.cpp:62
| | | | | | | | | | | | | | | 4    DB::Context::getDDLGuard at ../dbms/src/Interpreters/Context.cpp:1028
| | | | | | | | | | | | | | | | 4    0x00000000123bd575 in std::__1::make_unique<DB::DDLGuard, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::DDLGuard::Entry, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, DB::DDLGuard::Entry> > >&, std::__1::unique_lock<std::__1::mutex>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&> at /home/milovidov/ClickHouse/contrib/libcxx/include/memory:3131
| | | | | | | | | | | | | | | | | 4    DB::DDLGuard::DDLGuard at ../dbms/src/Interpreters/Context.cpp:1011
| | | | | | | | | | | | | | | | | | 4    0x00000000123bcd2e in std::__1::unique_lock<std::__1::mutex>::unique_lock at /home/milovidov/ClickHouse/contrib/libcxx/include/__mutex_base:130
| | | | | | | | | | | | | | | | | | | 4    std::__1::mutex::lock at ../contrib/libcxx/src/mutex.cpp:30
| | | | | | | | | | | | | | | | | | | | 4    0x0000000016c018bb in std::__1::__libcpp_mutex_lock at ../contrib/libcxx/include/__threading_support:255
| | | | | | | | | | | | | | | | | | | | | 4    0x00007f259eb95023 in __GI___pthread_mutex_lock at ../nptl/pthread_mutex_lock.c:78
| | | | | | | | | | | | | | | | | | | | | | 4    __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135

| | | | | | | | | | | | | | 5    0x000000001261cf43 in DB::InterpreterDropQuery::executeToTable at ../dbms/src/Interpreters/InterpreterDropQuery.cpp:89
| | | | | | | | | | | | | | | 1    0x000000001331f7e8 in DB::StorageReplicatedMergeTree::shutdown at ../dbms/src/Storages/StorageReplicatedMergeTree.cpp:2885
| | | | | | | | | | | | | | | | 1    0x0000000013995715 in DB::ReplicatedMergeTreeRestartingThread::shutdown at ../dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:361
| | | | | | | | | | | | | | | | | 1    DB::ReplicatedMergeTreeRestartingThread::partialShutdown at ../dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:346
| | | | | | | | | | | | | | | | | | 1    0x00000000139883fb in DB::ReplicatedMergeTreeAlterThread::stop at ../dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.h:29
| | | | | | | | | | | | | | | | | | | 1    DB::BackgroundSchedulePoolTaskInfo::deactivate at ../dbms/src/Core/BackgroundSchedulePool.cpp:64
| | | | | | | | | | | | | | | | | | | | 1    0x000000001240107d in std::__1::lock_guard<std::__1::mutex>::lock_guard at /home/milovidov/ClickHouse/contrib/libcxx/include/__mutex_base:103
| | | | | | | | | | | | | | | | | | | | | 1    std::__1::mutex::lock at ../contrib/libcxx/src/mutex.cpp:30
| | | | | | | | | | | | | | | | | | | | | | 1    0x0000000016c018bb in std::__1::__libcpp_mutex_lock at ../contrib/libcxx/include/__threading_support:255
| | | | | | | | | | | | | | | | | | | | | | | 1    0x00007f259eb95023 in __GI___pthread_mutex_lock at ../nptl/pthread_mutex_lock.c:78
| | | | | | | | | | | | | | | | | | | | | | | | 1    __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135

| | | | | | | | | | | | | | | | | | 16   DB::BackgroundSchedulePool::threadFunction at ../dbms/src/Core/BackgroundSchedulePool.cpp:262
| | | | | | | | | | | | | | | | | | | 1    0x00000000124079d0 in DB::TaskNotification::execute at ../dbms/src/Core/BackgroundSchedulePool.cpp:27
| | | | | | | | | | | | | | | | | | | | 1    DB::BackgroundSchedulePoolTaskInfo::execute at ../dbms/src/Core/BackgroundSchedulePool.cpp:112
| | | | | | | | | | | | | | | | | | | | | 1    std::__1::function<void ()>::operator()() const at /home/milovidov/ClickHouse/contrib/libcxx/include/functional:2347
| | | | | | | | | | | | | | | | | | | | | | 1    0x0000000012401a85 in std::__1::__function::__value_func<void ()>::operator()() const at /home/milovidov/ClickHouse/contrib/libcxx/include/functional:1799
| | | | | | | | | | | | | | | | | | | | | | | 1    0x0000000013899c19 in DB::ReplicatedMergeTreeAlterThread::run at ../dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp:109
| | | | | | | | | | | | | | | | | | | | | | | | 1    0x000000001311fc97 in DB::IStorage::lockExclusively at ../dbms/src/Storages/IStorage.cpp:357
| | | | | | | | | | | | | | | | | | | | | | | | | 1    DB::RWLockImpl::getLock at ../dbms/src/Common/RWLock.cpp:126
| | | | | | | | | | | | | | | | | | | | | | | | | | 1    0x0000000013128b7c in std::__1::condition_variable::wait<DB::RWLockImpl::getLock(DB::RWLockImpl::Type, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_1>(std::__1::unique_lock<std::__1::mutex>&, DB::RWLockImpl::getLock(DB::RWLockImpl::Type, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_1) at /home/milovidov/ClickHouse/contrib/libcxx/include/__mutex_base:374
| | | | | | | | | | | | | | | | | | | | | | | | | | | 1    std::__1::condition_variable::wait at ../contrib/libcxx/src/condition_variable.cpp:43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | 1    0x0000000016bae248 in std::__1::__libcpp_condvar_wait at ../contrib/libcxx/include/__threading_support:286
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1    __pthread_cond_wait at pthread_cond_wait.c:655
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1    __pthread_cond_wait_common at pthread_cond_wait.c:502
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1    0x00007f259eb989f3 in futex_wait_cancelable at ../sysdeps/unix/sysv/linux/futex-internal.h:88

…ickHouse into added-test-for-race-conditions

…is one

alexey-milovidov · 2019-08-20T18:18:59Z

The only thing left is internal assert in thread sanitizer.

alexey-milovidov · 2019-08-20T18:29:55Z

FATAL: ThreadSanitizer CHECK failed: /build/llvm-toolchain-8-9L7oe3/llvm-toolchain-8-8/projects/compiler-rt/lib/sanitizer_common/sanitizer_deadlock_detector.h:69 "((n_all_locks_)) < (((sizeof(all_locks_with_contexts_)/sizeof((all_locks_with_contexts_)[0]))))" (0x40, 0x40)
    #0 __tsan::TsanCheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (clickhouse+0x6ab6a9e)
    #1 __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (clickhouse+0x6acd20f)
    #2 __sanitizer::DeadlockDetectorTLS<__sanitizer::TwoLevelBitVector<1ul, __sanitizer::BasicBitVector<unsigned long> > >::addLock(unsigned long, unsigned long, unsigned int) <null> (clickhouse+0x6ac1d3c)
    #3 __sanitizer::DD::MutexAfterLock(__sanitizer::DDCallback*, __sanitizer::DDMutex*, bool, bool) <null> (clickhouse+0x6ac12ca)
    #4 __tsan::MutexPostLock(__tsan::ThreadState*, unsigned long, unsigned long, unsigned int, int) <null> (clickhouse+0x6ab4f93)
    #5 pthread_mutex_lock <null> (clickhouse+0x6a6f7a0)
    #6 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) /home/milovidov/ClickHouse/build_clang8_tsan/../contrib/libcxx/include/__threading_support:255:10 (clickhouse+0xe21d869)
    #7 std::__1::mutex::lock() /home/milovidov/ClickHouse/build_clang8_tsan/../contrib/libcxx/src/mutex.cpp:30 (clickhouse+0xe21d869)
    #8 std::__1::unique_lock<std::__1::mutex>::unique_lock(std::__1::mutex&) /home/milovidov/ClickHouse/contrib/libcxx/include/__mutex_base:130:61 (clickhouse+0x6b6b43d)
    #9 void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>) /home/milovidov/ClickHouse/build_clang8_tsan/../dbms/src/Common/ThreadPool.cpp:79 (clickhouse+0x6b6b43d)
    #10 ThreadPoolImpl<ThreadFromGlobalPool>::schedule(std::__1::function<void ()>, int) /home/milovidov/ClickHouse/build_clang8_tsan/../dbms/src/Common/ThreadPool.cpp:126:5 (clickhouse+0x6b6b350)
    #11 DB::StorageMergeTree::prepareAlterTransactions(DB::ColumnsDescription const&, DB::IndicesDescription const&, DB::Context const&) /home/milovidov/ClickHouse/build_clang8_tsan/../dbms/src/Storages/StorageMergeTree.cpp:225:26 (clickhouse+0xc6fab8d)
    #12 DB::StorageMergeTree::alter(DB::AlterCommands const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context const&, DB::TableStructureWriteLockHolder&) /home/milovidov/ClickHouse/build_clang8_tsan/../dbms/src/Storages/StorageMergeTree.cpp:275:25 (clickhouse+0xc6fb40a)
    #13 DB::InterpreterAlterQuery::execute() /home/milovidov/ClickHouse/build_clang8_tsan/../dbms/src/Interpreters/InterpreterAlterQuery.cpp:100:16 (clickhouse+0xc15d9da)
    #14 DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /home/milovidov/ClickHouse/build_clang8_tsan/../dbms/src/Interpreters/executeQuery.cpp:247:32 (clickhouse+0xc54080e)
    #15 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /home/milovidov/ClickHouse/build_clang8_tsan/../dbms/src/Interpreters/executeQuery.cpp:487:38 (clickhouse+0xc53fdca)
    #16 DB::TCPHandler::runImpl() /home/milovidov/ClickHouse/build_clang8_tsan/../dbms/programs/server/TCPHandler.cpp:210:24 (clickhouse+0x6bd0c16)
    #17 DB::TCPHandler::run() /home/milovidov/ClickHouse/build_clang8_tsan/../dbms/programs/server/TCPHandler.cpp:1083:9 (clickhouse+0x6bde387)
    #18 Poco::Net::TCPServerConnection::start() /home/milovidov/ClickHouse/build_clang8_tsan/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse+0xd0492a2)
    #19 Poco::Net::TCPServerDispatcher::run() /home/milovidov/ClickHouse/build_clang8_tsan/../contrib/poco/Net/src/TCPServerDispatcher.cpp:114:20 (clickhouse+0xd049b3a)
    #20 Poco::PooledThread::run() /home/milovidov/ClickHouse/build_clang8_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:214:14 (clickhouse+0xd9e4161)
    #21 Poco::(anonymous namespace)::RunnableHolder::run() /home/milovidov/ClickHouse/build_clang8_tsan/../contrib/poco/Foundation/src/Thread.cpp:43:11 (clickhouse+0xd9e260f)
    #22 Poco::ThreadImpl::runnableEntry(void*) /home/milovidov/ClickHouse/build_clang8_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:139:27 (clickhouse+0xd9e0d7a)
    #23 decltype(std::__1::forward<void* (*)(void*)>(fp)(std::__1::forward<Poco::ThreadImpl*>(fp0))) std::__1::__invoke<void* (*)(void*), Poco::ThreadImpl*>(void* (*&&)(void*), Poco::ThreadImpl*&&) /home/milovidov/ClickHouse/contrib/libcxx/include/type_traits:4410:1 (clickhouse+0xd9e303a)
    #24 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*>&, std::__1::__tuple_indices<2ul>) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:341 (clickhouse+0xd9e303a)
    #25 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*> >(void*) /home/milovidov/ClickHouse/contrib/libcxx/include/thread:351 (clickhouse+0xd9e303a)
    #26 __tsan_thread_start_func <null> (clickhouse+0x6a53422)
    #27 start_thread <null> (libpthread.so.0+0x76da)
    #28 clone /build/glibc-OTsEL5/glibc-2.27/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95 (libc.so.6+0x12188e)

alexey-milovidov · 2019-08-20T18:35:04Z

https://github.com/llvm-mirror/compiler-rt/blob/master/lib/sanitizer_common/sanitizer_deadlock_detector.h#L123

LockWithContext all_locks_with_contexts_[64];

alexey-milovidov · 2019-08-20T18:36:09Z

google/sanitizers#950

Added a test for race conditions.

alexey-milovidov · 2019-08-23T01:47:10Z

https://clickhouse-test-reports.s3.yandex.net/6343/086dd815ac19070e119643929164663aa196c035/functional_stateless_tests_(ubsan).html

alexey-milovidov and others added 3 commits August 16, 2019 04:28

Added failing test

ab827ca

Added another test

8d3ede9

Update 00991_system_parts_race_condition.sh

ebdc0eb

alexey-milovidov mentioned this pull request Aug 17, 2019

Fix race condition in system.parts vs. ALTER #6513

Merged

alexey-milovidov added 6 commits August 19, 2019 20:43

Merge branch 'master' into added-test-for-race-conditions

481b9d3

Fixed strange code in mutations that may lead to memory corruption

d2e5789

Attempt to fix race condition

22a1134

Updated instruction

4809586

Whitespaces

5575be0

Updated test

5d17e0e

alexey-milovidov added 2 commits August 20, 2019 03:10

Added another test that produce segfault under ASan easily

4886758

Merge branch 'added-test-for-race-conditions' of github.com:yandex/Cl…

6d8161e

…ickHouse into added-test-for-race-conditions

alexey-milovidov added 3 commits August 20, 2019 03:49

Fixed idiotic error in system.parts

99aa5cc

Merge branch 'added-test-for-race-conditions' of github.com:yandex/Cl…

666029d

…ickHouse into added-test-for-race-conditions

Improved test

ebbcd26

alexey-milovidov added 7 commits August 20, 2019 04:24

Attempt to fix deadlock

a574727

Merge branch 'added-test-for-race-conditions' of github.com:yandex/Cl…

818ceec

…ickHouse into added-test-for-race-conditions

Added a comment

b7cb55a

Fixed typo

182708e

Merge branch 'master' into added-test-for-race-conditions

c8859fe

Merge branch 'master' into added-test-for-race-conditions

593487c

Parallel ALTER modify: don't use ThreadPool if the number of threads …

295aafb

…is one

Checkpoint

3fa67f4

alexey-milovidov merged commit 58358ec into master Aug 20, 2019

alexey-milovidov added the pr-bugfix Pull request with bugfix, not backported by default label Aug 20, 2019

alexey-milovidov added a commit that referenced this pull request Aug 20, 2019

Merge pull request #6514 from yandex/added-test-for-race-conditions

a9ef0a5

Added a test for race conditions.

alexey-milovidov added a commit that referenced this pull request Aug 20, 2019

Merge pull request #6514 from yandex/added-test-for-race-conditions

c0cf605

Added a test for race conditions.

alexey-milovidov mentioned this pull request Aug 27, 2019

Segmentation fault in 19.8.3.8 #6683

Closed

abyss7 added v19.11 labels Sep 5, 2019

azat mentioned this pull request Mar 14, 2021

[RFC] Fix concurrent OPTIMIZE and DROP for ReplicatedMergeTree #21716

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added a test for race conditions.#6514

Added a test for race conditions.#6514
alexey-milovidov merged 22 commits intomasterfrom
added-test-for-race-conditions

alexey-milovidov commented Aug 16, 2019 •

edited

Loading

Uh oh!

alexey-milovidov commented Aug 19, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 23, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

alexey-milovidov commented Aug 16, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexey-milovidov commented Aug 19, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 20, 2019

Uh oh!

alexey-milovidov commented Aug 23, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

alexey-milovidov commented Aug 16, 2019 •

edited

Loading