MATERIALIZE INDEX may stuck due to race with DROP INDEX #38643
Description
Can be reproduced with test 02067_lost_part_s3, it's temporarily disabled: 7a1346a
If you run
ALTER TABLE partslost_0 ADD INDEX idx x TYPE tokenbf_v1(285000, 3, 12345) GRANULARITY 3;
ALTER TABLE partslost_0 MATERIALIZE INDEX idx;
ALTER TABLE partslost_0 DROP INDEX idx;
then mutation created by MATERIALIZE INDEX may stuck, because index is dropped:
2022.06.30 06:37:21.425471 [ 1088437 ] {} <Error> MutateFromLogEntryTask: virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 36. DB::Exception: Unknown index: idx. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):
0. ./build_docker/../contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x25fc2ed2 in /usr/bin/clickhouse
1. ./build_docker/../src/Common/Exception.cpp:69: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xfe6af8f in /usr/bin/clickhouse
2. ./build_docker/../src/Interpreters/MutationsInterpreter.cpp:595: DB::MutationsInterpreter::prepare(bool) @ 0x23d4f296 in /usr/bin/clickhouse
3. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:617: DB::MutationsInterpreter::MutationsInterpreter(std::__1::shared_ptr<DB::IStorage>, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::MutationCommands, std::__1::shared_ptr<DB::Context const>, bool) @ 0x23d49cb9 in /usr/bin/clickhouse
4. ./build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:0: std::__1::__unique_if<DB::MutationsInterpreter>::__unique_single std::__1::make_unique<DB::MutationsInterpreter, std::__1::shared_ptr<DB::IStorage>&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const>&, DB::MutationCommands&, std::__1::shared_ptr<DB::Context>&, bool>(std::__1::shared_ptr<DB::IStorage>&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const>&, DB::MutationCommands&, std::__1::shared_ptr<DB::Context>&, bool&&) @ 0x24dd2e39 in /usr/bin/clickhouse
5. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:816: DB::MutateTask::prepare() @ 0x24dc81d0 in /usr/bin/clickhouse
6. ./build_docker/../src/Storages/MergeTree/MutateTask.cpp:1394: DB::MutateTask::execute() @ 0x24dc7221 in /usr/bin/clickhouse
7. ./build_docker/../src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x24e1058f in /usr/bin/clickhouse
8. ./build_docker/../src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x24e0e2d6 in /usr/bin/clickhouse
9. ./build_docker/../src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:149: DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::routine(std::__1::shared_ptr<DB::TaskRuntimeData>) @ 0x24b8997d in /usr/bin/clickhouse
10. ./build_docker/../src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:236: DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::threadFunction() @ 0x24b8aafb in /usr/bin/clickhouse
11. ./build_docker/../contrib/libcxx/include/__functional/function.h:1157: ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) @ 0xff46961 in /usr/bin/clickhouse
12. ./build_docker/../src/Common/ThreadPool.cpp:0: ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0xff4a060 in /usr/bin/clickhouse
13. ./build_docker/../contrib/libcxx/include/__functional/function.h:1157: ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xff45161 in /usr/bin/clickhouse
14. ./build_docker/../src/Common/ThreadPool.cpp:0: void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda0'()> >(void*) @ 0xff47d62 in /usr/bin/clickhouse
15. ? @ 0x7f1b19647609 in ?
16. __clone @ 0x7f1b1956c133 in ?
(version 22.7.1.1 (official build))
ClickHouse/src/Interpreters/MutationsInterpreter.cpp
Lines 585 to 595 in 5cf2c67