Skip to content

correct index analysis of WITH aliases#18896

Merged
KochetovNicolai merged 1 commit intoClickHouse:masterfrom
amosbird:keyconditionalias
Jan 11, 2021
Merged

correct index analysis of WITH aliases#18896
KochetovNicolai merged 1 commit intoClickHouse:masterfrom
amosbird:keyconditionalias

Conversation

@amosbird
Copy link
Copy Markdown
Collaborator

@amosbird amosbird commented Jan 10, 2021 •
edited by KochetovNicolai
Loading

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Aliases declared in WITH statement are properly used in index analysis. Queries like WITH column AS alias SELECT ... WHERE alias = ... may use index now.

Allow WITH aliases being used in index analysis. cc @KochetovNicolai

Detailed description / Documentation draft:

.

@robot-clickhouse robot-clickhouse added the pr-improvement Pull request with some product improvements label Jan 10, 2021
KochetovNicolai
KochetovNicolai approved these changes Jan 11, 2021
View reviewed changes
Copy link
Copy Markdown
Member

@KochetovNicolai KochetovNicolai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@KochetovNicolai
Copy link
Copy Markdown
Member

KochetovNicolai commented Jan 11, 2021

Stress test:

2021.01.10 14:49:46.071781 [ 104615 ] {} <Fatal> BaseDaemon: ########################################
2021.01.10 14:49:46.071904 [ 104615 ] {} <Fatal> BaseDaemon: (version 20.13.1.5636, build id: E83FC1EB7D0C7632B82F4AF5561BFBACF121D6E0) (from thread 97249) (query_id: 8acd2c82-000f-43d0-82c6-4e2e3f6e9ce4) Received signal Aborted (6)
2021.01.10 14:49:46.071973 [ 104615 ] {} <Fatal> BaseDaemon: 
2021.01.10 14:49:46.072109 [ 104615 ] {} <Fatal> BaseDaemon: Stack trace: 0x7f24a904518b 0x7f24a9024859 0xed1bd14 0x1dbcfdd8 0x1dbcfd69 0xeaafb8b 0x19ce2be2 0x19cdcf48 0x1b7ff842 0x1b7fee79 0x1ba90752 0x1b84d019 0x1b84cf07 0x1b84b16f 0x1b849a5a 0x1b84952b 0x1b85cbea 0x1b85cb43 0x1b85ca02 0xeb14c4d 0xeb188a6 0x7f24a9200609 0x7f24a9121293
2021.01.10 14:49:46.072269 [ 104615 ] {} <Fatal> BaseDaemon: 3. gsignal @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.01.10 14:49:46.072353 [ 104615 ] {} <Fatal> BaseDaemon: 4. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.01.10 14:49:46.146476 [ 104615 ] {} <Fatal> BaseDaemon: 5. ./obj-x86_64-linux-gnu/../base/daemon/BaseDaemon.cpp:428: terminate_handler() @ 0xed1bd14 in /usr/bin/clickhouse
2021.01.10 14:49:46.206654 [ 104615 ] {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_handlers.cpp:61: std::__terminate(void (*)()) @ 0x1dbcfdd8 in /usr/bin/clickhouse
2021.01.10 14:49:46.214814 [ 104615 ] {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/include/atomic_support.h:78: std::terminate() @ 0x1dbcfd69 in /usr/bin/clickhouse
2021.01.10 14:49:46.221921 [ 104615 ] {} <Fatal> BaseDaemon: 8. ? @ 0xeaafb8b in /usr/bin/clickhouse
2021.01.10 14:49:46.230461 [ 104615 ] {} <Fatal> BaseDaemon: 9. ? @ 0x19ce2be2 in /usr/bin/clickhouse
2021.01.10 14:49:46.234235 [ 104615 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3832: DB::TablesBlockSource::generate() @ 0x19cdcf48 in /usr/bin/clickhouse
2021.01.10 14:49:46.240513 [ 104615 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Processors/Chunk.h:89: DB::ISource::tryGenerate() @ 0x1b7ff842 in /usr/bin/clickhouse
2021.01.10 14:49:46.246802 [ 104615 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:294: DB::ISource::work() @ 0x1b7fee79 in /usr/bin/clickhouse
2021.01.10 14:49:46.253343 [ 104615 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../src/Processors/Sources/SourceWithProgress.cpp:38: DB::SourceWithProgress::work() @ 0x1ba90752 in /usr/bin/clickhouse
2021.01.10 14:49:46.260195 [ 104615 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:87: DB::executeJob(DB::IProcessor*) @ 0x1b84d019 in /usr/bin/clickhouse
2021.01.10 14:49:46.270264 [ 104615 ] {} <Fatal> BaseDaemon: 15. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:99: decltype(std::__1::forward<DB::PipelineExecutor::addJob(DB::ExecutingGraph::Node*)::$_0&>(fp)()) std::__1::__invoke<DB::PipelineExecutor::addJob(DB::ExecutingGraph::Node*)::$_0&>(DB::PipelineExecutor::addJob(DB::ExecutingGraph::Node*)::$_0&) @ 0x1b84cf07 in /usr/bin/clickhouse
2021.01.10 14:49:46.284413 [ 104615 ] {} <Fatal> BaseDaemon: 16. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:0: DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic<bool>*) @ 0x1b84b16f in /usr/bin/clickhouse
2021.01.10 14:49:46.289776 [ 104615 ] {} <Fatal> BaseDaemon: 17. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::executeImpl(unsigned long) @ 0x1b849a5a in /usr/bin/clickhouse
2021.01.10 14:49:46.295257 [ 104615 ] {} <Fatal> BaseDaemon: 18. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:398: DB::PipelineExecutor::execute(unsigned long) @ 0x1b84952b in /usr/bin/clickhouse
2021.01.10 14:49:46.300983 [ 104615 ] {} <Fatal> BaseDaemon: 19. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:74: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::__1::shared_ptr<DB::ThreadGroupStatus>, unsigned long) @ 0x1b85cbea in /usr/bin/clickhouse
2021.01.10 14:49:46.306696 [ 104615 ] {} <Fatal> BaseDaemon: 20. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: decltype(std::__1::forward<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(fp)()) std::__1::__invoke_constexpr<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x1b85cb43 in /usr/bin/clickhouse
2021.01.10 14:49:46.313083 [ 104615 ] {} <Fatal> BaseDaemon: 21. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:0: decltype(std::__1::forward<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(fp)()) std::__1::__invoke<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'()&>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&) @ 0x1b85ca02 in /usr/bin/clickhouse
2021.01.10 14:49:46.313306 [ 104615 ] {} <Fatal> BaseDaemon: 22. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2446: ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xeb14c4d in /usr/bin/clickhouse
2021.01.10 14:49:46.313652 [ 104615 ] {} <Fatal> BaseDaemon: 23. ./obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:0: void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) @ 0xeb188a6 in /usr/bin/clickhouse
2021.01.10 14:49:46.313761 [ 104615 ] {} <Fatal> BaseDaemon: 24. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
2021.01.10 14:49:46.313835 [ 104615 ] {} <Fatal> BaseDaemon: 25. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.01.10 14:49:46.727765 [ 104615 ] {} <Fatal> BaseDaemon: Calculated checksum of the binary: D922194BF6CD94BBE6FE0146E7D8C238. There is no information about the reference checksum.

@KochetovNicolai KochetovNicolai merged commit ee094ed into ClickHouse:master Jan 11, 2021
@KochetovNicolai
Copy link
Copy Markdown
Member

KochetovNicolai commented Jan 11, 2021 

addr2line -afiCe clickhouse 0x7f24a904518b 0x7f24a9024859 0xed1bd14 0x1dbcfdd8 0x1dbcfd69 0xeaafb8b 0x19ce2be2 0x19cdcf48 0x1b7ff842 0x1b7fee79 0x1ba90752 0x1b84d019 0x1b84cf07 0x1b84b16f 0x1b849a5a 0x1b84952b 0x1b85cbea 0x1b85cb43 0x1b85ca02 0xeb14c4d 0xeb188a6 0x7f24a9200609 0x7f24a9121293
0x00007f24a904518b
??
??:0
0x00007f24a9024859
??
??:0
0x000000000ed1bd14
terminate_handler()
./obj-x86_64-linux-gnu/../base/daemon/BaseDaemon.cpp:428
0x000000001dbcfdd8
std::__terminate(void (*)())
./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_handlers.cpp:61
0x000000001dbcfd69
void (*std::__1::(anonymous namespace)::__libcpp_atomic_load<void (*)()>(void (* const*)(), int))()
./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/include/atomic_support.h:78
std::get_terminate()
./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_handlers.cpp:49
std::terminate()
./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_handlers.cpp:92
0x000000000eaafb8b
__clang_call_terminate
main.cpp:?
0x0000000019ce2be2
DB::IDatabase::tryGetCreateTableQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context const&) const
??:?
0x0000000019cdcf48
DB::TablesBlockSource::generate()
./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3832
DB::TablesBlockSource::generate()
./obj-x86_64-linux-gnu/../src/Storages/System/StorageSystemTables.cpp:358
0x000000001b7ff842
DB::ISource::tryGenerate()
./obj-x86_64-linux-gnu/../src/Processors/Chunk.h:89
??
./obj-x86_64-linux-gnu/../src/Processors/Chunk.h:91
??
./obj-x86_64-linux-gnu/../src/Processors/Chunk.h:92
DB::ISource::tryGenerate()
./obj-x86_64-linux-gnu/../src/Processors/ISource.cpp:80
0x000000001b7fee79
DB::ISource::work()
./obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:294
??
./obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:937
DB::ISource::work()
./obj-x86_64-linux-gnu/../src/Processors/ISource.cpp:53
0x000000001ba90752
DB::SourceWithProgress::work()
./obj-x86_64-linux-gnu/../src/Processors/Sources/SourceWithProgress.cpp:38
0x000000001b84d019
DB::executeJob(DB::IProcessor*)
./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:87
0x000000001b84cf07
DB::PipelineExecutor::addJob(DB::ExecutingGraph::Node*)::$_0::operator()() const
./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:99
_ZNSt3__18__invokeIRZN2DB16PipelineExecutor6addJobEPNS1_14ExecutingGraph4NodeEE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS8_DpOS9_
./obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3519
0x000000001b84b16f
DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic<bool>*)
??:?
??
./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2473
DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic<bool>*)
./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:580
0x000000001b849a5a
DB::PipelineExecutor::executeImpl(unsigned long)
??:?
DB::PipelineExecutor::executeImpl(unsigned long)
./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:807
0x000000001b84952b
DB::PipelineExecutor::execute(unsigned long)
./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:398
0x000000001b85cbea
DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::__1::shared_ptr<DB::ThreadGroupStatus>, unsigned long)::$_2::operator()() const
./obj-x86_64-linux-gnu/../src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:74
??
./obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:97
ext::basic_scope_guard<DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::__1::shared_ptr<DB::ThreadGroupStatus>, unsigned long)::$_2>::~basic_scope_guard()
./obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:47
DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::__1::shared_ptr<DB::ThreadGroupStatus>, unsigned long)
./obj-x86_64-linux-gnu/../src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:80
0x000000001b85cb43
_ZNSt3__118__invoke_constexprIRZN2DB28PullingAsyncPipelineExecutor4pullERNS1_5ChunkEmE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS7_DpOS8_
PullingAsyncPipelineExecutor.cpp:?
_ZNSt3__118__invoke_constexprIRZN2DB28PullingAsyncPipelineExecutor4pullERNS1_5ChunkEmE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS7_DpOS8_
./obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3525
0x000000001b85ca02
_ZNSt3__18__invokeIRZN20ThreadFromGlobalPoolC1IZN2DB28PullingAsyncPipelineExecutor4pullERNS3_5ChunkEmE3$_0JEEEOT_DpOT0_EUlvE_JEEEDTclclsr3std3__1E7forwardIS8_Efp_Espclsr3std3__1E7forwardISA_Efp0_EEES9_SC_
PullingAsyncPipelineExecutor.cpp:?
_ZNSt3__18__invokeIRZN20ThreadFromGlobalPoolC1IZN2DB28PullingAsyncPipelineExecutor4pullERNS3_5ChunkEmE3$_0JEEEOT_DpOT0_EUlvE_JEEEDTclclsr3std3__1E7forwardIS8_Efp_Espclsr3std3__1E7forwardISA_Efp0_EEES9_SC_
./obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3519
0x000000000eb14c4d
ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>)
./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2446
ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>)
./obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:246
0x000000000eb188a6
void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::{lambda()#3}> >(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::{lambda()#3}>)
ThreadPool.cpp:?
0x00007f24a9200609
??
??:0
0x00007f24a9121293
??
??:0

This was referenced Feb 1, 2021
Closed
Closed
@amosbird amosbird mentioned this pull request Mar 16, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@KochetovNicolai KochetovNicolai KochetovNicolai approved these changes

Assignees

No one assigned

Labels

pr-improvement Pull request with some product improvements

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@amosbird @KochetovNicolai @robot-clickhouse