Skip to content

extactAllGroups[Horizonta|Vertical] stuck and high memory usage with specific regexp expression #13383

@UnamedRus

Description

@UnamedRus

How to reproduce
Clickhouse version 20.5.2.7

SELECT extractAllGroupsHorizontal(' a=10e-2,b=c,c="d",d=2,f=1.3, g="1,3", s=, k=3', '\W(\w+)=("[\w.,-]*")|([\w.-]*)');
SELECT extractAllGroupsVertical(' a=10e-2,b=c,c="d",d=2,f=1.3, g="1,3", s=, k=3', '\W(\w+)=("[\w.,-]*")|([\w.-]*)');
SELECT extractAllGroupsVertical(' a=10e-2,b=c,c="d",d=2,f=1.3, g="1,3", s=, k=3', '\W(\w+)="[\w.,-]*"|[\w.-]*');

Expected behavior
Query return some result (even empty)

Error message and/or stacktrace

2020.08.05 15:25:42.644160 [ 3745 ] {96193b12-98c2-4d74-a600-ebcf597b770f} <Error> executeQuery: Code: 241, e.displayText() = DB::Exception: Memory limit (for query) exceeded: would use 11.00 GiB (attempt to allocate chunk of 4295000960 bytes), maximum: 9.31 GiB (version 20.5.2.7 (official build)) (from 127.0.0.1:43996) (in query: SELECT extractAllGroupsVertical(' a=10e-2,b=c,c="d",d=2,f=1.3, g="1,3", s=, k=3', '\W(\w+)=("[\w.,-]*")|([\w.-]*)');), Stack trace (when copying this message, always include the lines below):

0. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x10ed0da0 in /usr/lib/debug/usr/bin/clickhouse
1. /build/obj-x86_64-linux-gnu/../src/Common/Exception.cpp:37: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x95c923d in /usr/lib/debug/usr/bin/clickhouse
2. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/string:2134: MemoryTracker::alloc(long) (.cold) @ 0x95c8228 in /usr/lib/debug/usr/bin/clickhouse
3. /build/obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:134: MemoryTracker::alloc(long) @ 0x95c6724 in /usr/lib/debug/usr/bin/clickhouse
4. DB::FunctionExtractAllGroups<(anonymous namespace)::VerticalImpl>::executeImpl(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long, unsigned long) @ 0xbaa49ab in /usr/lib/debug/usr/bin/clickhouse
5. DB::ExecutableFunctionAdaptor::defaultImplementationForConstantArguments(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) @ 0xa1c5806 in /usr/lib/debug/usr/bin/clickhouse
6. DB::ExecutableFunctionAdaptor::executeWithoutLowCardinalityColumns(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) @ 0xa1c5a22 in /usr/lib/debug/usr/bin/clickhouse
7. DB::ExecutableFunctionAdaptor::execute(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) @ 0xa1c67a1 in /usr/lib/debug/usr/bin/clickhouse
8. /build/obj-x86_64-linux-gnu/../src/Interpreters/ExpressionActions.cpp:209: DB::ExpressionAction::prepare(DB::Block&, DB::Settings const&, std::__1::unordered_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&) @ 0xdd4d675 in /usr/lib/debug/usr/bin/clickhouse
9. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:1635: DB::ExpressionActions::addImpl(DB::ExpressionAction, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&) @ 0xdd4e235 in /usr/lib/debug/usr/bin/clickhouse
10. /build/obj-x86_64-linux-gnu/../src/Interpreters/ExpressionActions.cpp:577: DB::ExpressionActions::add(DB::ExpressionAction const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&) @ 0xdd4e89d in /usr/lib/debug/usr/bin/clickhouse
11. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:1549: DB::ScopeStack::addAction(DB::ExpressionAction const&) @ 0xddf31fd in /usr/lib/debug/usr/bin/clickhouse
12. /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:593: DB::ActionsMatcher::visit(DB::ASTFunction const&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0xddfb476 in /usr/lib/debug/usr/bin/clickhouse
13. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3826: DB::InDepthNodeVisitor<DB::ActionsMatcher, true, std::__1::shared_ptr<DB::IAST> const>::visit(std::__1::shared_ptr<DB::IAST> const&) @ 0xdde8beb in /usr/lib/debug/usr/bin/clickhouse
14. /build/obj-x86_64-linux-gnu/../src/Interpreters/InDepthNodeVisitor.h:45: DB::ExpressionAnalyzer::getRootActions(std::__1::shared_ptr<DB::IAST> const&, bool, std::__1::shared_ptr<DB::ExpressionActions>&, bool) (.constprop.0) @ 0xddda423 in /usr/lib/debug/usr/bin/clickhouse
15. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:656: DB::SelectQueryExpressionAnalyzer::appendSelect(DB::ExpressionActionsChain&, bool) @ 0xdddc1ed in /usr/lib/debug/usr/bin/clickhouse
16. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:662: DB::ExpressionAnalysisResult::ExpressionAnalysisResult(DB::SelectQueryExpressionAnalyzer&, bool, bool, bool, std::__1::shared_ptr<DB::FilterInfo> const&, DB::Block const&) @ 0xdde595d in /usr/lib/debug/usr/bin/clickhouse
17. /build/obj-x86_64-linux-gnu/../src/Interpreters/ExpressionAnalyzer.h:165: DB::InterpreterSelectQuery::getSampleBlockImpl() @ 0xdd94368 in /usr/lib/debug/usr/bin/clickhouse
18. /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:307: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::'lambda'(bool)::operator()(bool) const @ 0xdd9aa81 in /usr/lib/debug/usr/bin/clickhouse
19. /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:401: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xdda29a5 in /usr/lib/debug/usr/bin/clickhouse
20. /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:145: DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xdda4339 in /usr/lib/debug/usr/bin/clickhouse
21. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:1681: DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xdf050e7 in /usr/lib/debug/usr/bin/clickhouse
22. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:461: DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, DB::Context&, DB::QueryProcessingStage::Enum) @ 0xdd11d74 in /usr/lib/debug/usr/bin/clickhouse
23. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2587: DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) @ 0xe0748a4 in /usr/lib/debug/usr/bin/clickhouse
24. /build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:643: DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) @ 0xe07811a in /usr/lib/debug/usr/bin/clickhouse
25. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:251: DB::TCPHandler::runImpl() @ 0xe698946 in /usr/lib/debug/usr/bin/clickhouse
26. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1197: DB::TCPHandler::run() @ 0xe699660 in /usr/lib/debug/usr/bin/clickhouse
27. /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x10deebcb in /usr/lib/debug/usr/bin/clickhouse
28. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:856: Poco::Net::TCPServerDispatcher::run() @ 0x10def05b in /usr/lib/debug/usr/bin/clickhouse
29. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/Mutex_POSIX.h:59: Poco::PooledThread::run() @ 0x10f6db86 in /usr/lib/debug/usr/bin/clickhouse
30. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/AutoPtr.h:223: Poco::ThreadImpl::runnableEntry(void*) @ 0x10f68f80 in /usr/lib/debug/usr/bin/clickhouse
31. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so


Received exception from server (version 20.5.2):
Code: 241. DB::Exception: Received from localhost:9000. DB::Exception: Memory limit (for query) exceeded: would use 11.00 GiB (attempt to allocate chunk of 4295000960 bytes), maximum: 9.31 GiB.
Query was cancelled.

0 rows in set. Elapsed: 92.109 sec.

Additional context
ExtractAll works fine.

SELECT extractAllGroupsVertical(' a=10e-2,b=c,c="d",d=2,f=1.3, g="1,3", s=, k=3', '\\W(\\w+)=("[\\w.,-]*"|[\\w.-]*)') AS pairs

┌─pairs────────────────────────────────────────────────────────────────────────────────────────┐
│ [['a','10e-2'],['b','c'],['c','"d"'],['d','2'],['f','1.3'],['g','"1,3"'],['s',''],['k','3']] │
└──────────────────────────────────────────────────────────────────────────────────────────────┘

Looks like problem happens when we have | operator not surrounded with brackets.

Metadata

Metadata

Labels

bugConfirmed user-visible misbehaviour in official release

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions