jdbc-bridge not running or timed out #8723
Description
jdbc-bridge sometimes runs into below errors. They can be reproduced by issuing same query repeatedly in multiple threads. Perhaps PooledReadWriteBufferFromHTTP should be used along with retry like in #4837?
- not running
2020.01.18 23:30:14.811797 [ 25 ] {47d43ffc-64c4-45c5-ad35-047ba3c665c8} <Error> HTTPHandler: Code: 410, e.displayText() = DB::Exception: jdbc-bridge is not running. Please, start it manually, Stack trace:
0. /usr/bin/clickhouse-server(StackTrace::StackTrace()+0x30) [0x7d37f50]
1. /usr/bin/clickhouse-server(DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)+0x25) [0x3bbc1e5]
2. /usr/bin/clickhouse-server(DB::XDBCBridgeHelper<DB::JDBCBridgeMixin>::startBridgeSync() const+0x2e2) [0x69e8f72]
3. /usr/bin/clickhouse-server(DB::XDBCBridgeHelper<DB::JDBCBridgeMixin>::getIdentifierQuotingStyle()+0x40) [0x69e8680]
4. /usr/bin/clickhouse-server(DB::StorageXDBC::getReadPOSTDataCallback(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, DB::SelectQueryInfo const&, DB::Context const&, DB::QueryProcessingStage::Enum&, unsigned long) const+0x47) [0x6eac197]
5. /usr/bin/clickhouse-server(DB::IStorageURLBase::read(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, DB::SelectQueryInfo const&, DB::Context const&, DB::QueryProcessingStage::Enum, unsigned long, unsigned int)+0xf27) [0x6eaaf07]
6. /usr/bin/clickhouse-server(DB::StorageXDBC::read(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, DB::SelectQueryInfo const&, DB::Context const&, DB::QueryProcessingStage::Enum, unsigned long, unsigned int)+0x6d) [0x6eab75d]
7. /usr/bin/clickhouse-server(DB::InterpreterSelectQuery::executeFetchColumns(DB::QueryProcessingStage::Enum, DB::InterpreterSelectQuery::Pipeline&, std::shared_ptr<DB::PrewhereInfo> const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)+0x1fda) [0x6c332fa]
8. /usr/bin/clickhouse-server(DB::InterpreterSelectQuery::executeImpl(DB::InterpreterSelectQuery::Pipeline&, std::shared_ptr<DB::IBlockInputStream> const&, bool)+0x794) [0x6c36234]
9. /usr/bin/clickhouse-server(DB::InterpreterSelectQuery::executeWithMultipleStreams()+0x51) [0x6c38b81]
10. /usr/bin/clickhouse-server(DB::InterpreterSelectWithUnionQuery::executeWithMultipleStreams()+0x63) [0x6c479d3]
11. /usr/bin/clickhouse-server(DB::InterpreterSelectWithUnionQuery::execute()+0x3f) [0x6c47f5f]
12. /usr/bin/clickhouse-server() [0x6d4a44f]
13. /usr/bin/clickhouse-server(DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, DB::Context&, std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>, std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>)+0x112) [0x6d4b6a2]
14. /usr/bin/clickhouse-server(DB::HTTPHandler::processQuery(Poco::Net::HTTPServerRequest&, HTMLForm&, Poco::Net::HTTPServerResponse&, DB::HTTPHandler::Output&)+0x197e) [0x3bd0fee]
15. /usr/bin/clickhouse-server(DB::HTTPHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&)+0x52b) [0x3bd340b]
16. /usr/bin/clickhouse-server(Poco::Net::HTTPServerConnection::run()+0x2a9) [0x7718009]
17. /usr/bin/clickhouse-server(Poco::Net::TCPServerConnection::start()+0x10) [0x7712de0]
18. /usr/bin/clickhouse-server(Poco::Net::TCPServerDispatcher::run()+0xed) [0x77134fd]
19. /usr/bin/clickhouse-server(Poco::PooledThread::run()+0x81) [0x7e36ca1]
20. /usr/bin/clickhouse-server(Poco::ThreadImpl::runnableEntry(void*)+0x3c) [0x7e34a4c]
21. /usr/bin/clickhouse-server() [0xb80a200]
22. /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f5cd479b6db]
23. /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f5cd3f2288f]
(version 19.11.8.46 (official build))
- timeout
2020.01.18 23:30:15.687281 [ 34 ] {3cab6ea4-3752-4a89-a527-230bc0121c0e} <Error> executeQuery: Poco::Exception. Code: 1000, e.code() = 0, e.displayText() = Timeout: connect timed out: 192.168.6.110:8080 (version 19.11.8.46 (official build)) (from 172.19.0.1:40500) (in query: select * from jdbc('ch', '', 'select * from system.processes') FORMAT TabSeparatedWithNamesAndTypes;)
Update:
To reproduce the issues, please use https://github.com/zhicwu/clickhouse-jdbc-bridge-test and follow instructions there.