ClickHouse does not start if dns entry is invalid for external ClickHouse dictionary #36451
Description
Describe the unexpected behavior
Create a dictionary of source type ClickHouse using a remote instance. Remove the remote instance and its associated DNS address. If we restart the local instance with the dictionary, it will refuse to start with the following exception:
0. Poco::Net::HostNotFoundException::HostNotFoundException(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x186517ac in /usr/bin/clickhouse
1. Poco::Net::DNS::aierror(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x1862778f in /usr/bin/clickhouse
2. Poco::Net::SocketAddress::init(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned short) @ 0x1865d079 in /usr/bin/clickhouse
How to reproduce
Reproduced on version 22.3.1 and 22.4.1.2116
- Ensure you have 2 instances available, accessible over a DNS entry (using the host file if necessary)
- Designate one instance
localand oneremote - Create a table with the dictionary data on the remote instance. Example below with public data.
CREATE TABLE zips (
zip LowCardinality(String),
city LowCardinality(String)
) ENGINE = MergeTree() ORDER BY (zip, city);
INSERT INTO zips SELECT zip, city FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/zips/zips.tsv', 'TSV', 'zip String, city String');
- Create a dictionary on the local instance.
CREATE DICTIONARY zip_codes
(
zip String,
city String
)
PRIMARY KEY zip
SOURCE (ClickHouse(HOST 'remote_instance' PORT 9200 USER 'default' PASSWORD 'password' DB 'default' TABLE 'zips' SECURE 0 QUERY 'SELECT zip, city FROM default.zips'))
LAYOUT(FLAT())
LIFETIME(MIN 0 MAX 1000);
- Shutdown the remote instance and remove the DNS entry e.g. delete from hosts file.
- Restart the local instance. It will fail to restart with the following DNS entry.
Expected behavior
ClickHouse should start. ClickHouse does start if the DNS entry is valid but the remote instance is not started.
Error message and/or stacktrace
Example
3. DB::getInfoIfClickHouseDictionarySource(Poco::AutoPtr<Poco::Util::AbstractConfiguration>&, std::__1::shared_ptr<DB::Context const>) @ 0x14012de6 in /usr/bin/clickhouse
4. DB::DDLDependencyVisitor::visit(DB::ASTFunctionWithKeyValueArguments const&, DB::DDLDependencyVisitor::Data&) @ 0x14469b55 in /usr/bin/clickhouse
7. DB::InDepthNodeVisitor<DB::DDLDependencyVisitor, true, false, std::__1::shared_ptr<DB::IAST> const>::visit(std::__1::shared_ptr<DB::IAST> const&) @ 0x1446947a in /usr/bin/clickhouse
8. DB::InDepthNodeVisitor<DB::DDLDependencyVisitor, true, false, std::__1::shared_ptr<DB::IAST> const>::visit(std::__1::shared_ptr<DB::IAST> const&) @ 0x144694ce in /usr/bin/clickhouse
9. DB::InDepthNodeVisitor<DB::DDLDependencyVisitor, true, false, std::__1::shared_ptr<DB::IAST> const>::visit(std::__1::shared_ptr<DB::IAST> const&) @ 0x144694ce in /usr/bin/clickhouse
10. DB::getDependenciesSetFromCreateQuery(std::__1::shared_ptr<DB::Context const>, DB::QualifiedTableName const&, std::__1::shared_ptr<DB::IAST> const&) @ 0x14468eec in /usr/bin/clickhouse
11. ? @ 0x145129cd in /usr/bin/clickhouse
12. ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) @ 0xaf6546a in /usr/bin/clickhouse
13. ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda0'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda0'()&&...)::'lambda'()::operator()() @ 0xaf674a4 in /usr/bin/clickhouse
14. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xaf62837 in /usr/bin/clickhouse
15. ? @ 0xaf662fd in /usr/bin/clickhouse
16. ? @ 0x7f9f10f52609 in ?
17. clone @ 0x7f9f10e77163 in ?
(version 22.2.2.1)
2022.04.20 12:17:08.789285 [ 360788 ] {} <Error> Application: Host not found: single-node-clickhouse-blue-1.localdomain