ReplicatedMergeTree should cleanup table directories on failed conn to ZooKeeper. #14517
Description
Describe the unexpected behaviour
- configure a bad zookeeper connection (
zookeepersection inconfig.xmlfile) - create ReplicatedMergeTree table in the way described below
- It will fail with
Connection refusedas it can't reach zookeeper. - Retry table creation and it will fail again.
The behavior seems like it writes the table files and does the zookeeper transaction and when the exception happens, the table creation is interrupted.
Current work around is to manually delete the files.
root@shard-3:# rm -rf /var/lib/clickhouse/data/shard/test/
How to reproduce
shard-3 :) CREATE TABLE shard.test ( id Int64, event_time DateTime ) Engine=ReplicatedMergeTree('/clickhouse/tables/shard3/test', 'replica_1') PARTITION BY toYYYYMMDD(event_time) ORDER BY id;
CREATE TABLE shard.test
(
`id` Int64,
`event_time` DateTime
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/shard3/test', 'replica_1')
PARTITION BY toYYYYMMDD(event_time)
ORDER BY id
Received exception from server (version 20.7.2):
Code: 999. DB::Exception: Received from localhost:9000. DB::Exception: All connection tries failed while connecting to ZooKeeper. nodes: 192.168.0.51:2181, 192.168.0.53:2181, 192.168.0.52:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 20.7.2.30 (official build)), 192.168.0.51:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 20.7.2.30 (official build)), 192.168.0.53:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 20.7.2.30 (official build)), 192.168.0.52:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 20.7.2.30 (official build)), 192.168.0.51:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 20.7.2.30 (official build)), 192.168.0.53:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 20.7.2.30 (official build)), 192.168.0.52:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 20.7.2.30 (official build)), 192.168.0.51:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 20.7.2.30 (official build)), 192.168.0.53:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 20.7.2.30 (official build)), 192.168.0.52:2181
(Connection loss).
0 rows in set. Elapsed: 0.008 sec.
shard-3 :) CREATE TABLE shard.test ( id Int64, event_time DateTime ) Engine=ReplicatedMergeTree('/clickhouse/tables/shard3/test', 'replica_1') PARTITION BY toYYYYMMDD(event_time) ORDER BY id;
CREATE TABLE shard.test
(
`id` Int64,
`event_time` DateTime
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/shard3/test', 'replica_1')
PARTITION BY toYYYYMMDD(event_time)
ORDER BY id
Received exception from server (version 20.7.2):
Code: 57. DB::Exception: Received from localhost:9000. DB::Exception: Directory for table data data/shard/test/ already exists.
Expected behavior
Don't create the directory if connection to zookeeper fails.