DB::Exception: Trying to get CSN for too old TID when killing clickhouse server process while big atomic insert #60406
Description
I create database and table:
CREATE TABLE IF NOT EXISTS test_table_A (timestamp DateTime,host String,response_time Int32,Path String,Time DateTime,Value Float64,Timestamp Int64,sign Int8)
ENGINE = MergeTree
ORDER BY (host, timestamp)I make a big insert into the table and kill clickhouse server process in parallel:
BEGIN TRANSACTION;INSERT INTO test_table_A (timestamp, host, response_time, sign) SELECT now() + number/10, toString(number), number, 1 FROM numbers(50000000000); COMMIT;kill -SIGKILL {pid}
I get:
Error on processing query: Code: 32. DB::Exception: Attempt to read after eof: while receiving packet from localhost:9000. (ATTEMPT_TO_READ_AFTER_EOF) (version 23.3.19.32 (official build))
I restart server and get:
Code: 210. DB::NetException: Connection refused (localhost:9000). (NETWORK_ERROR)
In server error log I get
2024.02.26 12:01:18.988795 [ 1880 ] {} <Error> Application: Code: 49. DB::Exception: Trying to get CSN for too old TID (40, 35, 82e71eab-e7ff-4faa-8bb5-16b7bf3e4bfa), current tail_ptr is 41, probably it's a bug: Cannot attach table `test_database5c2304b9_d496_11ee_901a_cf40acb2a173`.`table_A5c2304b9_d496_11ee_901a_cf40acb2a173` from metadata file /var/lib/clickhouse/store/a6a/a6a01912-8d18-4880-9e7a-6803f2f31847/table_A5c2304b9_d496_11ee_901a_cf40acb2a173.sql from query ATTACH TABLE test_database5c2304b9_d496_11ee_901a_cf40acb2a173.table_A5c2304b9_d496_11ee_901a_cf40acb2a173 UUID '30b93009-9076-49d4-b71a-52eff8d377bb' (`timestamp` DateTime, `host` String, `response_time` Int32, `Path` String, `Time` DateTime, `Value` Float64, `Timestamp` Int64, `sign` Int8) ENGINE = MergeTree ORDER BY (host, timestamp) SETTINGS index_granularity = 8192. (LOGICAL_ERROR), Stack trace (when copying this message, always include the lines below):
0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0xbb942a4 in /usr/bin/clickhouse
1. ? @ 0x1078c0c8 in /usr/bin/clickhouse
2. DB::TransactionLog::assertTIDIsNotOutdated(DB::TransactionID const&, std::atomic<unsigned long> const*) @ 0x10788c4c in /usr/bin/clickhouse
3. DB::MergeTreeData::loadDataPart(DB::MergeTreePartInfo const&, String const&, std::shared_ptr<DB::IDisk> const&, DB::MergeTreeDataPartState, std::mutex&) @ 0x11002d7c in /usr/bin/clickhouse
4. ? @ 0x11075b80 in /usr/bin/clickhouse
5. ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0xbc4dbd0 in /usr/bin/clickhouse
6. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, long, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0xbc5051c in /usr/bin/clickhouse
7. ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0xbc49fa4 in /usr/bin/clickhouse
8. ? @ 0xbc4f448 in /usr/bin/clickhouse
9. start_thread @ 0x7624 in /usr/lib/aarch64-linux-gnu/libpthread-2.31.so
10. ? @ 0xd149c in /usr/lib/aarch64-linux-gnu/libc-2.31.so
(version 23.3.19.32 (official build))
2024.02.26 12:01:18.989006 [ 1880 ] {} <Error> Application: DB::Exception: Trying to get CSN for too old TID (40, 35, 82e71eab-e7ff-4faa-8bb5-16b7bf3e4bfa), current tail_ptr is 41, probably it's a bug: Cannot attach table `test_database5c2304b9_d496_11ee_901a_cf40acb2a173`.`table_A5c2304b9_d496_11ee_901a_cf40acb2a173` from metadata file /var/lib/clickhouse/store/a6a/a6a01912-8d18-4880-9e7a-6803f2f31847/table_A5c2304b9_d496_11ee_901a_cf40acb2a173.sql from query ATTACH TABLE test_database5c2304b9_d496_11ee_901a_cf40acb2a173.table_A5c2304b9_d496_11ee_901a_cf40acb2a173 UUID '30b93009-9076-49d4-b71a-52eff8d377bb' (`timestamp` DateTime, `host` String, `response_time` Int32, `Path` String, `Time` DateTime, `Value` Float64, `Timestamp` Int64, `sign` Int8) ENGINE = MergeTree ORDER BY (host, timestamp) SETTINGS index_granularity = 8192
Caught on versions 23.3. and 23.8 on arm machine.