2025-12-23 04:55:06 Reason: return code:  242
2025-12-23 04:55:06 [ae234aafe29b] 2025.12.23 00:55:05.532041 [ 32482 ] {520f7190-6019-4366-a3ed-3e8c20fed818} <Error> executeQuery: Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01459_manual_write_to_replicas_quorum_detach_attach_test_oxmeq0tg/r/replicas/r6). (TABLE_IS_READ_ONLY) (version 26.1.1.101) (from [::1]:43074) (comment: 01459_manual_write_to_replicas_quorum_detach_attach.sh-test_oxmeq0tg) (query 1, line 2) (in query: SYSTEM SYNC REPLICA r6;), Stack trace (when copying this message, always include the lines below):
2025-12-23 04:55:06 
2025-12-23 04:55:06 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:0: Poco::Exception::Exception(String const&, int) @ 0x000000003f336be0
2025-12-23 04:55:06 1. ./ci/tmp/build/./src/Common/Exception.cpp:136: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001dd5a83c
2025-12-23 04:55:06 2. DB::Exception::Exception(String&&, int, String, bool) @ 0x000000000f1c4b9c
2025-12-23 04:55:06 3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000f1c3e88
2025-12-23 04:55:06 4. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x000000000f1e2e60
2025-12-23 04:55:06 5. ./ci/tmp/build/./src/Storages/StorageReplicatedMergeTree.cpp:6063: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x0000000031b52e74
2025-12-23 04:55:06 6. ./ci/tmp/build/./src/Storages/StorageReplicatedMergeTree.cpp:342: DB::StorageReplicatedMergeTree::waitForProcessingQueue(unsigned long, DB::SyncReplicaMode, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>) @ 0x0000000031c9c790
2025-12-23 04:55:06 7. ./ci/tmp/build/./src/Interpreters/InterpreterSystemQuery.cpp:1667: DB::InterpreterSystemQuery::trySyncReplica(std::shared_ptr<DB::IStorage>, DB::SyncReplicaMode, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::Context const>) @ 0x000000002bcaf0bc
2025-12-23 04:55:06 8. ./ci/tmp/build/./src/Interpreters/InterpreterSystemQuery.cpp:1701: DB::InterpreterSystemQuery::syncReplica(DB::ASTSystemQuery&) @ 0x000000002bc93280
2025-12-23 04:55:06 9. ./ci/tmp/build/./src/Interpreters/InterpreterSystemQuery.cpp:825: DB::InterpreterSystemQuery::execute() @ 0x000000002bc7fa9c
2025-12-23 04:55:06 10. ./ci/tmp/build/./src/Interpreters/executeQuery.cpp:1748: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, std::unique_ptr<DB::ReadBuffer, std::default_delete<DB::ReadBuffer>>&, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::ImplicitTransactionControlExecutor>, std::function<void ()>, DB::QueryResultDetails&) @ 0x000000002bc1bb80
2025-12-23 04:55:06 11. ./ci/tmp/build/./src/Interpreters/executeQuery.cpp:1972: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002bc138c8
2025-12-23 04:55:06 12. ./ci/tmp/build/./src/Server/TCPHandler.cpp:774: DB::TCPHandler::runImpl() @ 0x00000000334e8fb8
2025-12-23 04:55:06 13. ./ci/tmp/build/./src/Server/TCPHandler.cpp:2862: DB::TCPHandler::run() @ 0x00000000335203a4
2025-12-23 04:55:06 14. ./ci/tmp/build/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x000000003f4f7238
2025-12-23 04:55:06 15. ./ci/tmp/build/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000003f4f7e68
2025-12-23 04:55:06 16. ./ci/tmp/build/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x000000003f40bd78
2025-12-23 04:55:06 17. ./base/poco/Foundation/src/Thread_POSIX.cpp:341: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003f406440
2025-12-23 04:55:06 18. asan_thread_start(void*) @ 0x000000000f172a28
2025-12-23 04:55:06 19. ? @ 0x0000000000080398
2025-12-23 04:55:06 20. ? @ 0x00000000000e9e9c
2025-12-23 04:55:06 
2025-12-23 04:55:06 Received exception from server (version 26.1.1):
2025-12-23 04:55:06 Code: 242. DB::Exception: Received from localhost:9000. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01459_manual_write_to_replicas_quorum_detach_attach_test_oxmeq0tg/r/replicas/r6). (TABLE_IS_READ_ONLY)
2025-12-23 04:55:06 (query: SYSTEM SYNC REPLICA r6;)
2025-12-23 04:55:06 
2025-12-23 04:55:06 /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/queries/0_stateless/test_oxmeq0tg/01459_manual_write_to_replicas_quorum_detach_attach.sh.debuglog:
2025-12-23 04:55:06 + [2025-12-23 04:53:31] [:11] NUM_REPLICAS=6
2025-12-23 04:55:06 ++ [2025-12-23 04:53:31] [:13] seq 1 6
2025-12-23 04:55:06 + [2025-12-23 04:53:31] [:13] for i in $(seq 1 $NUM_REPLICAS)
2025-12-23 04:55:06 + [2025-12-23 04:53:31] [:14] /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/clickhouse-client --send_logs_level=warning --database=test_oxmeq0tg --log_comment 01459_manual_write_to_replicas_quorum_detach_attach.sh-test_oxmeq0tg --max_insert_threads 12 --group_by_two_level_threshold 78309 --group_by_two_level_threshold_bytes 1 --distributed_aggregation_memory_efficient 1 --fsync_metadata 1 --output_format_parallel_formatting 1 --input_format_parallel_parsing 1 --min_chunk_bytes_for_parallel_parsing 11796308 --max_read_buffer_size 1029829 --prefer_localhost_replica 0 --max_block_size 72595 --max_joined_block_size_rows 77816 --joined_block_split_single_row 1 --join_output_by_rowlist_perkey_rows_threshold 384 --max_threads 2 --optimize_append_index 0 --use_hedged_requests 1 --optimize_if_chain_to_multiif 0 --optimize_if_transform_strings_to_enum 1 --optimize_read_in_order 0 --optimize_or_like_chain 0 --optimize_substitute_columns 1 --enable_multiple_prewhere_read_steps 0 --read_in_order_two_level_merge_threshold 80 --optimize_aggregation_in_order 1 --aggregation_in_order_max_block_bytes 35998395 --use_uncompressed_cache 0 --min_bytes_to_use_direct_io 9505121890 --min_bytes_to_use_mmap_io 8693970918 --local_filesystem_read_method mmap --remote_filesystem_read_method threadpool --local_filesystem_read_prefetch 1 --filesystem_cache_segments_batch_size 0 --read_from_filesystem_cache_if_exists_otherwise_bypass_cache 0 --throw_on_error_from_cache_on_write_operations 0 --remote_filesystem_read_prefetch 1 --allow_prefetched_read_pool_for_remote_filesystem 1 --filesystem_prefetch_max_memory_usage 128Mi --filesystem_prefetches_limit 10 --filesystem_prefetch_min_bytes_for_single_read_task 1Mi --filesystem_prefetch_step_marks 50 --filesystem_prefetch_step_bytes 100Mi --compile_expressions 1 --compile_aggregate_expressions 0 --compile_sort_description 1 --merge_tree_coarse_index_granularity 21 --optimize_distinct_in_order 1 --max_bytes_before_remerge_sort 1179420602 --min_compress_block_size 2128969 --max_compress_block_size 557103 --merge_tree_compact_parts_min_granules_to_multibuffer_read 79 --optimize_sorting_by_input_stream_properties 1 --http_response_buffer_size 9830613 --http_wait_end_of_query False --enable_memory_bound_merging_of_aggregation_results 1 --min_count_to_compile_expression 0 --min_count_to_compile_aggregate_expression 3 --min_count_to_compile_sort_description 0 --session_timezone America/Mazatlan --use_page_cache_for_disks_without_file_cache True --page_cache_inject_eviction False --merge_tree_read_split_ranges_into_intersecting_and_non_intersecting_injection_probability 0.06 --prefer_external_sort_block_bytes 100000000 --cross_join_min_rows_to_compress 1 --cross_join_min_bytes_to_compress 100000000 --min_external_table_block_size_bytes 0 --max_parsing_threads 0 --optimize_functions_to_subcolumns 1 --parallel_replicas_local_plan 1 --query_plan_join_swap_table false --enable_vertical_final 1 --optimize_extract_common_expressions 0 --use_async_executor_for_materialized_views 1 --use_query_condition_cache 0 --secondary_indices_enable_bulk_filtering 0 --use_skip_indexes_if_final 1 --use_skip_indexes_on_data_read 1 --optimize_rewrite_like_perfect_affix 1 --input_format_parquet_use_native_reader_v3 1 --enable_lazy_columns_replication 0 --allow_special_serialization_kinds_in_output_formats 1 --max_bytes_before_external_sort 9525137013 --max_bytes_before_external_group_by 3922677686 --max_bytes_ratio_before_external_sort 0 --max_bytes_ratio_before_external_group_by 0 --use_skip_indexes_if_final_exact_mode 1 --allow_merge_tree_settings --ratio_of_defaults_for_sparse_serialization 0.045773080549790346 --prefer_fetch_merged_part_size_threshold 10737418240 --vertical_merge_algorithm_min_rows_to_activate 1 --vertical_merge_algorithm_min_columns_to_activate 1 --allow_vertical_merges_from_compact_to_wide_parts 1 --min_merge_bytes_to_use_direct_io 3282821666 --index_granularity_bytes 2429783 --merge_max_block_size 17378 --index_granularity 5261 --min_bytes_for_wide_part 0 --marks_compress_block_size 51841 --primary_key_compress_block_size 36337 --replace_long_file_name_to_hash 1 --max_file_name_length 128 --min_bytes_for_full_part_storage 0 --compact_parts_max_bytes_to_buffer 36889637 --compact_parts_max_granules_to_buffer 190 --compact_parts_merge_max_bytes_to_prefetch_part 15841459 --cache_populated_by_fetch 0 --concurrent_part_removal_threshold 91 --old_parts_lifetime 129 --prewarm_mark_cache 1 --use_const_adaptive_granularity 0 --enable_index_granularity_compression 1 --enable_block_number_column 1 --enable_block_offset_column 1 --use_primary_key_cache 1 --prewarm_primary_key_cache 0 --object_serialization_version v3 --object_shared_data_serialization_version advanced --object_shared_data_serialization_version_for_zero_level_parts map_with_buckets --object_shared_data_buckets_for_compact_part 21 --object_shared_data_buckets_for_wide_part 27 --dynamic_serialization_version v3 --auto_statistics_types tdigest,minmax --serialization_info_version with_types --string_serialization_version single_stream --nullable_serialization_version basic --enable_shared_storage_snapshot_in_query 0 --allow_repeated_settings --client_logs_file=/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/queries/0_stateless/01459_manual_write_to_replicas_quorum_detach_attach.stderr-fatal -q '
2025-12-23 04:55:06         DROP TABLE IF EXISTS r1 SYNC;
2025-12-23 04:55:06         CREATE TABLE r1 (x UInt64) ENGINE = ReplicatedMergeTree('\''/clickhouse/tables/01459_manual_write_to_replicas_quorum_detach_attach_test_oxmeq0tg/r'\'', '\''r1'\'') ORDER BY x;
2025-12-23 04:55:06     '
2025-12-23 04:55:06 + [2025-12-23 04:53:32] [:13] for i in $(seq 1 $NUM_REPLICAS)
2025-12-23 04:55:06 + [2025-12-23 04:53:32] [:14] /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/clickhouse-client --send_logs_level=warning --database=test_oxmeq0tg --log_comment 01459_manual_write_to_replicas_quorum_detach_attach.sh-test_oxmeq0tg --max_insert_threads 12 --group_by_two_level_threshold 78309 --group_by_two_level_threshold_bytes 1 --distributed_aggregation_memory_efficient 1 --fsync_metadata 1 --output_format_parallel_formatting 1 --input_format_parallel_parsing 1 --min_chunk_bytes_for_parallel_parsing 11796308 --max_read_buffer_size 1029829 --prefer_localhost_replica 0 --max_block_size 72595 --max_joined_block_size_rows 77816 --joined_block_split_single_row 1 --join_output_by_rowlist_perkey_rows_threshold 384 --max_threads 2 --optimize_append_index 0 --use_hedged_requests 1 --optimize_if_chain_to_multiif 0 --optimize_if_transform_strings_to_enum 1 --optimize_read_in_order 0 --optimize_or_like_chain 0 --optimize_substitute_columns 1 --enable_multiple_prewhere_read_steps 0 --read_in_order_two_level_merge_threshold 80 --optimize_aggregation_in_order 1 --aggregation_in_order_max_block_bytes 35998395 --use_uncompressed_cache 0 --min_bytes_to_use_direct_io 9505121890 --min_bytes_to_use_mmap_io 8693970918 --local_filesystem_read_method mmap --remote_filesystem_read_method threadpool --local_filesystem_read_prefetch 1 --filesystem_cache_segments_batch_size 0 --read_from_filesystem_cache_if_exists_otherwise_bypass_cache 0 --throw_on_error_from_cache_on_write_operations 0 --remote_filesystem_read_prefetch 1 --allow_prefetched_read_pool_for_remote_filesystem 1 --filesystem_prefetch_max_memory_usage 128Mi --filesystem_prefetches_limit 10 --filesystem_prefetch_min_bytes_for_single_read_task 1Mi --filesystem_prefetch_step_marks 50 --filesystem_prefetch_step_bytes 100Mi --compile_expressions 1 --compile_aggregate_expressions 0 --compile_sort_description 1 --merge_tree_coarse_index_granularity 21 --optimize_distinct_in_order 1 --max_bytes_before_remerge_sort 1179420602 --min_compress_block_size 2128969 --max_compress_block_size 557103 --merge_tree_compact_parts_min_granules_to_multibuffer_read 79 --optimize_sorting_by_input_stream_properties 1 --http_response_buffer_size 9830613 --http_wait_end_of_query False --enable_memory_bound_merging_of_aggregation_results 1 --min_count_to_compile_expression 0 --min_count_to_compile_aggregate_expression 3 --min_count_to_compile_sort_description 0 --session_timezone America/Mazatlan --use_page_cache_for_disks_without_file_cache True --page_cache_inject_eviction False --merge_tree_read_split_ranges_into_intersecting_and_non_intersecting_injection_probability 0.06 --prefer_external_sort_block_bytes 100000000 --cross_join_min_rows_to_compress 1 --cross_join_min_bytes_to_compress 100000000 --min_external_table_block_size_bytes 0 --max_parsing_threads 0 --optimize_functions_to_subcolumns 1 --parallel_replicas_local_plan 1 --query_plan_join_swap_table false --enable_vertical_final 1 --optimize_extract_common_expressions 0 --use_async_executor_for_materialized_views 1 --use_query_condition_cache 0 --secondary_indices_enable_bulk_filtering 0 --use_skip_indexes_if_final 1 --use_skip_indexes_on_data_read 1 --optimize_rewrite_like_perfect_affix 1 --input_format_parquet_use_native_reader_v3 1 --enable_lazy_columns_replication 0 --allow_special_serialization_kinds_in_output_formats 1 --max_bytes_before_external_sort 9525137013 --max_bytes_before_external_group_by 3922677686 --max_bytes_ratio_before_external_sort 0 --max_bytes_ratio_before_external_group_by 0 --use_skip_indexes_if_final_exact_mode 1 --allow_merge_tree_settings --ratio_of_defaults_for_sparse_serialization 0.045773080549790346 --prefer_fetch_merged_part_size_threshold 10737418240 --vertical_merge_algorithm_min_rows_to_activate 1 --vertical_merge_algorithm_min_columns_to_activate 1 --allow_vertical_merges_from_compact_to_wide_parts 1 --min_merge_bytes_to_use_direct_io 3282821666 --index_granularity_bytes 2429783 --merge_max_block_size 17378 --index_granularity 5261 --min_bytes_for_wide_part 0 --marks_compress_block_size 51841 --primary_key_compress_block_size 36337 --replace_long_file_name_to_hash 1 --max_file_name_length 128 --min_bytes_for_full_part_storage 0 --compact_parts_max_bytes_to_buffer 36889637 --compact_parts_max_granules_to_buffer 190 --compact_parts_merge_max_bytes_to_prefetch_part 15841459 --cache_populated_by_fetch 0 --concurrent_part_removal_threshold 91 --old_parts_lifetime 129 --prewarm_mark_cache 1 --use_const_adaptive_granularity 0 --enable_index_granularity_compression 1 --enable_block_number_column 1 --enable_block_offset_column 1 --use_primary_key_cache 1 --prewarm_primary_key_cache 0 --object_serialization_version v3 --object_shared_data_serialization_version advanced --object_shared_data_serialization_version_for_zero_level_parts map_with_buckets --object_shared_data_buckets_for_compact_part 21 --object_shared_data_buckets_for_wide_part 27 --dynamic_serialization_version v3 --auto_statistics_types tdigest,minmax --serialization_info_version with_types --string_serialization_version single_stream --nullable_serialization_version basic --enable_shared_storage_snapshot_in_query 0 --allow_repeated_settings --client_logs_file=/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/queries/0_stateless/01459_manual_write_to_replicas_quorum_detach_attach.stderr-fatal -q '
2025-12-23 04:55:06         DROP TABLE IF EXISTS r2 SYNC;
2025-12-23 04:55:06         CREATE TABLE r2 (x UInt64) ENGINE = ReplicatedMergeTree('\''/clickhouse/tables/01459_manual_write_to_replicas_quorum_detach_attach_test_oxmeq0tg/r'\'', '\''r2'\'') ORDER BY x;
2025-12-23 04:55:06     '
2025-12-23 04:55:06 + [2025-12-23 04:53:33] [:13] for i in $(seq 1 $NUM_
Run attempt 2 out of 5