Skip to content

00601_kill_running_query is flaky #88563

@Ergus

Description

@Ergus

Test case history:

Example failure from #88501 here

2025-10-15 09:22:10 Reason: result differs with reference:  
2025-10-15 09:22:10 --- /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/queries/0_stateless/00601_kill_running_query.reference	2025-10-15 09:16:41.262381163 +0300
2025-10-15 09:22:10 +++ /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/queries/0_stateless/00601_kill_running_query.stdout	2025-10-15 09:22:10.420520685 +0300
2025-10-15 09:22:10 @@ -1 +0,0 @@
2025-10-15 09:22:10 -waiting	test_00601_default	default	SELECT sum(ignore(*)) FROM (SELECT number % 1000 AS k, groupArray(number) FROM numbers(50000000) GROUP BY k) SETTINGS max_rows_to_read = 0
2025-10-15 09:22:10 
2025-10-15 09:22:10 
2025-10-15 09:22:10 /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/queries/0_stateless/test_vvi3hdyy/00601_kill_running_query.sh.debuglog:
2025-10-15 09:22:10 + [2025-10-15 09:22:09] [:7] set -e -o pipefail
2025-10-15 09:22:10 + [2025-10-15 09:22:09] [:15] wait_for_query_to_start test_00601_test_vvi3hdyy
2025-10-15 09:22:10 + [2025-10-15 09:22:09] [:14] curl -q --max-time 30 -sS 'http://localhost:8123/?max_insert_threads=2&group_by_two_level_threshold=671121&group_by_two_level_threshold_bytes=42603766&distributed_aggregation_memory_efficient=0&fsync_metadata=0&output_format_parallel_formatting=0&input_format_parallel_parsing=1&min_chunk_bytes_for_parallel_parsing=6005719&max_read_buffer_size=548351&prefer_localhost_replica=1&max_block_size=84640&max_joined_block_size_rows=82145&joined_block_split_single_row=0&join_output_by_rowlist_perkey_rows_threshold=4&max_threads=2&optimize_append_index=1&use_hedged_requests=1&optimize_if_chain_to_multiif=0&optimize_if_transform_strings_to_enum=0&optimize_read_in_order=1&optimize_or_like_chain=1&optimize_substitute_columns=0&enable_multiple_prewhere_read_steps=0&read_in_order_two_level_merge_threshold=9&optimize_aggregation_in_order=0&aggregation_in_order_max_block_bytes=632734&use_uncompressed_cache=1&min_bytes_to_use_direct_io=1&min_bytes_to_use_mmap_io=10737418240&local_filesystem_read_method=read&remote_filesystem_read_method=read&local_filesystem_read_prefetch=1&filesystem_cache_segments_batch_size=5&read_from_filesystem_cache_if_exists_otherwise_bypass_cache=0&throw_on_error_from_cache_on_write_operations=1&remote_filesystem_read_prefetch=0&allow_prefetched_read_pool_for_remote_filesystem=1&filesystem_prefetch_max_memory_usage=64Mi&filesystem_prefetches_limit=10&filesystem_prefetch_min_bytes_for_single_read_task=8Mi&filesystem_prefetch_step_marks=50&filesystem_prefetch_step_bytes=100Mi&compile_expressions=0&compile_aggregate_expressions=1&compile_sort_description=0&merge_tree_coarse_index_granularity=8&optimize_distinct_in_order=0&max_bytes_before_remerge_sort=2435494765&min_compress_block_size=1196572&max_compress_block_size=1305927&merge_tree_compact_parts_min_granules_to_multibuffer_read=77&optimize_sorting_by_input_stream_properties=0&http_response_buffer_size=10330383&http_wait_end_of_query=False&enable_memory_bound_merging_of_aggregation_results=0&min_count_to_compile_expression=3&min_count_to_compile_aggregate_expression=3&min_count_to_compile_sort_description=0&session_timezone=Mexico%2FBajaSur&use_page_cache_for_disks_without_file_cache=False&page_cache_inject_eviction=False&merge_tree_read_split_ranges_into_intersecting_and_non_intersecting_injection_probability=0.82&prefer_external_sort_block_bytes=1&cross_join_min_rows_to_compress=0&cross_join_min_bytes_to_compress=1&min_external_table_block_size_bytes=100000000&max_parsing_threads=1&optimize_functions_to_subcolumns=1&parallel_replicas_local_plan=1&query_plan_join_swap_table=auto&enable_vertical_final=0&optimize_extract_common_expressions=1&use_async_executor_for_materialized_views=1&use_query_condition_cache=0&secondary_indices_enable_bulk_filtering=1&use_skip_indexes_if_final=1&use_skip_indexes_on_data_read=1&optimize_rewrite_like_perfect_affix=0&max_bytes_before_external_sort=0&max_bytes_before_external_group_by=0&max_bytes_ratio_before_external_sort=0.2&max_bytes_ratio_before_external_group_by=0.0&use_skip_indexes_if_final_exact_mode=1&database=test_vvi3hdyy&log_comment=00601_kill_running_query.sh-test_vvi3hdyy&query_id=test_00601_test_vvi3hdyy' -d 'SELECT sum(ignore(*)) FROM (SELECT number % 1000 AS k, groupArray(number) FROM numbers(50000000) GROUP BY k) SETTINGS max_rows_to_read = 0'
2025-10-15 09:22:10 ++ [2025-10-15 09:22:09] [:11] curl -q -s --max-time 120 -sS 'http://localhost:8123/?max_insert_threads=2&group_by_two_level_threshold=671121&group_by_two_level_threshold_bytes=42603766&distributed_aggregation_memory_efficient=0&fsync_metadata=0&output_format_parallel_formatting=0&input_format_parallel_parsing=1&min_chunk_bytes_for_parallel_parsing=6005719&max_read_buffer_size=548351&prefer_localhost_replica=1&max_block_size=84640&max_joined_block_size_rows=82145&joined_block_split_single_row=0&join_output_by_rowlist_perkey_rows_threshold=4&max_threads=2&optimize_append_index=1&use_hedged_requests=1&optimize_if_chain_to_multiif=0&optimize_if_transform_strings_to_enum=0&optimize_read_in_order=1&optimize_or_like_chain=1&optimize_substitute_columns=0&enable_multiple_prewhere_read_steps=0&read_in_order_two_level_merge_threshold=9&optimize_aggregation_in_order=0&aggregation_in_order_max_block_bytes=632734&use_uncompressed_cache=1&min_bytes_to_use_direct_io=1&min_bytes_to_use_mmap_io=10737418240&local_filesystem_read_method=read&remote_filesystem_read_method=read&local_filesystem_read_prefetch=1&filesystem_cache_segments_batch_size=5&read_from_filesystem_cache_if_exists_otherwise_bypass_cache=0&throw_on_error_from_cache_on_write_operations=1&remote_filesystem_read_prefetch=0&allow_prefetched_read_pool_for_remote_filesystem=1&filesystem_prefetch_max_memory_usage=64Mi&filesystem_prefetches_limit=10&filesystem_prefetch_min_bytes_for_single_read_task=8Mi&filesystem_prefetch_step_marks=50&filesystem_prefetch_step_bytes=100Mi&compile_expressions=0&compile_aggregate_expressions=1&compile_sort_description=0&merge_tree_coarse_index_granularity=8&optimize_distinct_in_order=0&max_bytes_before_remerge_sort=2435494765&min_compress_block_size=1196572&max_compress_block_size=1305927&merge_tree_compact_parts_min_granules_to_multibuffer_read=77&optimize_sorting_by_input_stream_properties=0&http_response_buffer_size=10330383&http_wait_end_of_query=False&enable_memory_bound_merging_of_aggregation_results=0&min_count_to_compile_expression=3&min_count_to_compile_aggregate_expression=3&min_count_to_compile_sort_description=0&session_timezone=Mexico%2FBajaSur&use_page_cache_for_disks_without_file_cache=False&page_cache_inject_eviction=False&merge_tree_read_split_ranges_into_intersecting_and_non_intersecting_injection_probability=0.82&prefer_external_sort_block_bytes=1&cross_join_min_rows_to_compress=0&cross_join_min_bytes_to_compress=1&min_external_table_block_size_bytes=100000000&max_parsing_threads=1&optimize_functions_to_subcolumns=1&parallel_replicas_local_plan=1&query_plan_join_swap_table=auto&enable_vertical_final=0&optimize_extract_common_expressions=1&use_async_executor_for_materialized_views=1&use_query_condition_cache=0&secondary_indices_enable_bulk_filtering=1&use_skip_indexes_if_final=1&use_skip_indexes_on_data_read=1&optimize_rewrite_like_perfect_affix=0&max_bytes_before_external_sort=0&max_bytes_before_external_group_by=0&max_bytes_ratio_before_external_sort=0.2&max_bytes_ratio_before_external_group_by=0.0&use_skip_indexes_if_final_exact_mode=1&database=test_vvi3hdyy&log_comment=00601_kill_running_query.sh-test_vvi3hdyy' -d 'SELECT count() FROM system.processes WHERE query_id = '\''test_00601_test_vvi3hdyy'\'''
2025-10-15 09:22:10 + [2025-10-15 09:22:09] [:11] [[ 0 == 0 ]]
2025-10-15 09:22:10 + [2025-10-15 09:22:09] [:11] sleep 0.1
2025-10-15 09:22:10 ++ [2025-10-15 09:22:09] [:11] curl -q -s --max-time 120 -sS 'http://localhost:8123/?max_insert_threads=2&group_by_two_level_threshold=671121&group_by_two_level_threshold_bytes=42603766&distributed_aggregation_memory_efficient=0&fsync_metadata=0&output_format_parallel_formatting=0&input_format_parallel_parsing=1&min_chunk_bytes_for_parallel_parsing=6005719&max_read_buffer_size=548351&prefer_localhost_replica=1&max_block_size=84640&max_joined_block_size_rows=82145&joined_block_split_single_row=0&join_output_by_rowlist_perkey_rows_threshold=4&max_threads=2&optimize_append_index=1&use_hedged_requests=1&optimize_if_chain_to_multiif=0&optimize_if_transform_strings_to_enum=0&optimize_read_in_order=1&optimize_or_like_chain=1&optimize_substitute_columns=0&enable_multiple_prewhere_read_steps=0&read_in_order_two_level_merge_threshold=9&optimize_aggregation_in_order=0&aggregation_in_order_max_block_bytes=632734&use_uncompressed_cache=1&min_bytes_to_use_direct_io=1&min_bytes_to_use_mmap_io=10737418240&local_filesystem_read_method=read&remote_filesystem_read_method=read&local_filesystem_read_prefetch=1&filesystem_cache_segments_batch_size=5&read_from_filesystem_cache_if_exists_otherwise_bypass_cache=0&throw_on_error_from_cache_on_write_operations=1&remote_filesystem_read_prefetch=0&allow_prefetched_read_pool_for_remote_filesystem=1&filesystem_prefetch_max_memory_usage=64Mi&filesystem_prefetches_limit=10&filesystem_prefetch_min_bytes_for_single_read_task=8Mi&filesystem_prefetch_step_marks=50&filesystem_prefetch_step_bytes=100Mi&compile_expressions=0&compile_aggregate_expressions=1&compile_sort_description=0&merge_tree_coarse_index_granularity=8&optimize_distinct_in_order=0&max_bytes_before_remerge_sort=2435494765&min_compress_block_size=1196572&max_compress_block_size=1305927&merge_tree_compact_parts_min_granules_to_multibuffer_read=77&optimize_sorting_by_input_stream_properties=0&http_response_buffer_size=10330383&http_wait_end_of_query=False&enable_memory_bound_merging_of_aggregation_results=0&min_count_to_compile_expression=3&min_count_to_compile_aggregate_expression=3&min_count_to_compile_sort_description=0&session_timezone=Mexico%2FBajaSur&use_page_cache_for_disks_without_file_cache=False&page_cache_inject_eviction=False&merge_tree_read_split_ranges_into_intersecting_and_non_intersecting_injection_probability=0.82&prefer_external_sort_block_bytes=1&cross_join_min_rows_to_compress=0&cross_join_min_bytes_to_compress=1&min_external_table_block_size_bytes=100000000&max_parsing_threads=1&optimize_functions_to_subcolumns=1&parallel_replicas_local_plan=1&query_plan_join_swap_table=auto&enable_vertical_final=0&optimize_extract_common_expressions=1&use_async_executor_for_materialized_views=1&use_query_condition_cache=0&secondary_indices_enable_bulk_filtering=1&use_skip_indexes_if_final=1&use_skip_indexes_on_data_read=1&optimize_rewrite_like_perfect_affix=0&max_bytes_before_external_sort=0&max_bytes_before_external_group_by=0&max_bytes_ratio_before_external_sort=0.2&max_bytes_ratio_before_external_group_by=0.0&use_skip_indexes_if_final_exact_mode=1&database=test_vvi3hdyy&log_comment=00601_kill_running_query.sh-test_vvi3hdyy' -d 'SELECT count() FROM system.processes WHERE query_id = '\''test_00601_test_vvi3hdyy'\'''
2025-10-15 09:22:10 + [2025-10-15 09:22:09] [:11] [[ 1 == 0 ]]
2025-10-15 09:22:10 + [2025-10-15 09:22:10] [:16] curl -q -s --max-time 120 -sS 'http://localhost:8123/?max_insert_threads=2&group_by_two_level_threshold=671121&group_by_two_level_threshold_bytes=42603766&distributed_aggregation_memory_efficient=0&fsync_metadata=0&output_format_parallel_formatting=0&input_format_parallel_parsing=1&min_chunk_bytes_for_parallel_parsing=6005719&max_read_buffer_size=548351&prefer_localhost_replica=1&max_block_size=84640&max_joined_block_size_rows=82145&joined_block_split_single_row=0&join_output_by_rowlist_perkey_rows_threshold=4&max_threads=2&optimize_append_index=1&use_hedged_requests=1&optimize_if_chain_to_multiif=0&optimize_if_transform_strings_to_enum=0&optimize_read_in_order=1&optimize_or_like_chain=1&optimize_substitute_columns=0&enable_multiple_prewhere_read_steps=0&read_in_order_two_level_merge_threshold=9&optimize_aggregation_in_order=0&aggregation_in_order_max_block_bytes=632734&use_uncompressed_cache=1&min_bytes_to_use_direct_io=1&min_bytes_to_use_mmap_io=10737418240&local_filesystem_read_method=read&remote_filesystem_read_method=read&local_filesystem_read_prefetch=1&filesystem_cache_segments_batch_size=5&read_from_filesystem_cache_if_exists_otherwise_bypass_cache=0&throw_on_error_from_cache_on_write_operations=1&remote_filesystem_read_prefetch=0&allow_prefetched_read_pool_for_remote_filesystem=1&filesystem_prefetch_max_memory_usage=64Mi&filesystem_prefetches_limit=10&filesystem_prefetch_min_bytes_for_single_read_task=8Mi&filesystem_prefetch_step_marks=50&filesystem_prefetch_step_bytes=100Mi&compile_expressions=0&compile_aggregate_expressions=1&compile_sort_description=0&merge_tree_coarse_index_granularity=8&optimize_distinct_in_order=0&max_bytes_before_remerge_sort=2435494765&min_compress_block_size=1196572&max_compress_block_size=1305927&merge_tree_compact_parts_min_granules_to_multibuffer_read=77&optimize_sorting_by_input_stream_properties=0&http_response_buffer_size=10330383&http_wait_end_of_query=False&enable_memory_bound_merging_of_aggregation_results=0&min_count_to_compile_expression=3&min_count_to_compile_aggregate_expression=3&min_count_to_compile_sort_description=0&session_timezone=Mexico%2FBajaSur&use_page_cache_for_disks_without_file_cache=False&page_cache_inject_eviction=False&merge_tree_read_split_ranges_into_intersecting_and_non_intersecting_injection_probability=0.82&prefer_external_sort_block_bytes=1&cross_join_min_rows_to_compress=0&cross_join_min_bytes_to_compress=1&min_external_table_block_size_bytes=100000000&max_parsing_threads=1&optimize_functions_to_subcolumns=1&parallel_replicas_local_plan=1&query_plan_join_swap_table=auto&enable_vertical_final=0&optimize_extract_common_expressions=1&use_async_executor_for_materialized_views=1&use_query_condition_cache=0&secondary_indices_enable_bulk_filtering=1&use_skip_indexes_if_final=1&use_skip_indexes_on_data_read=1&optimize_rewrite_like_perfect_affix=0&max_bytes_before_external_sort=0&max_bytes_before_external_group_by=0&max_bytes_ratio_before_external_sort=0.2&max_bytes_ratio_before_external_group_by=0.0&use_skip_indexes_if_final_exact_mode=1&database=test_vvi3hdyy&log_comment=00601_kill_running_query.sh-test_vvi3hdyy' -d 'KILL QUERY WHERE query_id = '\''test_00601_test_vvi3hdyy'\'''
2025-10-15 09:22:10 + [2025-10-15 09:22:10] [:17] wait
2025-10-15 09:22:10 
2025-10-15 09:22:10 Settings used in the test: --max_insert_threads 2 --group_by_two_level_threshold 671121 --group_by_two_level_threshold_bytes 42603766 --distributed_aggregation_memory_efficient 0 --fsync_metadata 0 --output_format_parallel_formatting 0 --input_format_parallel_parsing 1 --min_chunk_bytes_for_parallel_parsing 6005719 --max_read_buffer_size 548351 --prefer_localhost_replica 1 --max_block_size 84640 --max_joined_block_size_rows 82145 --joined_block_split_single_row 0 --join_output_by_rowlist_perkey_rows_threshold 4 --max_threads 2 --optimize_append_index 1 --use_hedged_requests 1 --optimize_if_chain_to_multiif 0 --optimize_if_transform_strings_to_enum 0 --optimize_read_in_order 1 --optimize_or_like_chain 1 --optimize_substitute_columns 0 --enable_multiple_prewhere_read_steps 0 --read_in_order_two_level_merge_threshold 9 --optimize_aggregation_in_order 0 --aggregation_in_order_max_block_bytes 632734 --use_uncompressed_cache 1 --min_bytes_to_use_direct_io 1 --min_bytes_to_use_mmap_io 10737418240 --local_filesystem_read_method read --remote_filesystem_read_method read --local_filesystem_read_prefetch 1 --filesystem_cache_segments_batch_size 5 --read_from_filesystem_cache_if_exists_otherwise_bypass_cache 0 --throw_on_error_from_cache_on_write_operations 1 --remote_filesystem_read_prefetch 0 --allow_prefetched_read_pool_for_remote_filesystem 1 --filesystem_prefetch_max_memory_usage 64Mi --filesystem_prefetches_limit 10 --filesystem_prefetch_min_bytes_for_single_read_task 8Mi --filesystem_prefetch_step_marks 50 --filesystem_prefetch_step_bytes 100Mi --compile_expressions 0 --compile_aggregate_expressions 1 --compile_sort_description 0 --merge_tree_coarse_index_granularity 8 --optimize_distinct_in_order 0 --max_bytes_before_remerge_sort 2435494765 --min_compress_block_size 1196572 --max_compress_block_size 1305927 --merge_tree_compact_parts_min_granules_to_multibuffer_read 77 --optimize_sorting_by_input_stream_propert

Metadata

Metadata

Labels

flaky testflaky test found by CI

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions