-
Notifications
You must be signed in to change notification settings - Fork 8.3k
test_distributed_queries_stress is flaky #41776
Copy link
Copy link
Closed
Labels
testingSpecial issue with list of bugs found by CISpecial issue with list of bugs found by CI
Description
> run_benchmark(payload, settings)
test_distributed_queries_stress/test.py:121:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_distributed_queries_stress/test.py:18: in run_benchmark
node1_r1.exec_in_container(
helpers/cluster.py:3408: in exec_in_container
return self.cluster.exec_in_container(
helpers/cluster.py:1752: in exec_in_container
result = subprocess_check_call(
helpers/cluster.py:151: in subprocess_check_call
return run_and_check(args, detach=detach, nothrow=nothrow)
helpers/cluster.py:96: in run_and_check
res = subprocess.run(
/usr/lib/python3.8/subprocess.py:495: in run
stdout, stderr = process.communicate(input, timeout=timeout)
/usr/lib/python3.8/subprocess.py:1028: in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
/usr/lib/python3.8/subprocess.py:1869: in _communicate
self._check_timeout(endtime, orig_timeout, stdout, stderr)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <subprocess.Popen object at 0x7f9298feb280>, endtime = 3483.629056787
orig_timeout = 300, stdout_seq = [], stderr_seq = [b'Loaded 5 queries.\n']
skip_check_and_raise = False
def _check_timeout(self, endtime, orig_timeout, stdout_seq, stderr_seq,
skip_check_and_raise=False):
"""Convenience for checking if a timeout has expired."""
if endtime is None:
return
if skip_check_and_raise or _time() > endtime:
> raise TimeoutExpired(
self.args, orig_timeout,
output=b''.join(stdout_seq) if stdout_seq else None,
stderr=b''.join(stderr_seq) if stderr_seq else None)
E subprocess.TimeoutExpired: Command '['docker', 'exec', 'roottestdistributedqueriesstress_node1_r1_1', 'bash', '-c', "echo 'select * from dist_two_over_dist where key = 0;\n select * from dist_two_over_dist where key = 1;\n select * from dist_two_over_dist where key = 2;\n select * from dist_two_over_dist where key = 3;\n select * from dist_two_over_dist;' | clickhouse benchmark --concurrency=100 --cumulative --delay=0 --timelimit=3 --hedged_connection_timeout_ms=200 --connect_timeout_with_failover_ms=200 --connections_with_failover_max_tries=5 --optimize_skip_unused_shards=1 --distributed_group_by_no_merge=2"]' timed out after 300 seconds
But I have no ideas why clickhouse-benchmark did not finish in 300 seconds when timelimit was set to 3 seconds
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
testingSpecial issue with list of bugs found by CISpecial issue with list of bugs found by CI