-
Notifications
You must be signed in to change notification settings - Fork 8.3k
test_s3_disk_reads_on_unstable_connection is flaky #88503
Copy link
Copy link
Closed
Labels
flaky testflaky test found by CIflaky test found by CI
Description
Failure example from #85920 here
File: test_merge_tree_s3/test.py:781 - in test_s3_disk_reads_on_unstable_connection
assert node.query("SELECT sum(id) FROM s3_test").splitlines() == [
File: helpers/cluster.py:4255 - in query
return self.client.query(
File: helpers/client.py:39 - in wrap
return func(self, *args, **kwargs)
File: helpers/client.py:79 - in query
).get_answer()
File: helpers/client.py:247 - in get_answer
raise QueryRuntimeException(
E helpers.client.QueryRuntimeException: Client failed! Return code: 232, stderr: Received exception from server (version 25.10.1):
E Code: 1000. DB::Exception: Received from 172.16.1.6:9000. DB::Exception: Malformed message: Unexpected EOF. Stack trace:
E
E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::IOException::IOException(String const&, int) @ 0x000000001e2639f2
E 1. ./ci/tmp/build/./base/poco/Net/src/NetException.cpp:26: Poco::Net::MessageException::MessageException(String const&, int) @ 0x000000001e2fdec9
E 2. ./ci/tmp/build/./base/poco/Net/src/HTTPFixedLengthStream.cpp:63: Poco::Net::HTTPFixedLengthStreamBuf::readFromDevice(char*, long) @ 0x000000001e2e1e77
E 3. ./ci/tmp/build/./src/IO/ReadBufferFromIStream.cpp:27: DB::ReadBufferFromIStream::nextImpl() @ 0x00000000123c3410
E 4. ./ci/tmp/build/./src/IO/ReadBuffer.cpp:96: DB::ReadBuffer::next() @ 0x00000000101184ed
E 5. ./ci/tmp/build/./src/IO/ReadBufferFromS3.cpp:171: DB::ReadBufferFromS3::nextImpl() @ 0x0000000013ff9b55
E 6. ./ci/tmp/build/./src/IO/ReadBuffer.cpp:96: DB::ReadBuffer::next() @ 0x00000000101184ed
E 7. ./ci/tmp/build/./src/Disks/IO/ReadBufferFromRemoteFSGather.cpp:122: DB::ReadBufferFromRemoteFSGather::nextImpl() @ 0x0000000014e8d14e
E 8. ./ci/tmp/build/./src/IO/ReadBuffer.cpp:96: DB::ReadBuffer::next() @ 0x00000000101184ed
E 9. ./ci/tmp/build/./src/Disks/IO/ThreadPoolRemoteFSReader.cpp:157: DB::ThreadPoolRemoteFSReader::execute(DB::IAsynchronousReader::Request, bool) @ 0x0000000013e5c9a3
E 10. ./ci/tmp/build/./src/Disks/IO/ThreadPoolRemoteFSReader.cpp:114: std::__packaged_task_func<std::function<std::future<DB::IAsynchronousReader::Result> (DB::ThreadPoolRemoteFSReader::submit(DB::IAsynchronousReader::Request)::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<DB::IAsynchronousReader::Result, DB::ThreadPoolRemoteFSReader::submit(DB::IAsynchronousReader::Request)::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>&, String const&)::'lambda'(DB::ThreadPoolRemoteFSReader::submit(DB::IAsynchronousReader::Request)::$_0&&, Priority)::operator()(DB::ThreadPoolRemoteFSReader::submit(DB::IAsynchronousReader::Request)::$_0&&, Priority)::'lambda'(), std::allocator<std::function<std::future<DB::IAsynchronousReader::Result> (DB::ThreadPoolRemoteFSReader::submit(DB::IAsynchronousReader::Request)::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<DB::IAsynchronousReader::Result, DB::ThreadPoolRemoteFSReader::submit(DB::IAsynchronousReader::Request)::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>&, String const&)::'lambda'(DB::ThreadPoolRemoteFSReader::submit(DB::IAsynchronousReader::Request)::$_0&&, Priority)::operator()(DB::ThreadPoolRemoteFSReader::submit(DB::IAsynchronousReader::Request)::$_0&&, Priority)::'lambda'()>, DB::IAsynchronousReader::Result ()>::operator()() @ 0x0000000013e5d873
E 11. ./contrib/llvm-project/libcxx/include/future:1589: std::packaged_task<DB::IAsynchronousReader::Result ()>::operator()() @ 0x0000000013e5b108
E 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001017d595
E 13. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: ThreadFromGlobalPoolImpl<false, true>::ThreadFromGlobalPoolImpl<void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::*)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool*>(void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool*&&)::'lambda'()::operator()() @ 0x0000000010182ce6
E 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001017afe0
E 15. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: void* std::__thread_proxy[abi:ne190107]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*>>(void*) @ 0x0000000010180dfb
E 16. ? @ 0x0000000000094ac3
E 17. ? @ 0x0000000000126850
E . (POCO_EXCEPTION)
E (query: SELECT sum(id) FROM s3_test)
===== Captured log setup =====
2025-10-09 02:08:43.602000 [ 609792 ] INFO : list_objects (0): [] (test.py:142, list_objects)
===== Captured stdout call =====
Read sequence 0
Read sequence 1
Read sequence 2
Read sequence 3
Read sequence 4
Read sequence 5
Read sequence 6
Read sequence 7
Read sequence 8
Read sequence 9
Read sequence 10
Read sequence 11
Read sequence 12
Read sequence 13
Read sequence 14
Read sequence 15
Read sequence 16
Read sequence 17
Read sequence 18
Read sequence 19
Read sequence 20
Read sequence 21
Read sequence 22
Read sequence 23
Read sequence 24
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
flaky testflaky test found by CIflaky test found by CI