-
Notifications
You must be signed in to change notification settings - Fork 11.1k
502 errors in cloud to prod interop tests for aspnetcore, php, and dart clients #19723
Copy link
Copy link
Closed
Closed
Copy link
Description
Saw a bunch of 502 errors from aspentcore, php, and dart clients when targetting the default cloud to prod interop server.
E.g.:
2019-07-22 21:01:52,284 START: cloud_to_prod:php7:default:ping_pong:tls
2019-07-22 21:01:52,289 WAITING: ETA 791.9 sec; 470 queued, 12 jobs running, 18 complete, 12 failed (load 12.00) next: cloud_to_prod:php7:default:empty_stream:tls @ 1.00 cpu
2019-07-22 21:02:02,289 WAITING: ETA 1059.7 sec; 470 queued, 12 jobs running, 18 complete, 12 failed (load 12.00) next: cloud_to_prod:php7:default:empty_stream:tls @ 1.00 cpu
2019-07-22 21:02:12,290 WAITING: ETA 1327.5 sec; 470 queued, 12 jobs running, 18 complete, 12 failed (load 12.00) next: cloud_to_prod:php7:default:empty_stream:tls @ 1.00 cpu
2019-07-22 21:02:12,964 Unhandled exception:
gRPC Error (14, 502:Bad Gateway)
#0 Tester.emptyUnary (package:interop/src/client.dart:177:22)
<asynchronous suspension>
#1 Tester.runTestCase (package:interop/src/client.dart:115:16)
<asynchronous suspension>
#2 Tester.runTest (package:interop/src/client.dart:108:11)
<asynchronous suspension>
#3 main (file:///var/local/git/grpc-dart/interop/bin/client.dart:107:20)
<asynchronous suspension>
#4 _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:299:32)
#5 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)
2019-07-22 21:02:12,964 FAILED: cloud_to_prod:dart:default:empty_unary:tls [ret=255, pid=10952, time=28.4sec]
2019-07-22 21:02:12,964 START: cloud_to_prod:php7:default:empty_stream:tls
2019-07-22 21:02:12,969 WAITING: ETA 1272.2 sec; 469 queued, 12 jobs running, 19 complete, 13 failed (load 12.00) next: cloud_to_prod:php7:default:client_streaming:tls @ 1.00 cpu
2019-07-22 21:02:13,354 Running test case ping_pong
Server returned too few responses
2019-07-22 21:02:13,355 FAILED: cloud_to_prod:php7:default:ping_pong:tls [ret=1, pid=12085, time=21.1sec]
2019-07-22 21:02:13,355 START: cloud_to_prod:php7:default:client_streaming:tls
2019-07-22 21:02:13,359 WAITING: ETA 1215.4 sec; 468 queued, 12 jobs running, 20 complete, 14 failed (load 12.00) next: cloud_to_prod:php7:default:server_streaming:tls @ 1.00 cpu
2019-07-22 21:02:13,398 Unhandled exception:
gRPC Error (14, 502:Bad Gateway)
#0 Tester.pingPong (package:interop/src/client.dart:585:5)
<asynchronous suspension>
#1 Tester.runTestCase (package:interop/src/client.dart:133:16)
<asynchronous suspension>
#2 Tester.runTest (package:interop/src/client.dart:108:11)
<asynchronous suspension>
#3 main (file:///var/local/git/grpc-dart/interop/bin/client.dart:107:20)
<asynchronous suspension>
#4 _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:299:32)
#5 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)
2019-07-22 21:02:13,398 FAILED: cloud_to_prod:dart:default:ping_pong:tls [ret=255, pid=10974, time=28.8sec]
2019-07-22 21:02:13,398 START: cloud_to_prod:php7:default:server_streaming:tls
2019-07-22 21:02:13,403 WAITING: ETA 1156.1 sec; 467 queued, 12 jobs running, 21 complete, 15 failed (load 12.00) next: cloud_to_prod:php7:default:cancel_after_begin:tls @ 1.00 cpu
2019-07-22 21:02:13,809 Unhandled exception:
gRPC Error (14, 502:Bad Gateway)
#0 Tester.serverStreaming (package:interop/src/client.dart:466:23)
<asynchronous suspension>
#1 Tester.runTestCase (package:interop/src/client.dart:129:16)
<asynchronous suspension>
#2 Tester.runTest (package:interop/src/client.dart:108:11)
<asynchronous suspension>
#3 main (file:///var/local/git/grpc-dart/interop/bin/client.dart:107:20)
<asynchronous suspension>
#4 _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:299:32)
#5 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)
2019-07-22 21:02:13,810 FAILED: cloud_to_prod:dart:default:server_streaming:tls [ret=255, pid=11011, time=29.2sec]
2019-07-22 21:02:13,810 START: cloud_to_prod:php7:default:cancel_after_begin:tls
2019-07-22 21:02:13,815 WAITING: ETA 1110.2 sec; 466 queued, 12 jobs running, 22 complete, 16 failed (load 12.00) next: cloud_to_prod:php7:default:cancel_after_first_response:tls @ 1.00 cpu
2019-07-22 21:02:13,833 Running test case empty_unary
Call did not complete successfully. Status object:
object(stdClass)#389 (3) {
["metadata"]=>
array(3) {
["content-length"]=>
array(1) {
[0]=>
string(1) "0"
}
["date"]=>
array(1) {
[0]=>
string(29) "Tue, 23 Jul 2019 04:02:14 GMT"
}
["expect-ct"]=>
array(1) {
[0]=>
string(7) "preload"
}
}
["code"]=>
int(14)
["details"]=>
string(15) "502:Bad Gateway"
}
2019-07-22 21:02:13,834 FAILED: cloud_to_prod:php7:default:empty_unary:tls [ret=1, pid=11956, time=22.0sec]
2019-07-22 21:02:13,834 START: cloud_to_prod:php7:default:cancel_after_first_response:tls
2019-07-22 21:02:13,839 WAITING: ETA 1060.2 sec; 465 queued, 12 jobs running, 23 complete, 17 failed (load 12.00) next: cloud_to_prod:php7:default:timeout_on_sleeping_server:tls @ 1.00 cpu
2019-07-22 21:02:13,841 Running test case large_unary
Call did not complete successfully. Status object:
object(stdClass)#390 (3) {
Seen in PR: #19693
Reactions are currently unavailable