Hello,
We recently upgrade our gcloud library to use the new streaming pubsub client. We've had a few encounters where the client appears to be idle even with a large number of Undelivered messages showing in stackdriver. After some investigation it appears the root cause is a resource limit:
rpc error: code = ResourceExhausted desc = Your project has exceeded a limit: (type="StreamingPull connections", current=1010, maximum=1000).
We've already reached out for a limit increase but I wanted to make sure from the client perspective that this error should be considered 'retryable'.
func isRetryable(err error) bool {
s, ok := status.FromError(err)
if !ok { // includes io.EOF, normal stream close, which causes us to reopen
return true
}
switch s.Code() {
case codes.DeadlineExceeded, codes.Internal, codes.Canceled, codes.ResourceExhausted:
return true
case codes.Unavailable:
return !strings.Contains(s.Message(), "Server shutdownNow invoked")
default:
return false
}
}
Or perhaps this error should be logged to make it apparent?
Hello,
We recently upgrade our gcloud library to use the new streaming pubsub client. We've had a few encounters where the client appears to be idle even with a large number of Undelivered messages showing in stackdriver. After some investigation it appears the root cause is a resource limit:
We've already reached out for a limit increase but I wanted to make sure from the client perspective that this error should be considered 'retryable'.
Or perhaps this error should be logged to make it apparent?