I want to set a server-side deadline on a gRPC call so that the call is cancelled if it takes too long, This works in Java/GAX by setting the totalTimeout in RetrySettings, so the server-side appears to be working correctly.

Attempting to do the same in Node.js/GAX seems to be very confusing, and not matching the documentation:

TL/DR:

• it seems that the CallOptions.timeout which is documented as a client-side timeout seems to also work as a server-side timeout, but only when RetryOptions are specified, which seems to go against the documentation.
• Certain values for RetryOptions give unexpected errors when specified together.
• RetryOptions are ignored completely unless at least one RetryCode is given.
• Cannot set both totalTimeoutMillis and maxRetries in BackoffSettings
• BackoffSettings.totalTimeoutMillis seems to be completely ignored

https://github.com/googleapis/gax-nodejs/blob/master/src/gax.ts#L157

Demonstration of CallOptions.timeout on DEADLINE_EXCEEDED

callOptions = {
  retry: {
    backoffSettings: {
      totalTimeoutMillis: 200
    }
  }
}

Has no effect - the long-running gRPC is not timed out.

callOptions = {
  timeout: 200
}

Has no effect - the long-running gRPC is not timed out.

callOptions = {
  timeout: 200,
  retry: {}
  }
}

Has no effect - the long-running gRPC is not timed out.

callOptions = {
  timeout: 200,
  retry: {
    retryCodes: [],
    }
  }

DEADLINE_EXCEEDED Raised

These tests indicate that the CallOptions.timeout is being passed as a deadline to the server (when it is documented as a client-side timeout), BUT only when a CallOptions.retry.retryCodes object is specified. The values in the CallOptions.retry object seem to be ignored.

Demonstration of totalTimeoutMillis

callOptions = {
  timeout: 200000,
  retry: {
    retryCodes: [],
    backoffSettings: {
      totalTimeoutMillis: 200
    }
  }
}

Has no effect - the long-running gRPC is not timed out.

callOptions = {
  timeout: 200000,
  retry: {
    retryCodes: [ 99999 ],
      backoffSettings: {
        totalTimeoutMillis: 200
     }
  }
}

Has no effect - the long-running gRPC is not timed out.
totalTimeoutMillis seems to be ignored

Demonstration of CallOptions.retry on DEADLINE_EXCEEDED

callOptions = {
  retry: {
    retryCodes: [ 99999 ],
      backoffSettings: {
        totalTimeoutMillis: 200
     }
  }
}

Has no effect - the long-running gRPC is not timed out.

callOptions = {
  retry: {
    retryCodes: [ 99999 ],
      backoffSettings: {
        maxRetries: 1,
        totalTimeoutMillis: 200
     }
  }
}

Client-side error:

Error: Cannot set both totalTimeoutMillis and maxRetries in backoffSettings

This makes no sense, as it seems perfectly possible to have retries and a total timeout (and the documentation implies this: regardless of the retrying attempts made meanwhile.

callOptions = {
  retry: {
    retryCodes: [ 99999 ],
      backoffSettings: {
        maxRetries: 1,
        initialRpcTimeoutMillis: 200,
        maxRpcTimeoutMillis: 200
     }
  }
}

DEADLINE_EXCEEDED Raised (woohoo!)

callOptions = {
  timeout: 200,
  retry: {
    retryCodes: [ 99999 ],
      backoffSettings: {
        maxRetries: 1,
        initialRpcTimeoutMillis: 200000,
        maxRpcTimeoutMillis: 200000
     }
  }
}

Has no effect - the long-running gRPC is not timed out.
This does correspond to the docs - that CallOptions.timeout is ignored for retryable calls.

callOptions = {
  retry: {
    retryCodes: [ ],
      backoffSettings: {
        maxRetries: 1,
        initialRpcTimeoutMillis: 200,
        maxRpcTimeoutMillis: 200
     }
  }
}

Has no effect - the long-running gRPC is not timed out.

So at least 1 retryCode must be specified

Environment details

OS: Debian Rodete (20210511.01.04RD)
node: v12.21.0
npm: 7.5.2
[email protected]
grpc/[email protected]
[email protected]