Set a deadline on every gRPC call and pass it on

How to do it

gRPC sets no deadline by default, so a stuck server holds the caller's thread and connection forever. gRPC sends the deadline as a time left, so clocks on two machines need not agree. Java and Go pass an incoming deadline to outgoing calls by default, and C++ does not. Pick the value as set timeouts on every call says. A call that ends with DEADLINE_EXCEEDED may still have succeeded on the server. Retry writes only with an idempotency key, as accept idempotency keys on POST says.

Examples

Go example

ctx, cancel := context.WithTimeout(ctx, 2*time.Second)
defer cancel()
invoice, err := client.GetInvoice(ctx, &pb.GetInvoiceRequest{Id: id})
if status.Code(err) == codes.DeadlineExceeded {
	return nil, fmt.Errorf("invoice service too slow: %w", err)
}

Python example

try:
    invoice = stub.GetInvoice(GetInvoiceRequest(id=invoice_id), timeout=2.0)
except grpc.RpcError as error:
    if error.code() == grpc.StatusCode.DEADLINE_EXCEEDED:
        raise InvoiceServiceSlow(invoice_id) from error
    raise

How to check

A test calls each method against a server stub that never answers. The call fails with DEADLINE_EXCEEDED within its deadline plus 1 second. A lint or code search finds no client call without a deadline or timeout.

Sources

• gRPC, Deadlines

Related rules

• API-35 Set timeouts on every call in API design
• API-32 Accept idempotency keys on POST in API design