using gRPC alongside REST for internal APIs
New internal APIs must have both gRPC and REST implementations so that we can provide a grace period for customers. The REST implementation should use the canonical JSON representation of the generated protobuf structs in the HTTP body for arguments and responses.
We expect only to maintain both implementations for the 5.1.X
release in June. Afterward, we'll only use the gRPC API and can delete the redundant REST implementations.
simple example
The following example demonstrates how to implement a simple service in Go that provides both gRPC and REST APIs, using the canonical JSON representation of the generated Protobuf structs.
Notes:
- The Go service uses google.golang.org/protobuf/encoding/protojson to Marshal and Unmarshal Protobuf structs to/from JSON. The standard "encoding/json" package should not be used here: it doesn't correctly operate on protobuf structs.
- In this example, the gRPC and REST implementations share a helper function that does the actual work. This is not strictly required, but it's a good practice to follow (especially if the service is more complex than this example).
gRPC definition
syntax = "proto3"; package greeting; service GreeterService { rpc SayHello (HelloRequest) returns (HelloReply); } message HelloRequest { string name = 1; } message HelloReply { string message = 1; }
generate the Go protobuf structs
Create the following buf configuration file:
buf.gen.yaml
The buf configuration file generates the Go code for the Protobuf definition. This file specifies the plugins to use and the output directory for the generated code. The generated code includes the Protobuf structs we can reuse in gRPC and REST implementations.
# Configuration file for https://buf.build/, which we use for Protobuf code generation. version: v1 plugins: - plugin: buf.build/protocolbuffers/go:v1.29.1 out: . opt: - paths=source_relative - plugin: buf.build/grpc/go:v1.3.0 out: . opt: - paths=source_relative
Now, run sg generate buf
to use the above configuration file to generate the Go code for the protobuf definition
above. That command creates the following files:
greeter.pb.go
package greeter type HelloRequest struct { // ... Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` } type HelloReply struct { // ... Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` } // ... (omitted)
greeter_grpc.pb.go
package greeter import ( context "context" grpc "google.golang.org/grpc" ) type GreeterServiceClient interface { SayHello(ctx context.Context, in *HelloRequest, opts ...grpc.CallOption) (*HelloReply, error) } // ... (omitted)
go service implementation
package main import ( "context" "fmt" "io" "log" "net" "net/http" "github.com/gorilla/mux" "google.golang.org/grpc" "google.golang.org/grpc/codes" "google.golang.org/grpc/status" "example.com/greeting" // 🚨🚨🚨 note the use of this package instead of "encoding/json"! // "encoding/json" doesn't correctly serialize protobuf structs "google.golang.org/protobuf/encoding/protojson" ) type server struct { greeting.UnimplementedGreeterServiceServer } func (s *server) SayHello(ctx context.Context, in *greeting.HelloRequest) (*greeting.HelloReply, error) { reply, err := getReply(ctx, in.GetName()) if err != nil { return nil, err } return &greeting.HelloReply{Message: reply}, nil } func main() { // Start gRPC server lis, err := net.Listen("tcp", ":50051") if err != nil { log.Fatalf("failed to listen: %v", err) } grpcServer := grpc.NewServer() greeting.RegisterGreeterServiceServer(grpcServer, &server{}) go func() { if err := grpcServer.Serve(lis); err != nil { log.Fatalf("failed to serve: %v", err) } }() // Start REST server r := mux.NewRouter() r.HandleFunc("/sayhello", sayHelloREST).Methods("POST") http.ListenAndServe(":8080", r) } func sayHelloREST(w http.ResponseWriter, r *http.Request) { // First, grab the arguments from the request body body, err := io.ReadAll(r.Body) if err != nil { http.Error(w, fmt.Sprintf("reading request json body: %s", err.Error()), http.StatusInternalServerError) } defer r.Body.Close() var req greeting.HelloRequest err = protojson.Unmarshal(body, &req) if err != nil { http.Error(w, "invalid request", http.StatusBadRequest) return } // Next, get the reply from the shared helper function reply, err := getReply(r.Context(), req.GetName()) if err != nil { code, message := convertGRPCErrorToHTTPStatus(err) http.Error(w, message, code) return } // Finally, prepare the response and send it resp := &greeting.HelloReply{Message: reply} jsonBytes, err := protojson.Marshal(resp) if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return } w.Header().Set("Content-Type", "application/json") w.WriteHeader(http.StatusOK) w.Write(jsonBytes) } // getReply is a helper function that we can reuse in both the gRPC and REST APIs // so that we don't have to duplicate the implementation logic. func getReply(_ context.Context, name string) (message string, err error) { if name == "" { return "", status.Error(codes.InvalidArgument, "name was not provided") } return fmt.Sprintf("Hello, %s!", name), nil } // convertGRPCErrorToHTTPStatus translates gRPC error codes to HTTP status codes. See // https://chromium.googlesource.com/external/github.com/grpc/grpc/+/refs/tags/v1.21.4-pre1/doc/statuscodes.md // for more information. func convertGRPCErrorToHTTPStatus(err error) (httpCode int, errorText string) { s, ok := status.FromError(err) if !ok { return http.StatusInternalServerError, err.Error() } switch s.Code() { case codes.InvalidArgument: return http.StatusBadRequest, s.Message() default: return http.StatusInternalServerError, s.Message() } }
As you can see, this service reuses the generated protobuf structs in both the gRPC and REST APIs.
It also extracts the core implementation logic into a shared helper function, getReply
, that can be reused in both interfaces. This:
- reduces code duplication (reducing the chance of drift in either implementation)
- makes testing easier (we only need to test
getReply
once) - limits the scope of what the gRPC and REST functions are doing (only deserializing the requests and serializing the responses)