Protobuf Implementation

The file gnes/proto/gnes.proto defines the protobuf used in GNES. It is the core message protocol used in communicating between services. It also defines the interface of a gGRPC service.

gnes_pb2.py and gnes_pb2_grpc.py are python interfaces automatically generated by protobuf tools.

For developers who want to change the protobuf definition, one needs to first edit gnes/proto/gnes.proto and then regenerate the python codes (i.e. gnes_pb2.py and gnes_pb2_grpc.py).

Generating gnes_pb2.py and gnes_pb2_grpc.py

Take MacOS as an example,

  1. Download protoc-$VERSION-$PLATFORM.zip from the official site and decompress it.
  2. Copy the binary file and include to your system path:
cp ~/Downloads/protoc-3.7.1-osx-x86_64/bin/protoc /usr/local/bin/

cp -r ~/Downloads/protoc-3.7.1-osx-x86_64/include/* /usr/local/include/
  1. Install gRPC tools dependencies: brew install automake autoconf libtool
  2. Install gRPC and grpc_python_plugin from the source:
git clone https://github.com/grpc/grpc.git
git submodule update --init
make grpc_python_plugin
  1. This will compile the grpc-python-plugin and build it to, e.g., /Documents/grpc/bins/opt/grpc_python_plugin
  2. Generate the python codes:
SRC_DIR=gnes/proto/
PLUGIN_PATH=/Documents/grpc/bins/opt/grpc_python_plugin

protoc -I $SRC_DIR --python_out=$SRC_DIR --grpc_python_out=$SRC_DIR --plugin=protoc-gen-grpc_python=${PLUGIN_PATH} ${SRC_DIR}gnes.proto
  1. Fixing the import in gnes_pb2_grpc.py. For some reason (probably a bug of gRPC?), the generated code of import is not correct in gnes_pb2_grpc.py, you have to change it to the following:
# Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT!
import grpc

from . import gnes_pb2 as gnes__pb2