Machine-learning models are starting to creep into everyday business applications. When enough training data is available, these algorithms can address problems that might have previously required complex statistical models or heuristics. As we move from experimental use to production, we need a reliable way to host and deploy the models that can be accessed remotely and scale with the number of consumers. TensorFlow Serving addresses part of that problem by exposing a remote gRPC interface to an exported model; this allows a trained model to be deployed in a variety of ways. TensorFlow Serving also accepts a stream of models to incorporate continuous training updates, and its authors maintain a Dockerfile to ease the deployment process. Presumably, the choice of gRPC is to be consistent with the TensorFlow execution model; however, we’re generally wary of protocols that require code generation and native bindings.
Machine-learning models are starting to creep into everyday business applications. When enough training data is available, these algorithms can address problems that might have previously required complex statistical models or heuristics. As we move from experimental use to production, we need a reliable way to host and deploy the models that can be accessed remotely and scale with the number of consumers. TensorFlow Serving addresses part of that problem by exposing a remote gRPC interface to an exported model; this allows a trained model to be deployed in a variety of ways. TensorFlow Serving also accepts a stream of models to incorporate continuous training updates, and its authors maintain a Dockerfile to ease the deployment process. Presumably, the choice of gRPC is to be consistent with the TensorFlow execution model; however, we’re generally wary of protocols that require code generation and native bindings.
