Shulou(Shulou.com)06/03 Report--

This article mainly introduces "how to deploy TensorFlow Serving". In daily operation, I believe many people have doubts about how to deploy TensorFlow Serving. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the doubts about "how to deploy TensorFlow Serving"! Next, please follow the editor to study!

Prepare the environment

Prepare the TensorFlow environment and import dependencies:

Import sys# Confirm that we're using Python 3assert sys.version_info.major = = 3, 'Oops Not running Python 3. Use Runtime > Change runtime type'import tensorflow as tffrom tensorflow import keras# Helper librariesimport numpy as npimport matplotlib.pyplot as pltimport osimport subprocessprint (f'TensorFlow version: {tf.__version__}') print (f'TensorFlow GPU support: {tf.test.is_built_with_gpu_support ()}') physical_gpus = tf.config.list_physical_devices ('GPU') print (physical_gpus) for gpu in physical_gpus: # memory growth must be set before GPUs have been Initialized tf.config.experimental.set_memory_growth (gpu True) logical_gpus = tf.config.experimental.list_logical_devices ('GPU') print (len (physical_gpus), "Physical GPUs," len (logical_gpus), "Logical GPUs") TensorFlow version: 2.4.1TensorFlow GPU support: True [PhysicalDevice (name='/physical_device:GPU:0', device_type='GPU')] 1 Physical GPUs, 1 Logical GPUs create the model

Load the Fashion MNIST dataset:

Fashion_mnist = keras.datasets.fashion_mnist (train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data () # scale the values to 0.0 to 1.0train_images = train_images / 255.0test_images = test_images / 255.The reshape for feeding into the modeltrain_images = train_images.reshape (train_images.shape [0], 28,28,1) test_images = test_images.reshape (test_images.shape [0], 28,28 1) class_names = ['Trouser',' Pullover', 'Dress',' Coat', 'Sandal',' Shirt', 'Sneaker',' Bag', 'Ankle boot'] print ('\ ntrain_images.shape: {}, of {} '.format (train_images.shape, train_images.dtype)) print (' test_images.shape: {}, of {} '.format (test_images.shape) Test_images.dtype)) train_images.shape: (60000, 28, 28, 1), of float64test_images.shape: (10000, 28, 28, 1), of float64

Use the simplest CNN training model

Model = keras.Sequential ([keras.layers.Conv2D (input_shape= (28recover28 model.summary 1), filters=8, kernel_size=3, strides=2, activation='relu', name='Conv1'), keras.layers.Flatten (), keras.layers.Dense (10, name='Dense')]) model.summary () testing = Falseepochs = 5model.compile (optimizer='adam', loss=tf.keras.losses.SparseCategoricalCrossentropy (from_logits=True)) Metrics= [keras.metrics.SparseCategoricalAccuracy ()]) model.fit (train_images, train_labels, epochs=epochs) test_loss, test_acc = model.evaluate (test_images Test_labels) print ('\ nTest accuracy: {} '.format (test_acc)) Model: "sequential" _ Layer (type) Output Shape Param # = Conv1 (Conv2D) (None 13, 13, 8) 80_flatten (Flatten) (None 1352) 0_Dense (Dense) (None 10) 13530=Total params: 13610Trainable params: 13610Non-trainable params: 0_Epoch 1ax 51875 1875 [=]-3s 722us/step-loss: 0.7387-sparse_categorical_accuracy: 0.7449Epoch 2 / 51875 loss 1875 [=]-1s 793us/step-loss: 0.4561-sparse_categorical_accuracy: 0.8408Epoch 3Universe 51875 720us/step-loss: 0.4097-sparse_categorical_accuracy: 0.8566Epoch 4Compare 51875 718us/step-loss: 0.3899-sparse_categorical_accuracy: 0.8636Epoch 51875 719us/step-loss: 0.3673-sparse_categorical_accuracy: 0.8701313hand 313 [=]-0s 782us/step-loss: 0.3937-sparse_categorical_accuracy: 0.8630Test accuracy: 0.8629999756813049 Save the model

Save the model in SavedModel format with a version number in the path so that the model version can be selected when TensorFlow Serving.

# Fetch the Keras session and save the model# The signature definition is defined by the input and output tensors,# and stored with the default serving keyimport tempfileMODEL_DIR = os.path.join (tempfile.gettempdir (), 'tfx') version = 1export_path = os.path.join (MODEL_DIR, str (version)) print (' export_path = {}\ n'.format (export_path)) tf.keras.models.save_model (model, export_path, overwrite=True, include_optimizer=True, save_format=None, signatures=None Options=None) print ('\ nSaved model:')! ls-l {export_path} export_path = / tmp/tfx/1INFO:tensorflow:Assets written to: / tmp/tfx/1/assetsSaved model:total 88drwxr-xr-x 2 john john 4096 Apr 13 15:10 assets-rw-rw-r-- 1 john john 78169 Apr 13 15:12 saved_model.pbdrwxr-xr-x 2 john john 4096 Apr 13 15:12 variables View the model

Use the saved_model_cli tool to view the model's MetaGraphDefs (the models) and SignatureDefs (the methods you can call) for information.

! saved_model_cli show-- dir'/ tmp/tfx/1'-- all2021-04-13 1515 all2021 12 all2021 29.433576: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:signature_ defe [' _ saved_model_init_op']: The given SavedModel SignatureDef contains the following input (s): The given SavedModel SignatureDef contains the following output (s) : outputs ['_ saved_model_init_op'] tensor_info: dtype: DT_INVALID shape: unknown_rank name: NoOp Method name is:signature_def ['serving_default']: The given SavedModel SignatureDef contains the following input (s): inputs [' Conv1_input'] tensor_info: dtype: DT_FLOAT shape: (- 1 28, 28, 1) name: serving_default_Conv1_input:0 The given SavedModel SignatureDef contains the following output (s): outputs ['Dense'] tensor_info: dtype: DT_FLOAT shape: (- 1,10) name: StatefulPartitionedCall:0 Method name is: tensorflow/serving/predictDefined Functions: Function Name:' _ call__' Option # 1 Callable with: Argument # 1 Conv1_input: TensorSpec (None, 28 28, 1), dtype=tf.float32, name='Conv1_input') Argument # 2 DType: bool Value: False Argument # 3 DType: NoneType Value: None Option # 2 Callable with: Argument # 1 inputs: TensorSpec (shape= (None, 28,28,1), dtype=tf.float32 Name='inputs') Argument # 2 DType: bool Value: False Argument # 3 DType: NoneType Value: None Option # 3 Callable with: Argument # 1 inputs: TensorSpec (shape= (None, 28,28,1), dtype=tf.float32 Name='inputs') Argument # 2 DType: bool Value: True Argument # 3 DType: NoneType Value: None Option # 4 Callable with: Argument # 1 Conv1_input: TensorSpec (shape= (None, 28,28,1), dtype=tf.float32 Name='Conv1_input') Argument # 2 DType: bool Value: True Argument # 3 DType: NoneType Value: None. Deployment model installs Servingecho "deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | sudo tee / etc/apt/sources.list.d/tensorflow-serving.list & &\ curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | sudo apt-key add-sudo apt updatesudo apt install tensorflow-model-server enables Serving

Enable TensorFlow Serving and provide REST API:

Rest_api_port: REST request port.

Model_name: REST request URL, custom name.

Model_base_path: the directory where the model is located.

Nohup tensorflow_model_server\-rest_api_port=8501\-model_name=fashion_model\-model_base_path= "/ tmp/tfx" > server.log 2 > & 1 & $tail server.logTo enable them in other operations Rebuild TensorFlow with the appropriate compiler flags.2021-04-13 15 external/org_tensorflow/tensorflow/core/platform/profile_utils/cpu_utils.cc:112 12 Restoring SavedModel bundle.2021 10.706648: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:206] Restoring SavedModel bundle.2021-04-13 15 15 Restoring SavedModel bundle.2021 10.726722: I external/org_tensorflow/tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2599990000 Hz2021-04-13 15 external/org_tensorflow/tensorflow/core/platform/profile_utils/cpu_utils.cc:112 12 Restoring SavedModel bundle.2021 10.756506: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc: 1900] Running initialization op on SavedModel bundle at path: / tmp/tfx/12021-04-13 1515 external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277 12buret 10.759935: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags {serve} Status: success: OK. Took 110653 microseconds.2021-04-13 15 tensorflow_serving/servables/tensorflow/saved_model_warmup_util.cc:59 12 tensorflow_serving/servables/tensorflow/saved_model_warmup_util.cc:59 10.760277: I] No warmup data file found at / tmp/tfx/1/assets.extra/tf_serving_warmup_requests2021-04-13 15 tmp/tfx/1/assets.extra/tf_serving_warmup_requests2021 12 No warmup data file found at 10.760486: I tensorflow_serving/core/loader_harness.cc:87] Successfully loaded servable version {name: fashion_model version: 1} 2021-04-13 1515 No warmup data file found at 10.763938: I tensorflow_serving / model_servers/server.cc:371] Running gRPC ModelServer at 0.0.0.0 NET_LOG 8500... [evhttp_server.cc: 238] NET_LOG: Entering the event loop. 2021-04-13 15 Entering the event loop 12 evhttp_server.cc 10.765308: I tensorflow_serving/model_servers/server.cc:391] Exporting HTTP/REST API at:localhost:8501. Access servic

Randomly display a test chart:

Def show (idx, title): plt.figure () plt.imshow (test_ images [IDX] .reshape (28)) plt.axis ('off') plt.title ('\ n\ n {} '.format (title), fontdict= {' size': 16}) import randomrando = random.randint (0dlen (test_images)-1) show (rando,'An Example Image: {} '.format (class_ namespace [test _ labelsrando]))

Create a JSON object and give you three diagrams to predict:

Import jsondata = json.dumps ({"signature_name": "serving_default", "instances": test_images [0:3] .tolist ()}) print ('Data: {}. {}' .format (data [: 50], data [len (data)-52:]) Data: {"signature_name": "serving_default", "instances":. [0.0], [0.0], [0.0], [0.0] [0.0], [0.0], [0.0]} REST request

Forecast the latest version of the model:

! pip install-q requestsimport requestsheaders = {"content-type": "application/json"} json_response = requests.post ('http://localhost:8501/v1/models/fashion_model:predict', data=data, headers=headers) predictions = json.loads (json_response.text) [' predictions'] show (0, 'The model thought this was a {} (class {}), and it was actually a {} (class {})' .format (class_ [namesnp.argmax (predictions [0])] Np.argmax (predictions [0]), class_ namespace [test _ labels [0]], test_ labels [0]))

Specify the version of the model for prediction:

Headers= {"content-type": "application/json"} json_response = requests.post ('http://localhost:8501/v1/models/fashion_model/versions/1:predict', data=data, headers=headers) predictions = json.loads (json_response.text) [' predictions'] for i in range (0L3): show (I, 'The model thought this was a {} (class {})) And it was actually a {} (class {}) '.format (class_ namesnp.argmax (substitutions [I]), np.argmax (substitutions [I]), class_ namespace [test_ labels [I]])

At this point, the study on "how to deploy TensorFlow Serving" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.