Shulou(Shulou.com)06/02 Report--

This article focuses on "how python uses Cortex to deploy the PyTorch model to production". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how python uses Cortex to deploy the PyTorch model into production.

Guide reading

Using Cortex, you can easily deploy the PyTorch model.

Using PyTorch Models in Production with Cortex

Caleb Kaiser

Https://medium.com/pytorch/how-to-build-production-software-with-pytorch-9a8725382f2a

This is the year when PyTorch has become the most popular machine learning (ML) framework for researchers.

The python style of the framework, the gentleness of its learning curve, and its convenient implementation of fast and simple prototypes make PyTorch obviously a favorite of researchers. So it is promoting some of the coolest machine learning programs:

The popular natural language processing (NLP) library generated by Transformers,Hugging Face is built on PyTorch.

Selene, the biological frontier ML library, is built on PyTorch.

CrypTen, a popular, new, privacy-focused machine learning framework, is based on PyTorch.

In almost any field of ML, from computer vision to NLP to computational biology, you will find PyTorch at the forefront of providing energy for experiments.

The most natural problem, however, is how to incorporate these experiments into the software. How to convert from a "cross-language language model" to Google translation?

In this blog post, we will learn what it means to use the PyTorch model in a production environment, and then introduce a way to allow any PyTorch model to be deployed for use in software.

What does it mean to use PyTorch in production?

Running machine learning in a production environment may mean different things, depending on the production environment. In general, there are two types of machine learning design patterns in production:

Provide a predictive API through the inference server. This is the standard method used in general software development, that is, not mobile software or stand-alone devices. Embedded. Embed your model directly into your application. This is commonly used for robots and stand-alone devices, and sometimes for mobile applications.

If you plan to embed your model directly into your application, you should take a look at PyTorch's TorchScript. Using just-in-time compilation, PyTorch can compile Python into TorchScript that can be run without a Python interpreter, which is useful for resource-constrained deployment targets such as mobile devices.

In most cases, you will use the model server. Many of the ML applications you see today-from the recommendation engine behind your favorite streaming service to the autocomplete function in the online search bar-depend on this form of deployment, or rather, real-time reasoning.

In real-time reasoning, a model is usually deployed as a microservice (usually a JSON API) through which software can query the model and receive predictions.

Let's take RoBERTa of Facebook artificial intelligence as an example, a leading NLP model. It makes inferences by analyzing sentences that remove a word (or "shielding word") and guessing what the shielding word is. For example, if you want to use a pre-trained RoBERTa model to guess the next word in a sentence, the Python method you want to use is very simple:

Roberta.fill_mask (input_text + "")

It turns out that predicting missing words in a sequence is the function behind functions such as autocomplete. To implement autocomplete in your application, you can deploy RoBERTa as JSON API, and then query on the RoBERTa node using the user's input in the application.

Setting up JSON API sounds fairly simple, but deploying the model as a microservice actually requires a lot of infrastructure work.

You need to automatically control the fluctuation of the flow. You need to monitor your predictions. You need to deal with model updates. You need to know the log records. A lot of work.

The question, then, is how to deploy RoBERTa as a JSON API without manually scrolling through all of these custom infrastructures?

Putting PyTorch model into production together with Cortex

You can automate most of the infrastructure work required to deploy the PyTorch model using Cortex, an open source tool for deploying the model as an api on AWS. This article is not a complete guide to using Cortex, but a high-level way to use Cortex. What you need is:

Provide inferred Python scripts to define your API configuration file Cortex CLI to start your deployment

This approach is not limited to RoBERTa. Want to automatically generate alt text for your images to make your website easier to access? You can deploy an AlexNet model and use PyTorch and Cortex to mark the image.

What about language classifiers, such as Chrome, which are used to detect that pages are not written in the default language? FastText is the perfect model for this task, and you can deploy it using PyTorch and Cortex.

With Cortex, you can add many ML features supported by PyTorch to your application for real-time inference.

PyTorch is used in production

More than 25 research models are stored in PyTorch Hub], from NLP to computer vision. All of this can be done through Cortex, using the same process as we just demonstrated.

There is no doubt that the PyTorch team has more production-centric features on their roadmap, but just looking at the progress made so far, it is clear that the idea that PyTorch is not a framework built for production is out of date.

At this point, I believe you have a deeper understanding of "how python uses Cortex to deploy the PyTorch model to production". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.