Shulou(Shulou.com)06/02 Report--

background

Use the specific device for deep learning model reasoning, the machine only provides C++ packaged API for model loading and reasoning, the model training is still using caffe, the model needs to be converted to the format supported by the device, the model conversion is not introduced here. In order to make the model reasoning into a service, you can only use C++ to build an HTTP service, so that the user posts a picture through the http service, the server starts the model reasoning, realizes the prediction of the model, and returns the result to the client.

overall framework

The short service content is to preprocess the received images, and then conduct model reasoning. One of the points that needs to be done at present is to introduce HTTP services.

preliminary research

For a C++ novice, the preliminary research is of course the first search, http server c++ search processing results are also varied, some teach you how to implement an http server, some are a third-party library, some are directly a bunch of code. I saw it on stackoverflow:

why

not try NGINX with fcgi-function mapping?

implementation steps

Nginx is a proxy artifact. It is often used when doing Load Balancer. As long as my client content is sent to nginx, nginx will forward the data to fcgi-related applications. What I need to do is to combine fcgi with my inference program.

nginx

Simply put, nginx is a middleman. The client sends the request to the middleman, and the middleman goes to the source to get the goods. After that, he responds to the customer:

The customer tells nginx that I want to buy ** goods, nginx goes to the service provider for the service and returns it to the customer.

What is needed now is to implement the FCGI part, so what is FCGI?

cgi

Common Gateway Interface (CGI) is an important Internet technology that allows a client to request data from a web browser to a program executing on a web server. CGI describes a standard for transferring data between a server and a request handler.

The standard input and output here are some corresponding environmental variables, which mainly include request-related environmental variables, server-related environmental variables, and client-related environmental variables.

fastcgi

FastCGI is actually CGI with some extensions, an improvement on CGI, and a standard for describing the transfer of data between client and Web server programs.

FastCGI aims to reduce the overhead of interaction between Web servers and CGI programs so that Web servers can handle more Web requests simultaneously. Unlike CGI, which creates a new process for each Web request, FastCGI uses persistent processes to process a stream of Web requests, which are managed by the FastCGI process manager rather than the Web server.

Why reduce the cost of interaction? This depends on the difference between the two methods of treatment!

CGI workflow:

Every time the client makes a new request, it first creates a CGI child process, and then CGI processes the request, and how many CGI child processes are started as many connections as there are, which will consume a lot of system resources when the request volume is large.

fastcgi

fastcgi uses persistent processes to process a series of requests. These processes are managed by fastcgi's process manager. The specific process flow is as follows:

It can also be compared like this:

Cgi was selling egg crackers, and when the customer wanted to eat, he started lighting the fire, cracking the eggs, spreading the crackers, and then turning off the fire. and wait for the next customer.

Fastcgi is an old version of breakfast restaurant, employing a group of waiters to cook meals that need to be cooked on site. The boss only needs to arrange the order and the waiters are responsible for serving porridge pancakes.

concrete steps

Build c++ development environment Build nginx install fastcgi install fastcgi process manager spawn-cgi write running program compile run

To do a good job, we must first sharpen our tools. First, we must build an environment.

By reading a lot of blog content to find the easiest installation steps, many are by downloading the source code, and then compile through make, but for these more commonly used libraries, the package has been integrated.

C++ Development Environment Installation

apt-get install build-essential

nginx

apt-get install nginx

fastcgi

sudo apt-get install libfcgi-dev

spawn-fcgi

apt-get install spawn-fcgi

Write a running program

#include #include "fcgio.h" using namespace std; int main(void) { // Backup the stdio streambufs streambuf * cin_streambuf = cin.rdbuf(); streambuf * cout_streambuf = cout.rdbuf(); streambuf * cerr_streambuf = cerr.rdbuf(); FCGX_Request request; FCGX_Init(); FCGX_InitRequest(&request, 0, 0); while (FCGX_Accept_r(&request) == 0) { fcgi_streambuf cin_fcgi_streambuf(request.in); fcgi_streambuf cout_fcgi_streambuf(request.out); fcgi_streambuf cerr_fcgi_streambuf(request.err); cin.rdbuf(&cin_fcgi_streambuf); cout.rdbuf(&cout_fcgi_streambuf); cerr.rdbuf(&cerr_fcgi_streambuf); cout

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.