Shulou(Shulou.com)06/01 Report--

This article shares with you the content of a sample analysis of the nginx request header data reading process. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

1. Request header read main flow

Before introducing the request header reading process, let's first show an example of a http request message:

POST / web/book/read HTTP/1.1Host: localhostConnection: keep-aliveContent-Length: 365Accept: application/json, text/plain, * / *

The first row of data in the example is the request row, and the next few lines are the request headers. Each request header is assembled in the format name: value, and each request header occupies a line. In the previous article on the request row reading process, we talked about that once the request row read is complete, nginx will change the callback function of the current read event to the ngx_http_process_request_headers () method and directly call this method to attempt to read the request header data. This method is the main process of reading request line data. The following is the source code of this method:

/ * parse the header data sent by the client * / static void ngx_http_process_request_headers (ngx_event_t * rev) {u_char * p; size_t len; ssize_t n; ngx_int_t rc, rv; ngx_table_elt_t * h; ngx_connection_t * c; ngx_http_header_t * hh; ngx_http_request_t * r; ngx_http_core_srv_conf_t * cscf Ngx_http_core_main_conf_t * cmcf; c = rev- > data; r = c-> data; if (rev- > timedout) {ngx_log_error (NGX_LOG_INFO, c-> log, NGX_ETIMEDOUT, "client timedout"); c-> timedout = 1; ngx_http_close_request (r, NGX_HTTP_REQUEST_TIME_OUT); return;} cmcf = ngx_http_get_module_main_conf (r, ngx_http_core_module); rc = NGX_AGAIN; for ( ) {if (rc = = NGX_AGAIN) {/ / if there is no space left in the current header buffer, apply for a new space if (r-> header_in- > pos = = r-> header_in- > end) {/ / apply for a new space rv = ngx_http_alloc_large_header_buffer (r, 0); if (rv = = NGX_ERROR) {ngx_http_close_request (r, NGX_HTTP_INTERNAL_SERVER_ERROR) Return;} / / the header sent by the client is too long, exceeding the maximum size specified by large_client_header_buffers if (rv = = NGX_DECLINED) {p = r-> header_name_start; r-> lingering_close = 1; if (p = = NULL) {ngx_log_error (NGX_LOG_INFO, c-> log, 0, "client sent too large request") Ngx_http_finalize_request (r, NGX_HTTP_REQUEST_HEADER_TOO_LARGE); return;} len = r-> header_in- > end-p; if (len > NGX_MAX_ERROR_STR-300) {len = NGX_MAX_ERROR_STR-300;} ngx_http_finalize_request (r, NGX_HTTP_REQUEST_HEADER_TOO_LARGE); return }} / / attempt to read the new data sent by the client on the connection n = ngx_http_read_request_header (r); if (n = = NGX_AGAIN | | n = = NGX_ERROR) {return;}} cscf = ngx_http_get_module_srv_conf (r, ngx_http_core_module) / / this is mainly to convert the read data rc = ngx_http_parse_header_line (r, r-> header_in, cscf- > underscores_in_headers); / / NGX_OK indicates that a header data if (rc = = NGX_OK) {r-> request_length + = r-> header_in- > pos-r-> header_name_start has been successfully parsed. / / filter invalid header if (r-> invalid_header & & cscf- > ignore_invalid_headers) {continue;} / / create a structure to store header h = ngx_list_push (& r-> headers_in.headers); if (h = = NULL) {ngx_http_close_request (r, NGX_HTTP_INTERNAL_SERVER_ERROR); return;} h-> hash = r-> header_hash / take the name of header as the key h-> key.len = r-> header_name_end-r-> header_name_start; h-> key.data = r-> header_name_start; h-> key.data [h-> key.len] ='\ 0operations; / / take the value of header as the value h-> value.len = r-> header_end-r-> header_start; h-> value.data = r-> header_start of hash table H-> value.data [h-> value.len] ='\ 0values; h-> lowcase_key = ngx_pnalloc (r-> pool, h-> key.len); if (h-> lowcase_key = = NULL) {ngx_http_close_request (r, NGX_HTTP_INTERNAL_SERVER_ERROR); return;} if (h-> key.len = = r-> lowcase_index) {ngx_memcpy (h-> lowcase_key, r-> lowcase_header, h-> key.len) } else {ngx_strlow (h-> lowcase_key, h-> key.data, h-> key.len);} / / all header are stored in the headers_in_hash. This is to find out whether the header passed by the client is a valid header hh = ngx_hash_find (& cmcf- > headers_in_hash, h-> hash, h-> lowcase_key, h-> key.len). / / the handler here is the processing method defined for each header in ngx_http_headers_in. After being processed by the / / handler () method of each header, the header from the client is converted to each attribute in the r-> headers_in structure and if (hh & & hh- > handler (r, h, hh- > offset)! = NGX_OK) {return;} continue } / / NGX_HTTP_PARSE_HEADER_DONE indicates that all header have been processed if (rc = = NGX_HTTP_PARSE_HEADER_DONE) {r-> request_length + = r-> header_in- > pos-r-> header_name_start; r-> http_state = NGX_HTTP_PROCESS_REQUEST_STATE; / / check the validity of header data sent by the client rc = ngx_http_process_request_header (r) If (rc! = NGX_OK) {return;} ngx_http_process_request (r); return;} / / NGX_AGAIN indicates that the read header row data is incomplete, and you need to continue reading if (rc = = NGX_AGAIN) {continue;} ngx_log_error (NGX_LOG_INFO, c-> log, 0, "client sent invalid header line"); ngx_http_finalize_request (r, NGX_HTTP_BAD_REQUEST); return }}

Here, the reading of the request header is divided into the following steps:

First check whether the current read event has timed out, and if so, close the current connection directly

Determine whether r-> header_in- > pos = = r-> header_in- > end is established. This is mainly to check whether there is memory space in the current read buffer to store the newly read data. If not, apply for a new memory space from the memory pool.

Call the ngx_http_read_request_header () method to read the data on the current connection handle. If the returned value is greater than 0, it indicates the length of the data read. If it is equal to 0, the client has broken the connection. If it is NGX_ERROR, it means that an exception has occurred in the read. If it is NGX_AGAIN, no data has been read this time and new data needs to be read on. As you can see, the first thing here is to determine whether the return value is NGX_AGAIN, and return it directly without doing any other processing, mainly because the callback function of the current read event or ngx_http_process_request_headers (), when a new read event is triggered, it will still call ngx_http_read_request_header () to read the data again. On the other hand, in the ngx_http_read_request_header () method, if the return value is NGX_AGAIN, it adds the current read event to the event queue again and registers the read event on the epoll handle for the current connection

Call the ngx_http_parse_header_line () method to parse the read request header data. It is important to note that only one request header is parsed each time the method is called, but after an infinite for loop and a constant event trigger mechanism, all the request header data will eventually be read.

According to the return value of the ngx_http_parse_header_line () method, if NGX_OK, the newly read header is stored in the r-> headers_in.headers linked list

If the return value of the ngx_http_parse_header_line () method is NGX_HTTP_PARSE_HEADER_DONE, it means that all the header have been read successfully. At this time, the ngx_http_process_request_header () method will be called first to check the validity of the read header, and then the ngx_http_process_request () method will be called to start the 11 stages of the http module in the nginx. The implementation principle of this method will be explained in a later article.

two。 Request header data reading

As you can see, there are two main methods for reading request headers: ngx_http_read_request_header () and ngx_http_parse_header_line (). The second method here is relatively long, but its logic is very simple. It mainly parses whether the read data can form a complete request header (in the form of name: value, and occupies a row). If so, it returns NGX_OK, otherwise it returns NGX_AGAIN to expect to continue reading the data.

Static ssize_t ngx_http_read_request_header (ngx_http_request_t * r) {ssize_t n; ngx_event_t * rev; ngx_connection_t * c; ngx_http_core_srv_conf_t * cscf; c = r-> connection; rev = c-> read; / / calculate how much data remains unprocessed n = r-> header_in- > last-r-> header_in- > pos / / if n is greater than 0, it means that the read data is still unprocessed, then n if (n > 0) {return n is returned directly. } / / this indicates that the data currently read has been processed, so it will be judged here. If the ready parameter of the current event is 1, / / it means that the unread data is stored on the handle of the current connection, so call the c-> recv () method to read the data, otherwise continue to add the current event to the / / event queue. And continue to listen for the current connection handle read event if (rev- > ready) {/ / read data n = c-> recv on the connection file descriptor (c, r-> header_in- > last, r-> header_in- > end-r-> header_in- > last) } else {n = NGX_AGAIN;} / / if n is NGX_AGAIN, add the current event to the event listener and continue listening to the read event if of the current epoll handle (n = = NGX_AGAIN) {if (! rev- > timer_set) {cscf = ngx_http_get_module_srv_conf (r, ngx_http_core_module); ngx_add_timer (rev, cscf- > client_header_timeout) } if (ngx_handle_read_event (rev, 0)! = NGX_OK) {ngx_http_close_request (r, NGX_HTTP_INTERNAL_SERVER_ERROR); return NGX_ERROR;} return NGX_AGAIN;} / / if n is 0, the client closes the connection if (n = 0) {ngx_log_error (NGX_LOG_INFO, c-> log, 0, "client prematurely closed connection") } / / if the client closes the connection or reads an exception, the current request structure if (n = 0 | | n = = NGX_ERROR) {c-> error = 1; c-> log- > action = "reading client request headers"; ngx_http_finalize_request (r, NGX_HTTP_BAD_REQUEST); return NGX_ERROR;} / / updates the currently read data pointer r-> header_in- > last + = n; return n;}

Here, the reading of request header data is mainly divided into the following steps:

Determines whether there is any unprocessed data in the current buffer, and if so, returns directly. The main reason for the existence of unread data is that some or all of the request header data may be read during the previous process of reading the request row data, so a check will be made here.

Determine whether the current read event is ready, and if so, call the c-> recv () method to read the data on the current connection handle

If the current read event is not ready, add the current read event to the event queue again and register the read event on the epoll handle for the current connection

Judge the return value of step 2. If 0, the client has been disconnected. If it is NGX_ERROR, the data is read abnormally. In both cases, the current connection will be closed and the status code will be returned to the client. If the value NGX_AGAIN is returned, follow the steps in step 3 to continue listening for read events. If the return value is greater than 0, the read is successful, and the value greater than 0 indicates the length of the data read.

Updates the pointer data of the buffer that stores the read data.

Thank you for reading! This is the end of the article on "sample analysis of nginx request header data reading process". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it out for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.