Shulou(Shulou.com)06/03 Report--

Course objectives:

1) help people to have a certain understanding of Nginx

2) be familiar with the application scenarios of Nginx

3) be familiar with Nginx characteristics, architecture model and related processes

4) familiar with several module classifications of customized development of Nginx

Course outline: introduction and characteristics of Nginx Nginx application scenario Nginx framework model introduction to Nginx internal flow introduction to Nginx custom module development introduction to Nginx core time point module introduction to Nginx dynamic upstream module introduction to Nginx query_upstrem module introduction to Nginx query_conf module introduction to Nginx shared memory support redis protocol module introduction to Nginx log playback pressure testing tools introduction 1. Nginx introduction and features

Introduction to Nginx:

Nginx (engine x) is a high-performance web server and reverse proxy server, as well as an IMAP/POP3/SMTP server

Russian programmer Igor Sysoev started Nginx as the fastest growing Web server in 2002, and its market share has reached 33.3%. It ranks second in global usage and established a commercial company in 2011.

Nginx Community Branch:

Developed by Openresty author @ agentzh (Zhang Yichun), the biggest feature is the introduction of the ngx_lua module, support for the use of lua development plug-ins, and a collection of many rich modules, as well as lua libraries. Tengine is mainly developed by Taobao team. It is characterized by the integration of new functions brought about by some of Taobao's own business. Nginx official version, update iteration is faster, and free version and commercial version are available.

Nginx source code structure:

The code volume is about 110000 lines C code source code directory structure core (backbone and basic settings) event (event-driven model and different IO reuse modules) http (HTTP servers and modules) mail (mail proxy servers and modules) os (operating system related implementations) misc (miscellaneous)

Nginx features:

Reverse proxy, load balancer with high reliability, single master and multi-worker mode, high scalability, highly modular, non-blocking event-driven low memory consumption hot deployment 2. Nginx application scenario

The scene is as follows:

Static file server reverse proxy, load balancing security defense intelligent routing (enterprise grayscale test, map POI one-click cut stream) grayscale release static message push picture real-time compression hotlink protection 3. Nginx framework model introduction

Process component roles:

The master process monitors the status of the worker process when the worker process dies and restarts a new processing signal and notifies the worker process that the worker process processes the client request to get a signal from the main process to do the corresponding thing. The cache loader process loads the cache index file information, and then exits the cache size of the cache manager process management disk. If the minimum usage data exceeds the predetermined size, the data will be deleted.

Frame model:

Framework model process:

4. Nginx Internal process introduction 4.1 Framework Model process

4.2 master initialization process

4.3 worker initialization

4.4 worker initialization proc

4.5 static file request IO process

4.6 http request process

4.7 11 phases of http request

4.8 upstream module access to third-party server underlying HTTP communication is very perfect asynchronous non-blocking upstream and downstream memory zero copy, memory saving support custom module development 4.8.1 upstream framework flow

4.8.2 upstream internal process

4.9 reverse proxy process

5. Nginx customized module development 5.1 Nginx modular design features highly abstract module interface module interface is very simple, with high flexibility to configure module design core module interface simplification of multi-level, multi-category module design 5.1 internal core module

The handler module accepts requests from the client and builds the response header and response body.

The filter module filter (filter) module is a module that filters response headers and content, which can be used to process reply headers and content. Its processing time is after the reply content is obtained and before the response is sent to the user.

Upstream module enables nginx to receive, process and forward network data beyond the limitation of a single machine, and access the back-end service asynchronously.

Load_balance:

Load balancing module, to achieve a specific algorithm, in many back-end servers, select a server as a request forwarding server.

5.3 ngx_lua module scripting language, low memory overhead, fast and powerful Lua protocol non-blocking business logic is written in natural logic

5.4 customized development Demo

Handler module:

Write config file and write module to generate content response information # configuration file: server {... Location test {test_counter on;}} # configngx_addon_name=ngx_http_test_moduleHTTP_MODULES= "$HTTP_MODULES ngx_http_test_module" NGX_ADDON_SRCS= "$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_test_module.c" # ngx_http_test_module.cstatic ngx_int_tngx_http_test_handler (ngx_http_request_t * r) {ngx_int_t rc Ngx_buf_t * b; ngx_chain_t out; ngx_http_test_conf_t * lrcf; ngx_str_t ngx_test_string = ngx_string ("hello test") Lrcf = ngx_http_get_module_loc_conf (r, ngx_http_test_module); if (lrcf- > test_counter = = 0) {return NGX_DECLINED;} / * we response to 'GET' and' HEAD' requests only * / if (! (r-> method & (NGX_HTTP_GET | NGX_HTTP_HEAD) {return NGX_HTTP_NOT_ALLOWED } / * discard request body, since we don't need it here * / rc = ngx_http_discard_request_body (r); if (rc! = NGX_OK) {return rc;} / * set the 'Content-type' header * / / * * r-> headers_out.content_type.len = sizeof ("text/html")-1 * r-> headers_out.content_type.data = (u_char *) "text/html"; * / ngx_str_set (& r-> headers_out.content_type, "text/html"); / * send the header only, if the request type is http 'HEAD' * / if (r-> method = = NGX_HTTP_HEAD) {r-> headers_out.status = NGX_HTTP_OK R-> headers_out.content_length_n = ngx_test_string.len; return ngx_http_send_header (r);} / * set the status line * / r-> headers_out.status = NGX_HTTP_OK; r-> headers_out.content_length_n = ngx_test_string.len; / * send the headers of your response * / rc = ngx_http_send_header (r) If (rc = = NGX_ERROR | | rc > NGX_OK | | r-> header_only) {return rc;} / * allocate a buffer for your response body * / b = ngx_pcalloc (r-> pool, sizeof (ngx_buf_t)); if (b = = NULL) {return NGX_HTTP_INTERNAL_SERVER_ERROR;} / * attach this buffer to the buffer chain * / out.buf = b; out.next = NULL / * adjust the pointers of the buffer * / b-> pos = ngx_test_string.data; b-> last = ngx_test_string.data + ngx_test_string.len; b-> memory = 1; / * this buffer is in memory * / b-> last_buf = 1; / * this is the last buffer in the buffer chain * / / * send the buffer chain of your response * / return ngx_http_output_filter (r, & out) } 6. Introduction of Nginx core time point module

Solve the problem of slow fault location in the access layer, help OP quickly determine the root cause of the problem, give priority to self-certification, and improve the efficient productivity of the access layer.

7. Introduction of Nginx shunt module

Features:

Implement a very flexible and dynamic modification strategy to cut traffic.

Achieve smooth and lossless way to switch traffic.

By switching traffic in seconds, the scope of influence can be reduced, thus reducing losses.

Switch or disable traffic in seconds according to a city or a characteristic.

Tolerate single-room-level capacity failures and shorten the stop-loss time of single-room failures.

Quickly isolate or sample traffic.

Efficient grayscale testing to improve productivity.

8. Introduction of Nginx dynamic upstream module

Let the access layer adapt to the cloud environment of dynamic scheduling, and achieve smooth online and elastic expansion / reduction of services.

So as to improve the efficient productivity and stability of the access layer, and ensure the smooth and lossless business traffic.

9. Introduction to Nginx query_upstream module

Link tracking, combing the interface to the back-end link. Query the upstream server information corresponding to the API location.

10. Introduction to Nginx query_conf module

Gets the nginx configuration file formatted as json format information.

Introduction of 11.Nginx shared memory support redis protocol module

Add shared memory dynamically according to the configuration file.

Https://github.com/lidaohang/ngx_shm_dict

Ngx_shm_dict

Shared memory core module (red-black tree, queue)

Ngx_shm_dict_manager

Add timer events to regularly clear expired key in shared memory

Add read events, support redis protocol, through redis-cli get,set,del,ttl

Ngx_shm_dict_view

Shared memory View

12. Nginx log playback pressure test tool parses the log for playback pressure test to simulate various abnormal situations such as slow back-end server

The https://github.com/lidaohang/playback-testing scheme states that the client parses the host,port,url,body of the access.log build request and puts the back-end response time, the back-end response status code and the back-end response size into the header header. The back-end server obtains the corresponding header, simulates the response body size, response status code, and response time to copy the log of the access.log that needs to be tested to the logs folder to build the nginx server to test. And configure upstream to point to the backend server to break the port to start the backend server instance server/backserver/main.go for stress testing bin/wrk-c30-T1-s conf/nginx_log.lua http://localhost:8095 author: Li Hang

If you register Didi Yun now, you can get a 30-yuan Didi coupon without a threshold.

50% discount for newly purchased cloud services in January, 4.5% discount in March and 60% discount in June.

Didi Yun messenger recruits, recommending up to 50% rebate

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.