Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to understand the Nginx event handling module". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to understand the Nginx event handling module.

I. explanation of the main flow of the event module

The event module mainly deals with two types of events: ① timing tasks, ② Imax O events. Scheduled task refers to the event added by nginx through ngx_add_timer. The prototype function is as follows:

Ngx_event_timer.hstatic ngx_inline voidngx_event_add_timer (ngx_event_t * ev, ngx_msec_t timer) {/ / calculate expiration time point / / expiration time = current time + expiration time key = ngx_current_msec + timer; / / set timeout to ev- > timer.key = key / / added red-black tree ngx_rbtree_insert (& ngx_event_timer_rbtree, & ev- > timer);} / / and set global variables to facilitate calling # define ngx_add_timer ngx_event_add_timer anywhere

Here is a list of timeout management covered in this article:

When the ① worker process scrambles for mutexes, because only one process can acquire the lock, and if other processes cannot acquire the lock, it is impossible to wait all the time, so setting accept_mutex_delay will cause timeout processing beyond the set event, and will give up the option of scrambling for lock 2 to release resources.

Ngx_event_accept.cvoidngx_event_accept (ngx_event_t * ev) {if (ngx_use_accept_mutex) {/ / xxx} else {ngx_add_timer (ev, ecf- > accept_mutex_delay);}}

The second is the IWeiO event, which is easy to understand that it deals with read and write events. Taking epoll as an example, let's take a look at the main flow of nginx handling events:

Ngx_process_cycle.cstatic voidngx_worker_process_cycle (ngx_cycle_t * cycle, void * data) {for (;;) {/ / continuously loop event ngx_process_events_and_timers (cycle);}}

The entry of event handling is in the code of the nginx process model, and the worker process will always loop through the processing event, in which the blocking point is epoll_wait, and then cycle the process after processing a round of events until the master process sends a signal to shut down. Here's where you really start dealing with events:

/ / start processing event voidngx_process_events_and_timers (ngx_cycle_t * cycle) {ngx_uint_t flags; ngx_msec_t timer, delta / / ngx_timer_resolution is used to determine which timeout policy to use / / if ngx_timer_resolution is non-zero, then set timer is not infinite / / in addition, ngx_timer_resolution has the function of controlling the frequency of gettimeofday calls, but the effect can be ignored in x86 / 64 system. / / the timing scheme is used at this time. At the specified time, the default is 500ms. Scan the elements in the red-black tree once and deal with the timeout nodes if (ngx_timer_resolution) {/ / first set the time to infinitely wait, and then set not 500 timer = NGX_TIMER_INFINITE Flags = 0;} else {/ / if the ngx_timer_resolution is 0 / / if the timeout scheme is adopted, first calculate the fastest timeout time, then wait for this time period to process the timeout event, / / finish the timeout task, calculate the next timeout time again, and process it continuously. / / query the red-black tree and return the node with the least timeout, that is, the leftmost element / * while (node- > left! = sentinel) {node = node- > left } * / nginx maintains all time nodes through the red-black tree / / sets the timeout detection time to the difference between the timeout time and the current time of the event object that does not have the fastest timeout / / timer = (node- > key-ngx_current_msec); timer = ngx_event_find_timer (); flags = NGX_UPDATE_TIME # if (NGX_WIN32) / * handle signals from master in case of network inactivity * / if the timer is infinite or greater than 500, the setting will be set to 500ms if (timer = = NGX_TIMER_INFINITE | | timer > 500) {timer = 500 if the timing scheme is selected } # endif} / / handle the swarm phenomenon (nginx is a master with multiple work contention requests) / / it means that multiple threads / processes listen for a socket event at the same time (nginx is a multi-process program that shares port 80), / / when the event occurs, it wakes up all waiting threads / processes Contention event / / but in the end, only one thread / process can read the event successfully / / other processes / threads wait again or other operations after a failed scramble. This waste of resources is called surprise / / set to solve the panic phenomenon through Accept mutexes / / when the number of worker in the nginx configuration file is greater than 1 / / and the configuration file opens accept_mutex Ngx_use_accept_mutex will be set to 1 / * * nginx to configure events {accept_mutex on # set network connection serialization to prevent panic. Default is on multi_accept on; # set whether a process accepts multiple network connections at the same time, default is off # use epoll; # event-driven model, select | poll | kqueue | epoll | resig | / dev/poll | eventport worker_connections 1024 # maximum number of connections. Default is 512} / if (ngx_use_accept_mutex) {/ / ngx_accept_disabled = ngx_cycle- > connection_n / 8-ngx_cycle- > free_connection_n / / where connection_n represents the maximum number of connections allowed in the current worker process, and free_connection_n represents the current number of idle connections / / assuming that the current number of connections is x, then ngx_cycle- > connection_n / 8-connection_n + x = > x-7 connection_n / / that is, when the number of connections exceeds 7 / 8 of the maximum, the value of ngx_accept_disabled will be greater than 0 The current connection handled by work has reached saturation / / at this time work will not compete for new connections, nginx will task the current function has experienced a round of event handling / / that is, the corresponding load has been reduced a little bit, and the ngx_accept_disabled is reduced by if (ngx_accept_disabled > 0) {ngx_accept_disabled-- } else {/ / if the number of active connections has not reached saturation, acquire accept locks / / multiple work, only one lock can be obtained, / / and the process of acquiring the lock is not blocking. After successful acquisition, the ngx_accept_mutex_held will be set to 1 if (ngx_trylock_accept_mutex (cycle) = = NGX_ERROR) {return } / / if ngx_accept_mutex_held = 1 indicates that the process that successfully acquired the lock if (ngx_accept_mutex_held) {/ / the process that grabbed the lock will be marked with the NGX_POST_EVENTS flag / / and added to the post_events queue / / set the flag Delay processing event flags | = NGX_POST_EVENTS } else {if (timer = = NGX_TIMER_INFINITE | | timer > ngx_accept_mutex_delay) {timer = ngx_accept_mutex_delay;}} delta = ngx_current_msec / / the process of acquiring the mutex starts to process the request / / mainly calls epoll_wait to wait for the event / / and joins the queue ngx_posted_accept_events or ngx_posted_events (void) ngx_process_events (cycle, timer, flags) according to the event type / / delta is the time spent calling ngx_process_events / / the value of ngx_current_msec is obtained through the ngx_time_update function / / if ngx_time_update is not executed, delta is still 0 delta = ngx_current_msec-delta; ngx_log_debug1 (NGX_LOG_DEBUG_EVENT, cycle- > log, 0, "timer delta:% M", delta) / / handle the new connection event ngx_event_process_posted (cycle, & ngx_posted_accept_events); / / once the accept event is processed, the current process releases the mutex if (ngx_accept_mutex_held) {/ / release lock ngx_shmtx_unlock (& ngx_accept_mutex) } if (delta) {/ / constantly fetch the one with the lowest time value from the nodes of the red-black tree to determine whether it times out / / if it times out, execute their event function until the minimum time does not time out / / handle the time-out event ngx_event_expire_timers () } / / the normal read and write event ngx_event_process_posted (cycle, & ngx_posted_events) in the ngx_posted_events queue is processed after the lock is released;}

The previous piece of code is relatively long, so here's how to do more:

1. The first step is to determine the current timing policy based on the value of ngx_timer_resolution, which is optional in the nginx.conf file:

Timer_resolution 10ms

If timer_resolution is not 0, select the timing check scheme, that is, set the timing time, 500ms, during which time to check in the red-black tree when there is an event timeout, and if so, handle the timeout event. If ngx_timer_resolution is 0, the timeout detection scheme is adopted. First, the fastest timeout time timer = ngx_event_find_timer () is calculated, then the timeout event is processed at the time of timer, and then the timer is calculated, and the timeout event is processed in turn.

two。 According to the ngx_use_accept_mutex, it determines whether the mutex configuration is turned on, and if the mutex configuration is turned on, it starts to scramble for the lock, otherwise the event is handled directly. First of all, understand the process of scrambling for locks, the core of which is the function ngx_trylock_accept_mutex:

/ / each worker thread will attempt to acquire the lock ngx_int_tngx_trylock_accept_mutex (ngx_cycle_t * cycle) {/ / non-blocking. A return of 1 indicates success, and 0 means acquisition failed if (ngx_shmtx_trylock (& ngx_accept_mutex)) {ngx_log_debug0 (NGX_LOG_DEBUG_EVENT, cycle- > log, 0, "accept mutex locked"). If (ngx_accept_mutex_held & & ngx_accept_events = = 0) {return NGX_OK;} / / add the port information in cycle- > listening to the epoll event / / that is, add listening socket to epoll to listen if (ngx_enable_accept_events (cycle) = = NGX_ERROR) {ngx_shmtx_unlock (& ngx_accept_mutex) Return NGX_ERROR;} / / after listening, you will get ngx_accept_events = 0; ngx_accept_mutex_held = 1; return NGX_OK } / / failure to acquire lock ngx_log_debug1 (NGX_LOG_DEBUG_EVENT, cycle- > log, 0, "accept mutex lock failed:% ui", ngx_accept_mutex_held) / / failed to acquire lock: if you already have a lock, if (ngx_accept_mutex_held) {/ / if you already have a lock, delete the listening socket from its own event listening mechanism if (ngx_disable_accept_events (cycle, 0) = = NGX_ERROR) {return NGX_ERROR } / / have lock flag set 0 ngx_accept_mutex_held = 0;} return NGX_OK;}

The code is more cumbersome, so draw a diagram to illustrate the process:

After getting the lock, we started to deal with the event. Flags | = NGX_POST_EVENTS the event that is actually handled by the delay event before starting processing, so how is it delayed? In fact, in the function ngx_epoll_process_events:

Static ngx_int_tngx_epoll_process_events (ngx_cycle_t * cycle, ngx_msec_t timer, ngx_uint_t flags) {if (flags & NGX_POST_EVENTS) {queue = rev- > accept? & ngx_posted_accept_events: & ngx_posted_events; ngx_post_event (rev, queue) } else {rev- > handler (rev);}}

We have until the value that affects flags is the variable ngx_timer_resolution, with the following code:

Voidngx_process_events_and_timers (ngx_cycle_t * cycle) {if (ngx_timer_resolution) {flags = 0;} else {flags = NGX_UPDATE_TIME;}}

Flags is set to 0 or 1, so executing flags | = NGX_POST_EVENTS causes flags & NGX_POST_EVENTS to be non-zero, so events from epoll_wait are cached in the queue.

3. The next operation is to handle the event. According to the process of scrambling for locks in 2, the process that grabs the lock will be used for the listening socket of the event. When the ngx_process_events method is called to handle the event, the ngx_epoll_process_events method in ngx_epoll_module.c is actually called:

Ngx_epoll_module.cstatic ngx_int_tngx_epoll_process_events (ngx_cycle_t * cycle, ngx_msec_t timer, ngx_uint_t flags) {events = epoll_wait (ep, event_list, (int) nevents, timer) / / timing scheme: the callback function will call the ngx_timer_signal_handler method and set ngx_event_timer_alarm = 1. If the callback / / callback function is not executed, ngx_time_update will not be executed. / / timeout scheme: flags = 1, then flags & NGX_UPDATE_TIME must be true, so every time the method if (flags & NGX_UPDATE_TIME | | ngx_event_timer_alarm) {/ / update the value of ngx_current_msec ngx_time_update () } / / Cache events to queue if (flags & NGX_POST_EVENTS) {queue = rev- > accept? & ngx_posted_accept_events: & ngx_posted_events; ngx_post_event (rev, queue) } else {rev- > handler (rev);}}

You can see in the code that accept events (that is, readable events on the listening port) are cached to queue ngx_posted_accept_events and normal events are cached to queue ngx_posted_events.

4. After the event is cached, the next step is to deal with the new connection event (accept event). Because the current process has listened to the port of a client, the readable event in the request for this port needs to be handled first. The read data is finished, that is, the new connection event in queue ngx_posted_accept_events is processed. If there is a new request connection event during the processing of the new connection, it will block and wait for the next process to acquire the lock before reading. After reading the readable event, the unlock operation ngx_shmtx_unlock is performed.

5. After the lock is released, the connection events after the connection socket interface are processed, that is, the events stored in the queue ngx_posted_events.

Voidngx_event_process_posted (ngx_cycle_t * cycle, ngx_queue_t * posted) {ngx_queue_t * Q; ngx_event_t * ev; while (! ngx_queue_empty (posted)) {Q = ngx_queue_head (posted); ev = ngx_queue_data (Q, ngx_event_t, queue) Ngx_log_debug1 (NGX_LOG_DEBUG_EVENT, cycle- > log, 0, "posted event% p", ev); ngx_delete_posted_event (ev); ev- > handler (ev);}}

As you can see, you are constantly traversing the queue, calling the corresponding handler to handle events.

Second, introduce the initialization of the event handling model

Since the above explanation takes epoll as an example, let's explain how ngnix chooses the event handling mechanism. First, take a look at the events module configuration in nginx.conf:

Events {/ / choose the event handling mechanism to be used. Here we use epoll use epoll; / / whether to accept multiple network requests multi_accept on; / / whether to activate the mutex accept_mutex on; / / to set the maximum number of available connections worker_connections 65535 / / configure http connection timeout keepalive_timeout 60; / / cache size of header in client request request client_header_buffer_size 4k; / / cache size and cache time of static files, such as html/css/image open_file_cache max=65535 inactive=60s / / set the time interval for each cache validity check open_file_cache_valid 80s; / / use static files at least during the valid cache time open_file_cache_min_uses 1; / / set whether to allow cache error messages open_file_cache_errors on;}

The configuration is more detailed, in fact, the relevant is the use epoll configuration.

First of all, nginx defines a unified event handling interface and encapsulates the execution functions of various event handling mechanisms (epoll/poll/select, etc.):

Ngx_event.htypedef struct {/ / add an event (read event / write event) to the event-driven mechanism ngx_int_t (* add) (ngx_event_t * ev, ngx_int_t event, ngx_uint_t flags) / / remove an event (read event / write event) from the event driver ngx_int_t (* del) (ngx_event_t * ev, ngx_int_t event, ngx_uint_t flags); / / enable an added event (code temporarily unused) ngx_int_t (* enable) (ngx_event_t * ev, ngx_int_t event, ngx_uint_t flags) / / disable an added event (the code is temporarily unused) ngx_int_t (* disable) (ngx_event_t * ev, ngx_int_t event, ngx_uint_t flags); / / add the descriptor associated with the specified connection to the multiplexing monitor ngx_int_t (* add_conn) (ngx_connection_t * c) / / delete the descriptor ngx_int_t (* del_conn) (ngx_connection_t * c, ngx_uint_t flags) of the specified connection association from the multiplexing monitoring; / / it is only called in a multithreaded environment. Ngx_int_t (* notify) (ngx_event_handler_pt handler) is not currently used. / / blocking waits for occurrence, and processes ngx_int_t (* process_events) (ngx_cycle_t * cycle, ngx_msec_t timer,ngx_uint_t flags) one by one; / / initializes the event-driven module ngx_int_t (* init) (ngx_cycle_t * cycle, ngx_msec_t timer) / / Recycle resource void (* done) (ngx_cycle_t * cycle);} ngx_event_actions_t

Define global variables for the encapsulated interface, and the interface function also defines the corresponding global function

Ngx_event_actions_t ngx_event_actionsngx_process_events ngx_event_actions.process_eventsngx_done_events ngx_event_actions.donengx_add_event ngx_event_actions.addngx_del_event ngx_event_actions.delngx_add_conn ngx_event_actions.add_connngx_del_conn ngx_event_actions.del_connngx_notify ngx_event_actions.notify

So to add an event (read / write) just call ngx_add_event (ngx_event_t * ev, ngx_int_t event, ngx_uint_t flags). So the point is how to assign a value to ngx_event_actions? If you search event/modules, you will find that there are ngx_event_actions assignments in the modules of various event handling mechanisms:

D:\ mycode\ nginx\ src\ event\ modules\ ngx_devpoll_module.c: 186ngx_event_actions = ngx_devpoll_module_ctx.actions;D:\ mycode\ nginx\ src\ event\ modules\ ngx_epoll_module.c: 189ngx_event_actions = ngx_epoll_module_ctx.actionsD:\ mycode\ nginx\ src\ event\ modules\ ngx_eventport_module.c: 279: ngx_event_actions = ngx_eventport_module_ctx.actions D:\ mycode\ nginx\ src\ event\ modules\ ngx_kqueue_module.c: 224: ngx_event_actions = ngx_kqueue_module_ctx.actions;D:\ mycode\ nginx\ src\ event\ modules\ ngx_poll_module.c: 96: ngx_event_actions = ngx_poll_module_ctx.actions;D:\ mycode\ nginx\ src\ event\ modules\ ngx_select_module.c: 105: ngx_event_actions = ngx_select_module_ctx.actions D:\ mycode\ nginx\ src\ event\ modules\ ngx_win32_select_module.c: 106: ngx_event_actions = ngx_select_module_ctx.actions

Which assignment statement the system uses depends on the user's use epoll in the configuration of the events module.

The specific initialization process:

Static char * ngx_event_core_init_conf (ngx_cycle_t * cycle, void * conf) {ngx_conf_init_uint_value (ecf- > use, module- > ctx_index) } # define ngx_conf_init_size_value (conf, default) / / set the default value if (conf = = NGX_CONF_UNSET_SIZE) {conf = default;} if not specified by the user

The above configuration items are obtained through the function ecf = ngx_event_get_conf (cycle- > conf_ctx, ngx_event_core_module), that is, the value of ecf- > use is the sequence number of the ePoll value, and the sequence number is set in ngx_module.c:

Ngx_int_tngx_preinit_modules (void) {ngx_uint_t I; for (I = 0; ngx_ modules [I]; I modules +) {ngx_ modules [I]-> index = I; ngx_ modules [I]-> name = ngx_module_ namespace [I];} ngx_modules_n = I; ngx_max_module = ngx_modules_n + NGX_MAX_DYNAMIC_MODULES; return NGX_OK;}

The ngx_preinit_modules iterates through the values in all ngx_modules and assigns a value to the index value of the module to identify the module, so the question here is where does the value of ngx_modules come from? Look at the source code and find that there is an assignment statement modules = ngx_dlsym (handle, "ngx_modules") in nginx.c:

Ngx_event.c:ngx_event_process_init (ngx_cycle_t * cycle) {/ / returns the configuration set of events modules ecf = ngx_event_get_conf (cycle- > conf_ctx, ngx_event_core_module); / / traverses all modules for (m = 0; cycle- > modules [m] Cycle- +) {if (cycle- > modules [m]-> type! = NGX_EVENT_MODULE) {continue;} / / if it is not equal to the value of ecf- > use, continue traversing if (cycle- > modules [m]-> ctx_index! = ecf- > use) {continue } / / find the user-specified event handling mechanism module = cycle- > modules [m]-> ctx; / / execute the module's initialization function if (module- > actions.init (cycle, ngx_timer_resolution)! = NGX_OK) {/ * fatal * / exit (2);} break;}}

So we can know that the specific event handling logic of nginx is defined in the modules of each processing mechanism of event/modules. We take one of them as an example (ngx_epoll_module.c). In the above assignment statement ngx_event_actions = ngx_epoll_module_ctx.actions, ngx_event_actions encapsulates the unified event handling call function of nginx, while ngx_epoll_module_ctx defines the context information of the epoll module. Is a static variable of type ngx_event_module_t, which is defined in ngx_epoll_module.c as follows:

Typedef struct {ngx_str_t * name; void * (* create_conf) (ngx_cycle_t * cycle); char * (* init_conf) (ngx_cycle_t * cycle, void * conf); ngx_event_actions_t actions;} ngx_event_module_t

The specific assignment statement is to define the static variable ngx_epoll_module_ctx:

Static ngx_event_module_t ngx_epoll_module_ctx = {& epoll_name, ngx_epoll_create_conf, / * create configuration * / ngx_epoll_init_conf, / * init configuration * / / a pair of actions assignment {/ / add event (read / write event) ngx_epoll_add_event / * add an event * / ngx_epoll_del_event, / * delete an event * / ngx_epoll_add_event, / * enable an event * / ngx_epoll_del_event, / * disable an event * / ngx_epoll_add_connection, / * add an connection * / ngx_epoll_del_connection / * delete an connection * / # if (NGX_HAVE_EVENTFD) ngx_epoll_notify, / * trigger a notify * / # else NULL, / * trigger a notify * / # endif ngx_epoll_process_events, / * process the events * / ngx_epoll_init / * init the events * / ngx_epoll_done, / * done the events * /}}

At this point, the handler of epoll has been assigned to the global variable ngx_event_actions of nginx through the ngx_event_actions = ngx_epoll_module_ctx.actions assignment statement, so when we call the global function ngx_add_event (ngx_event_t * ev, ngx_int_t event, ngx_uint_t flags), we are actually calling ngx_epoll_add_event.

At this point, I believe you have a deeper understanding of "how to understand the Nginx event handling module". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.