Shulou(Shulou.com)05/31 Report--

Editor today to take you to understand how to analyze the gunicorn Arbiter source code, the knowledge points in the article are introduced in great detail. Friends who feel helpful can browse the content of the article together with the editor, hoping to help more friends who want to solve this problem to find the answer to the problem. Follow the editor to learn more about "how to analyze the gunicorn Arbiter source code".

As mentioned earlier, Arbiter is the core of the gunicorn master process. Arbiter is mainly responsible for managing worker processes, including starting, monitoring and killing Worker processes; at the same time, Arbiter can also reload App applications or upgrade gunicorn online when certain signals occur. The core code of Arbiter is in a file, the amount of code is not large, the source code is here: https://github.com/benoitc/gunicorn.

The main methods of Arbiter are as follows:

Setup:

Deal with configuration items, the most important of which are the number of worker and the worker working model

Init_signal:

Register signal processing function

Handle_xxx:

The specific processing function of each signal

Kill_worker,kill_workers:

Signal the worker process

Spawn_worker, spawn_workers:

Fork creates a new worker process

Murder_workers:

Kill worker processes that have not responded for a period of time

Manage_workers:

Based on the number of worker in the configuration file and the current number of worker in active, decide whether to fork or kill worker process

Reexec:

Receive the signal SIGUSR2 call and upgrade gunicorn online

Reload:

When the signal SIGHUP call is received, the worker process will be newly started according to the new configuration and the previous worker process will be killed.

Sleep:

When there is no signal processing, sleep using the timeout of select can be awakened.

Wakeup:

Wake up the process by writing a message to the pipeline

Run:

Main cycle

The only functions Arbiter really called by other code (Application) are the _ _ init__ and run methods, in one sentence of code:

Arbiter (self) .run ()

The self in the above code is the Application instance, where _ _ init__ calls setup to set the configuration item. The following is the pseudo code for the run method

Def run () self.init_signal () self.LISTENERS = create_sockets (self.cfg self.log) self.manage_workers () while True: if no signal in SIG_QUEUE self.sleep () else: handle_signal ()

About the fork child process

The code of the fork child process is in spawn_worker, and the source code is as follows:

Arbiter.spawn_worker

Main process:

(1) load worker_class and instantiate it (default is synchronous model SyncWorker)

(2) the parent process (master process) return after fork, and the logic after that runs in the child process.

(3) call worker.init_process to enter the cycle, and all the work of IELTS training for the new channel is in this cycle.

(4) after the loop ends, call sys.exit (0)

(5) finally, in finally, record the exit of the worker process

Here is a little bit of code I wrote myself to simplify the main fork process

1 # prefork.py 2 import sys 3 import socket 4 import select 5 import os 6 import time 7 8 def do_sub_process (): 9 pid = os.fork () 10 if pid

0:14 print 'fork sub process% d'% pid15 return16 17 # must be child process18 time.sleep (1) 19 print' sub process will exit', os.getpid (), os.getppid () 20 sys.exit (0) 21 22 def main (): 23 sub_num = 224 for i in range (sub_num): 25 do_sub_process () 26 time.sleep (10) 27 print 'main process will exit' Os.getpid () 28 29 if _ _ name__ = ='_ main__':30 main ()

Output in a test environment:

Fork sub process 9601

Fork sub process 9602

Sub process will exit 9601 9600

Sub process will exit 9602 9600

Main process will exit 9600

It is important to note that line 20 calls sys.exit to ensure the end of the child process, otherwise the for loop in the main function and the logic after that will continue. Comment out line 19 and rerun, just look at the output.

About the Kill sub-process

It is very easy for a master process to kill worker a process. Send a signal directly. The source code is as follows:

1 def kill_worker (self, pid, sig): 2 ""\ 3 Kill a worker 4 5: attr pid: int, worker pid 6: attr sig: `signal.SIG* `value 7 "8 try: 9 os.kill (pid Sig) 10 except OSError as self.WORKERS.pop 11 if e.errno = = errno.ESRCH:12 try:13 worker = self.WORKERS.pop (pid) 14 worker.tmp.close () 15 self.cfg.worker_exit (self, worker) 16 return17 except (KeyError) OSError): 18 return19 raise

About sleep and wakeup

Let's take a look at Arbiter's sleep and wakeup. Arbiter will "sleep" when there is no signal to be processed. Of course, it does not really call time.sleep, otherwise the signal cannot be processed as soon as it comes. It has to be implemented cleverly here, using pipelines and select's timeout. Just look at the code.

Def sleep (self): "\ Sleep until PIPE is readable or we timeout. A readable PIPE means a signal occurred." Ready = select.select ([self.PIPE [0]], [], [], 1.0) # self.PIPE = os.pipe () if not ready [0]: return while os.read (self.PIPE [0], 1): pass

The comments in the code are so clear that either PIPE can read and return immediately, or wait for a timeout. The pipe is readable because there is a signal. Here, look at the pipe function.

Os.pipe ()

Create a pipe. Return a pair of file descriptors (rjime w) usable for reading and writing, respectively.

So let's see when the pipe is readable: it must be something written to the pipe, and that's what the wakeup function does.

Def wakeup (self): "Wake up the arbiter by writing to the PIPE" os.write (self.PIPE [1], bread.')

Finally, the signal processing of Arbiter is attached:

Exit, INT: quick close

TERM: graceful shutdown. Wait for the staff to complete their current request until it times out.

HUP: reload the configuration, start the new worker process with the new configuration, and gracefully close the old worker process. If the application is not preloaded (using the-- preload option), Gunicorn will also load the new version.

TTIN: increase the number of processes by one

TTOU: reduce the number of processes by one

USR1: reopen the log file

USR2: upgrade Gunicorn in flight. A separate term signal should be used to terminate the old process. This signal can also be used to use new versions of preloaded applications.

Winch: gracefully shuts down the worker process when Gunicorn is guarded.

Thank you for your reading, the above is the whole content of "how to analyze the gunicorn Arbiter source code", learn friends to hurry up to operate it. I believe that the editor will certainly bring you better quality articles. Thank you for your support to the website!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.