Shulou(Shulou.com)06/03 Report--

1.1 treading pit case

The trample program is a resident Agent-like management process that includes, but is not limited to, the following types of tasks:

a. Multithreaded network communication packet processing and control Master node interaction has a fixed Listen port b. Regular job tasks, execute the shell command c. Etc through subprocess.Pipe

The process of discovering the pit is interesting:

a. Restart Agent found that Port was occupied = > immediately thought that the process might not be killed, whether there was a problem with stopping the script = > excluding the discovery that no, the Agent process did die = > through netstat-tanop | grep port_number found that the port was indeed occupied = > debug environment, directly killed the occupied process and missed the opportunity to find the problem for the first time. The problem reappears after a period of time, and the Port is still occupied after restart. The location problem occurs in a script called xxxxxx.sh, which occupies the port used by Agent. It is strange that what a xxx.sh script does with this weird Port (Port is greater than 60000, interested friends can think about why Agent uses 6W + port by default) = > review the script does not have the code for port listening. The process shares the resources of the parent process. Tracing back to the script, it is found that it is indeed one of the scripts in the task started by Agent = > the problem is basically located. The script belongs to the script called by Agent = > the Agent inherits the original resource FD of Agent, that is, the port= > although the script passively triggers the terminate mechanism due to timeout, terminate does not kill the child process = > the parent process (ppid) of the script process is reset to 1D. The problem * * is that the script process timed out kill logic * * 1.2 fill in the pit solution

Through the code review, you can find the library code that shell executes as follows:

Self._subpro = subprocess.Popen (cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, preexec_fn=_signal_handle) # focus on shell=True!

Change the above code to:

Self._subpro = subprocess.Popen (cmd.split (), stdout=subprocess.PIPE, stderr=subprocess.PIPE, preexec_fn=_signal_handle) # focus on removing shell=True1.3 pit analysis

Agent executes this code in a newly created threading thread, and if the thread times out (xx seconds), it calls self._subpro.terminate () to terminate the script.

The surface is normal:

Enable a new thread to execute the script if there is a problem, the execution timeout prevents hang from calling terminate to kill the process when other tasks are executed

Deep questions:

Subprocess.Pipe in Python 2.7.x if shell=True, the relevant pid will be set to shell (sh/bash/etc) itself (the shell parent process executing the command) by default, not the process child process executing the cmd task, because it will copy the opened FD table of the parent process, so that even if it is killed, it still retains the ownership of the Listened Port FD.

This kills the shell process (not necessarily dead, but may enter the defunct state), but the actual execution process is indeed alive. As a result, the pit in 1.1 was firmly stepped on.

1.4 Post-pit expansion 1.4.1 expand knowledge

The extended knowledge in this section includes two parts:

In the Linux system, what information will the child process inherit from the parent process? the meaning of the resident process choosing > 60000 port Agent

Expand the knowledge and save it at the end of the next chapter. Those who are interested can search on their own.

1.4.1 Technical keywords Linux system process Linux random port selector multithreaded execution Shell execution 1.5 pit filling summary child process inherits the resource information of the parent process

If you kill only the parent process of a process, the child process that integrates the resources of the parent process will continue to occupy the resources of the parent process without releasing them, including but not limited to

In the use of listened portopened fdetcPython Popen, the bool status of shell determines the logic of process kill. You need to choose how to use it according to the scenario.

Life is short. We use Python

Job number: programmer's guide to somniloquy

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.