Shulou(Shulou.com)06/02 Report--

This article mainly introduces how to solve the problem of process suspension caused by calling Process.waitfor. It is very detailed and has a certain reference value. Interested friends must read it!

Problem background

If you want to invoke the shell script in Java, you can use Runtime.exec or ProcessBuilder.start. They all return a Process object through which the Process can be used to get the output executed by the script and then process it in Java.

For example, the following code:

Try {Process process = Runtime.getRuntime () .exec (cmd); process.waitFor () / / do something...} catch (Exception e) {e.printStackTrace ();}

Usually, it is pointed out in the security coding specification that when using Process.waitfor, it may cause process blocking or even deadlock. So how should this sentence be understood? Use a practical example to illustrate.

Problem description

Call the shell script with Java code, and you will find that both the Java process and the Shell process hang and cannot be terminated.

Java Code processtest.java

Try {Process process = Runtime.getRuntime () .exec (cmd); System.out.println ("start run cmd=" + cmd); process.waitFor (); System.out.println ("finish run cmd=" + cmd) } catch (Exception e) {e.printStackTrace ();}

Called Shell script doecho.sh

#! / bin/bashfor ((iTuno;; iTunes +)) do echo-n "0123456789" echo $I > > reason why count.logdone hangs

Calling Runtime.exec in the main process creates a child process to execute the shell script. After the child process is created, it will run independently from the main process.

Because the main process needs to wait for the script to complete and then process the return value or output of the script, the main process calls Process.waitfor and waits for the child process to complete.

Through the shell script, you can see that the execution process of a child process is the constant printing of information. It can be obtained and processed through Process.getInputStream and Process.getErrorStream in the main process.

At this time, the waiting child process keeps sending data to the main process, and the main process has suspended after calling Process.waitfor. When the buffer between the previous child process and the main process is full, the child process can no longer write data, and then it will hang.

In this way, the process waits for the main process to read the data, the main process waits for the child process to finish, and the two processes wait for each other, resulting in a deadlock.

Solution method

Based on the above analysis, as long as the main process can constantly process the data in the buffer before waitfor. Because, before waitfor, we can start two additional threads separately, which can be used to deal with InputStream and ErrorStream. The example code is as follows:

Try {final Process process = Runtime.getRuntime () .exec (cmd); System.out.println ("start run cmd=" + cmd) / / Thread new Thread () {@ Override public void run () {processing InputStream BufferedReader in = new BufferedReader (new InputStreamReader (process.getInputStream () String line = null Try {while ((line = in.readLine ())! = null) {System.out.println ("output:" + line) }} catch (IOException e) { E.printStackTrace () } finally {try { In.close () } catch (IOException e) {e.printStackTrace () }. Start () New Thread () {@ Override public void run () {BufferedReader err = new BufferedReader (new InputStreamReader (process.getErrorStream () String line = null Try {while ((line = err.readLine ())! = null) {System.out.println ("err:" + line) }} catch (IOException e) { E.printStackTrace () } finally {try { Err.close () } catch (IOException e) {e.printStackTrace () Start (); process.waitFor () Description on System.out.println ("finish run cmd=" + cmd);} catch (Exception e) {e.printStackTrace ();} JDK

By default, the created subprocess does not have its own terminal or console.

All its standard I take O (i.e. Stdin, stdout, stderr) operations will be redirected to the parent process, where they can be accessed via the streams obtained using the methods getOutputStream (), getInputStream (), and getErrorStream ().

The parent process uses these streams to feed input to and get output from the subprocess.

Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, or even deadlock.

Two points can be seen from JDK's description:

If the size of the bufffer used by the standard input and output streams in the system is limited, blocking or deadlock may occur during all reads and writes. -this has been analyzed above

The standard Ithumb O of the child process has been redirected to the parent process. The parent process can obtain the Imax O of the child process through the corresponding interface. -how do I redirect IPUBO?

The story behind it

To answer the above questions, you can try to analyze from the system level.

First of all, through the ps command, you can see that there are two more processes on linux: a Java process, a shell process, and shell is a child process of java.

You can then see that the status of the shell process is displayed as pipe_w. At first I thought pipe_w meant pipe_write. Take a closer look at / proc/pid/wchan and find that pipe_w actually represents pipe_wait. Usually / proc/pid/wchan represents a memory address or the name of the method that the process is executing. Therefore, this seems to indicate that the process waited while operating on the pipe and was suspended. We know that pipe is a kind of IPC, which is usually used for communication between parent and child processes. In this way, we can guess that there may be a blocking when the parent and child processes communicate through pipe.

In addition, observe the fd information of the parent-child process, namely / proc/pid/fd. You can see that the 0 stdin/stdout/stderr 1 stdin/stdout/stderr of the child process is redirected to three pipe files respectively; there are also references to the three pipe files in the parent process.

To sum up, the process should be like this: the child process keeps writing data to the pipe, while the parent process does not read the data in the pipe, resulting in the pipe being stuffed up and the child process unable to continue writing, so the state of pipe_wait occurs. So how big is pipe?

Test the size of the pipe

Because I have recorded the number of characters printed in the footsteps of doecho.sh, I can see how much data the child process finally sent by looking at count.log. After the child process hangs, the data of count.log remains unchanged at 6543. Therefore, when the previous child process writes 6543*10=65430bytes to the pipe, the process hangs. 65536-65430=106byte means that the distance is 64K less than 106bytes.

Change to another test method, write 1k each time, record how much can be written. The process code is shown in test_pipe_size.sh. The test result is 64K. There is a difference between the two results, 106byte. How big is this pipe?

Pipe Analysis on Linux

The most direct way is to look at the source code. The implementation code of Pipe is mainly in linux/fs/pipe.c, and we mainly look at the pipe_wait method.

Pipe_read (struct kiocb * iocb, struct iov_iter * to) {size_t total_len = iov_iter_count (to); struct file * filp = iocb- > ki_filp; struct pipe_inode_info * pipe = filp- > private_data; int do_wakeup; ssize_t ret; / * Null read succeeds. * / if (unlikely (total_len = 0)) return 0; do_wakeup = 0; ret = 0; _ pipe_lock (pipe); for (;;) {int bufs = pipe- > nrbufs; if (bufs) {int curbuf = pipe- > curbuf Struct pipe_buffer * buf = pipe- > bufs + curbuf; const struct pipe_buf_operations * ops = buf- > ops; size_t chars = buf- > len; size_t written; int error If (chars > total_len) chars = total_len; error = ops- > confirm (pipe, buf); if (error) {if (! ret) ret = error Break;} written = copy_page_to_iter (buf- > page, buf- > offset, chars, to); if (unlikely (written

< chars)) { if (!ret) ret = -EFAULT; break; } ret += chars; buf->

Offset + = chars; buf- > len-= chars; / * Was it a packet buffer? Clean up and exit * / if (buf- > flags & PIPE_BUF_FLAG_PACKET) {total_len = chars; buf- > len = 0 } if (! buf- > len) {buf- > ops = NULL; ops- > release (pipe, buf); curbuf = (curbuf + 1) & (pipe- > buffers-1) Pipe- > curbuf = curbuf; pipe- > nrbufs =-- bufs; do_wakeup = 1;} total_len-= chars If (! total_len) break; / * common path: read succeeded * /} if (bufs) / * More to do? * / continue; if (! pipe- > writers) break If (! pipe- > waiting_writers) {/ * syscall merging: Usually we must not sleep * if O_NONBLOCK is set, or if we got some data. * But if a writer sleeps in kernel space, then * we can wait for that data without violating POSIX. * / if (ret) break; if (filp- > f_flags & O_NONBLOCK) {ret =-EAGAIN; break }} if (signal_pending (current)) {if (! ret) ret =-ERESTARTSYS; break } if (do_wakeup) {wake_up_interruptible_sync_poll (& pipe- > wait, POLLOUT | POLLWRNORM); kill_fasync (& pipe- > fasync_writers, SIGIO, POLL_OUT);} pipe_wait (pipe) } _ pipe_unlock (pipe); / * Signal writers asynchronously that there is more room. * / if (do_wakeup) {wake_up_interruptible_sync_poll (& pipe- > wait, POLLOUT | POLLWRNORM); kill_fasync (& pipe- > fasync_writers, SIGIO, POLL_OUT);} if (ret > 0) file_accessed (filp); return ret;}

You can see that the Pipe is organized into a circular structure, that is, a circular linked list. The elements in the linked list are the structure of struct pipe_buffer, each pipe_buffer for one page. There are 16 elements in the linked list, that is, the total size of pipe buffer is 16*page. If the page size is 4K, then the total size of the pipe buffer should be 16*4K=64K.

The above is all the contents of this article entitled "how to solve the process hang problem caused by calling Process.waitfor". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.