In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-01-21 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >
Share
Shulou(Shulou.com)06/02 Report--
Linux-xargs Command http://www.jb51.net/article/44720.htm
Http://blog.csdn.net/yangshangwei/article/details/52666202
Xargs is a filter for passing parameters to commands and a tool for combining multiple commands. It splits a data stream into blocks small enough to facilitate filter and command processing. Typically, xargs reads data from a pipe or stdin, but it can also read data from the output of a file. The default command for xargs is echo, which means that input passed to xargs through pipes will contain line breaks and whitespace, but with xargs processing, newlines and whitespace will be replaced by spaces.
The main function of this command is to build and execute shell commands from the input.
When processing matched files using the-exec option of the find command, the find command passes all matched files together to exec for execution. However, some systems have limits on the length of commands that can be passed to exec, so that an overflow error occurs a few minutes after the find command is run. The error message is usually "Parameter column too long" or "Parameter column overflow". This is what the xargs command is for, especially when used with the find command.
The find command passes the matching files to the xargs command, while the xargs command fetches only some files at a time, not all of them, unlike the-exec option. In this way, it can first process the first part of the files obtained, then the next batch, and so on.
In some systems, using the-exec option will initiate a corresponding process to process each matched file, instead of executing all the matched files as parameters at once; in some cases, there will be too many processes and poor system performance, so the efficiency is not high.
There is only one process using the xargs command. In addition, when using the xargs command, whether to get all the parameters at once or in batches, and the number of parameters each time will be determined according to the options of the command and the corresponding adjustable parameters in the system kernel.
Let's see what parameters xargs has to choose from.
Options
-a file
Read data from file
$cat 1.txt aaa bbb ccc ddd a b $xargs-a 1.txt aaa bbb ccc ddd a b
-0
When a special character is entered, it is treated as a general character, such as "" and spaces
$echo "/ /" | xargs / / $echo "/ /" | xargs-0 / /
-d
Specify delimiter
$cat 1.txtaaa bbb ccc ddda b $cat 1.txt | xargs-d 'c' aaa bbb ddda b
-E eof-str
Specify the end flag as eof-str,xargs to stop processing when this flag is used
$xargs-E 'ddd'-a 1.txtaaa bbb ccc $xargs-E' dd'-a 1.txtaaa bbb ccc ddd a b $cat 1.txt | xargs-E 'ddd' aaa bbb ccc
-I replace-str
Replace each line of input with replace-str
$cat 1.txtaaa bbb ccc ddda b $cat 1.txt | xargs-t-I {} echo {} > > 1.txt echo aaa bbb ccc ddd echo a b $cat 1.txtaaa bbb ccc ddda baaa bbb ccc ddda b
-I
: equivalent to-I {}
$cat 1.txtaaa bbb ccc ddda b $cat 1.txt | xargs-t-I echo {} > > 1.txt echo aaa bbb ccc ddd echo a b $cat 1.txtaaa bbb ccc ddda baaa bbb ccc ddda b
-L max-lines
Each time max-line line input is read and handed over to xargs for processing
$cat 1.txtaaa bbb ccc ddda b $cat 1.txt | xargs-L 2aaa bbb ccc ddda b $cat 1.txt | xargs-L 1aaa bbb ccc ddda b
-l
Similar to-L, except that-l can not specify parameters, the default is 1.
-n max-args
Max-args inputs are executed per line, and all inputs are executed by default
$cat 1.txt | xargs-n 2 aaa bbbccc ddda b
-p
Interactive mode, ask whether to execute before execution
$cat 1.txt | xargs-p/bin/echo aaa bbb ccc ddd a b?. Yaaa bbb ccc ddd a b $cat 1.txt | xargs-p/bin/echo aaa bbb ccc ddd a b?. N
-r
: if there is no input, the execution will be stopped. It will be executed at least once by default.
$echo "" | xargs-t mvmv mv: missing file operandTry `mv-- help` for more information.$ echo "" | xargs-t-r exit directly
-s max-chars
The maximum length of each command executed by xargs (including spaces)
$cat 1.txtaaa bbb ccc ddd a b$ cat 1.txt | xargs-t-s 30 / bin/echo aaa bbb ccc ddd a b aaa bbb ccc ddd a b # length (/ bin/echo aaa bbb ccc ddd a b) = 30$ cat 1.txt | xargs-t-s 14 / bin/echo aaa aaa/bin/echo bbb bbb/bin/echo ccc ccc/bin/echo ddd ddd/bin/echo a b a b # length (/ bin/echo aaa) = 14
-t
Print the executed command first, and then execute
$cat 1.txt | xargs-t/bin/echo aaa bbb ccc ddd a baaa bbb ccc ddd a b
-x
When the length of the command executed by xargs is greater than-s max-char, stop execution
-P max-procs
: modify the number of threads. Default is single thread. When max-procs is 0, as many processes as possible
/ > ls-l
-rw-r--r--. 1 root root 0 Nov 12 10:02 datafile3
-rw-r--r--. 1 root root 10530 Nov 11 23:08 test.tar.bz2
-rwxr--r--. 1 root root 183 Nov 11 08:02 users
-rw-r--r--. 1 root root 279 Nov 11 08:45 users2
# find every ordinary file in the current directory, and then use the xargs command to test which type of file they belong to.
/ > find. -type f-print | xargs file
. / users2: ASCII text
. / datafile3: empty
. / users: ASCII text
. / test.tar.bz2: bzip2 compressed data, block size = 900k
# reclaim the execution permissions of all ordinary files in the current directory.
/ > find. -type f-print | xargs chmod Amurx
/ > ls-l
-rw-r--r--. 1 root root 0 Nov 12 10:02 datafile3
-rw-r--r--. 1 root root 10530 Nov 11 23:08 test.tar.bz2
-rw-r--r--. 1 root root 183 Nov 11 08:02 users
-rw-r--r--. 1 root root 279 Nov 11 08:45 users2
# find all ordinary files in the face-to-face directory, and use the grep command to find the word hostname in the searched files
/ > find. -type f-print | xargs grep "hostname"
# look for the memory information dump file (core dump) throughout the system, and then save the results to the / tmp/core.log file.
/ > find /-name "core"-print | xargs echo "" > / tmp/core.log / > pgrep mysql | xargs kill-9 # directly kills the mysql process
[1] + Killed mysql
Examples are as follows:
1. When you try to delete too many files with rm, you may get an error message: / bin/rm Argument list too long. Use xargs to avoid this problem
Find ~-name'* .log'- print0 | xargs-0 rm-f
two。 To get a list of all the files at the end of * .conf under / etc/, there are several different ways to get the same result. The following example is just a demonstration of how to use xargs, in which practical xargs passes the output of the find command to ls-l
# find / etc-name "* .conf" | xargs ls-l
3. If you have a file that contains a lot of URL you want to download, you can use xargs to download all the links
# cat url-list.txt | xargs wget-c
4. Find all the jpg files and compress it
# find /-name * .jpg-type f-print | xargs tar-cvzf images.tar.gz
5. Copy all the picture files to an external hard drive
# ls * .jpg | xargs-N1-I cp {} / external-hard-drive/directory
EXAMPLES
Find / tmp-name core-type f-print | xargs / bin/rm-f
Find files named core in or below the directory / tmp and delete them. Note that this will work incorrectly if there are any filenames containing newlines or spaces.
Find / tmp-name core-type f-print0 | xargs-0 / bin/rm-f
Find files named core in or below the directory / tmp and delete them, processing filenames in such a way that file or directory names containing spaces or newlines are correctly handled.
Find / tmp-depth-name core-type f-delete
Find files named core in or below the directory / tmp and delete them, but more efficiently than in the previous example (because we avoid the need to use fork (2) and exec (2) to launch rm and we don't need the extra xargs process).
Cut-d:-F1 < / etc/passwd | sort | xargs echo
Generates a compact listing of all the users on the system.
Xargs sh-c 'emacs "$@" < / dev/tty' emacs
Launches the minimum number of copies of Emacs needed, one after the other, to edit the files listed on xargs' standard input. This example achieves the same effect as BSD's-o option, but in a more flexible and portable way.
For example:
If there are too many files in the path directory, the error cannot be executed because the parameter list is too long. However, after switching to xargs, the problem was solved.
Find / path-type f-print0 | xargs-0 rm
Xargs splits the long list of files generated by find into multiple substrings and then calls rm on each substring. -print0 indicates that the output is separated by null (- print uses line feeds), and-0 indicates that the input is separated by null. This is much more efficient than using the find command as follows.
Find / path-type f-exec rm'{}'\
The xargs command, which should immediately follow the pipe operator, takes standard input as the main source data stream, and uses stdin to execute other commands by providing command-line arguments, such as:
Command | xargs
The example applies 1 to convert multi-line input into single-line output:
Amosli@amosli-pc:~/learn$ cat example.txt
1 2 3 4 5
6 7
eight
Amosli@amosli-pc:~/learn$ cat example.txt | xargs
1 2 3 4 5 6 7 8
For example, apply 2 to convert single-line input to multiple-line output:
Amosli@amosli-pc:~/learn$ cat example.txt | xargs-n 2
1 2
3 4
5 6
7 8
The space is the default delimiter, and-n indicates how many parameters are displayed per line
You can also use the-d parameter to separate parameters, as follows:
Amosli@amosli-pc:~/learn$ echo "splitXhiXamosliXsplit" | xargs-d "X"-n 1
Split
Hi
Amosli
Split
Instance application 3, read stdin, and pass formatting parameters to the command
# define an echo command to add # to the output parameters every time
Amosli@amosli-pc:~/learn$ cat cecho.sh
Echo $*'#
# requirement 1: output multiple parameters
Amosli@amosli-pc:~/learn$ sh cecho.sh arg1
Arg1#
Amosli@amosli-pc:~/learn$ sh cecho.sh arg2
Arg2#
Amosli@amosli-pc:~/learn$ sh cecho.sh arg3
Arg3#
# requirement 2: provide all command parameters at once
Amosli@amosli-pc:~/learn$ sh cecho.sh arg1 arg2 arg3
Arg1 arg1 arg2 arg3#
# for requirements 1 and 2, use xargs instead, and first create a new file args.txt with vi, as follows:
Amosli@amosli-pc:~/learn$ cat args.txt
Arg1
Arg2
Arg3
# batch output parameters:
Amosli@amosli-pc:~/learn$ cat args.txt | xargs-n 1
Arg1
Arg2
Arg3
Amosli@amosli-pc:~/learn$ cat args.txt | xargs-n 2 sh cecho.sh
Arg1 arg2#
Arg3#
# output all parameters at once:
Amosli@amosli-pc:~/learn$ cat args.txt | xargs sh cecho.sh
Arg1 arg2 arg3#
Requirement 3, how to embed parameters into a fixed command line? As follows:
Amosli@amosli-pc:~/learn$ sh cecho.sh-p args1-1
-p args1-cake
Amosli@amosli-pc:~/learn$ sh cecho.sh-p args2-1
-p args2-cake
Amosli@amosli-pc:~/learn$ sh cecho.sh-p args3-1
-p args3-cake
Solutions that use xargs:
Amosli@amosli-pc:~/learn$ cat args.txt | xargs-I {} sh cecho.sh-p {}-1
-p arg1-cake
-p arg2-cake
-p arg3-cake
#-I {} approved the replacement string, and the string {} will be replaced by the parameters read from stdin. When using-I, the corresponding parameters can be replaced in a loop as required.
Example application 4, use xargs with find
As an example has been given earlier, it should be noted here that the file name delimiter should be output separated by the character null, as shown below, otherwise the file may be mistakenly deleted
Amosli@amosli-pc:~/learn$ find. -type f-name "* test*.txt"-print0 | xargs-0 rm-f
Other:
Cat file | (while read arg; do cat $arg; done)
Cat file | xargs-I {} cat {}
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 267
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.