Shulou(Shulou.com)06/02 Report--

Grep command 1, basic regular expression example-find specific characters

Here we will take the / etc/passwd file that stores all users on the machine as an example.

Demo1

[root@localhost ~] # grep-n "root" / etc/passwd / /-n shows line number 1:root:x:0:0:root:/root:/bin/bash10:operator:x:11:0:operator:/root:/sbin/nologin2, basic regular expression instance-find collection characters

When there are duplicate characters, you can use "[]" for set matching, matching only one character in "[]" at a time.

Demo2

[root@localhost ~] # grep-n "[fn] tp" / etc/passwd12:ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin27:ntp:x:38:38::/etc/ntp:/sbin/nologin3, basic regular expression example-reverse selection

Adding "^" in parentheses in "[]" indicates reverse selection (friends with a certain foundation must know that "^ []" means to locate the beginning of the line, where the meaning of the internal and external position of "^" will be completely different. )

Demo3

[root@localhost ~] # grep-n "^ [^ root]" / etc/passwd / / matches all options except 2:bin:x:1:1:bin:/bin:/sbin/nologin3:daemon:x:2:2:daemon:/sbin:/sbin/nologin.42:named:x:25:25:Named:/var/named:/sbin/nologin4 that start with root, basic regular expression instance-escape character

A metacharacter in a regular expression, so here you need to use the escape character "\" to convert a character with a special meaning into a normal character.

Demo4

[root@localhost] # grep-n'\. $'test.txt 1:he was short and fat.2:He was wearing a blue polo shirt with black pants. 3:The home of Football on BBC Sport online.5:google is the best tools for search keyword.5, basic regular expression instance-find any character & find duplicate characters

The decimal point (.) in a regular expression is also a metacharacter that represents any character.

Demo5-1

[root@localhost] # grep-n "r.. t" / etc/passwd / / (.) the decimal point represents any character 1:root:x:0:0:root:/root:/bin/bash10:operator:x:11:0:operator:/root:/sbin/nologin12:ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin

In the above results, the "root" string "r... t" matches the rule. If you want to query oo, ooo, ooooo, and so on, you need to use asterisk (*) metacharacters. It is important to note, however, that "*" represents the repetition of zero or more previous single characters. "o *" means to have zero (that is, empty characters) or a character greater than or equal to one "o"

Demo5-2

[root@localhost ~] # grep-n "oo*" / etc/passwd1:root:x:0:0:root:/root:/bin/bash2:bin:x:1:1:bin:/bin:/sbin/nologin3:daemon:x:2:2:daemon:/sbin:/sbin/nologin4:adm:x:3:4:adm:/var/adm:/sbin/nologin5:lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin.6, An example of a basic regular expression-- finding a range of consecutive characters

For example, if you look for consecutive characters of three to five o, you need to use the bounded character "{}" in the underlying regular expression. Because "{}" has a special meaning in Shell, when using the "{}" character, you need to use the escape character "\" to convert the "{}" character into a normal character.

Demo6

[root@localhost ~] # grep-n "0\ {2,\}" / etc/passwd / / indicates a string 11:games:x:12:100:games:/usr/games:/sbin/nologin41:zhy:x:1000:1000:zhy:/home/zhy:/bin/bashEgrep command with more than 2 o in the middle

In addition, the grep command only supports basic regular expressions, and if you use extended regular expressions, you need to use the egrep or awk command. The awk command is explained later, and here we use the egrep command directly. The usage of the egrep command is similar to that of the grep command. (egrep commands that can be used by the grep command can also be used.)

Extended regular expression metacharacter function + function: repeat one or more of the previous characters? Function: the first character of zero or one | function: use or (or) to find multiple characters () function: find "group" string () + function: identify multiple repeating groups

Demo demo

[root@localhost ~] # egrep-n "10 +" / etc/passwd / / use the "+" extended metacharacter 11:games:x:12:100:games:/usr/games:/sbin/nologin31:qemu:x:107:107:qemu user:/:/sbin/nologin41:zhy:x:1000:1000:zhy:/home/zhy:/bin/bash [root@localhost ~] # egrep-n "10?" / etc/passwd / / use "?" Extended metacharacter 2:bin:x:1:1:bin:/bin:/sbin/nologin9:mail:x:8:12:mail:/var/spool/mail:/sbin/nologin10:operator:x:11:0:operator:/root:/sbin/nologin11:games:x:12:100:games:/usr/games:/sbin/ nologin [root @ localhost ~] # egrep-n 'root | zhy' / etc/passwd / / use "|" extended metacharacter 1: Root:x:0:0:root:/root:/bin/bash10:operator:x:11:0:operator:/root:/sbin/nologin41:zhy:x:1000:1000:zhy:/home/zhy:/bin/bash [root@localhost ~] # egrep-n'(f | n) tp' / etc/passwd / / use the "()" extended metacharacter Use the 12:ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin27:ntp:x:38:38::/etc/ntp:/sbin/nologinSed command with "|"

Sed is a good file processing tool, itself is a pipeline command, mainly in the behavior unit for processing, data rows can be replaced, deleted, new, selected and other specific work

The workflow of sed mainly includes three processes: read, execute and display.

Read: sed reads a line from the input stream (file, pipe, standard input) and stores it to a temporary buffer

Punch zone (also known as pattern space, pattern space).

Sed execution: by default, all sed commands are executed sequentially in the pattern space, and unless the address of the line is specified, the sed command will be executed on all lines in turn.

Z. display: send the modified content to the output stream. After the data is sent again, the schema space will be emptied.

Before all the contents of the file are processed, the above process will be repeated until all the contents have been processed.

Note: by default, all sed commands are performed in schema space and are not saved.

Sed command format

Sed [option] 'Operation' parameter

Sed [options]-f scriptfile parameter / / scriptfile represents the script file

Common option

-e: indicates that the input text file is processed with a specified command or script.

-f: indicates that the input text file is processed with the specified script file.

-h: displays help.

-n: indicates that only the processed results are displayed.

-I: edit the text file directly.

Commonly used "operation" parameters

A: increase by adding a specified line below the current line.

C: replace, replacing the selected line with the specified content.

D: delete, delete selected row

I: insert, inserting a specified line above the selected row.

P: print, which is usually used with the "- n" option

S: replace, replace the specified character.

Y: character conversion.

Examples of basic usage:

Output all, the effect is equivalent to the cat command

[root@localhost] # sed-n 'p' / etc/passwd / / is equivalent to the cat command root:x:0:0:root:/root:/bin/bashbin:x:1:1:bin:/bin:/sbin/nologindaemon:x:2:2:daemon:/sbin:/sbin/nologinadm:x:3:4:adm:/var/adm:/sbin/nologin.

Output a specific line, or a segment of a line

[root@localhost ~] # sed-n '10p' / etc/passwd / / output line 10 operator:x:11:0:operator:/root:/sbin/nologin [root@localhost ~] # sed-n' 2 4p'/ etc/passwd / / output 2mm 4 lines bin:x:1:1:bin:/bin:/sbin/nologindaemon:x:2:2:daemon:/sbin:/sbin/nologinadm : x:3:4:adm:/var/adm:/sbin/nologin

Output all odd lines

[root@localhost] # sed-n 'ntransferp' / etc/passwd / / output odd line, even-numbered behavior pbin _ nbin _ Similarly, in addition to the basic usage, the sed command can also be used in combination with regular expressions

Output lines that contain specific content (like the grep command, you can use ^, $to locate the beginning and end of a line)

[root@localhost ~] # sed-n'/ root/p' / etc/passwdroot:x:0:0:root:/root:/bin/bashoperator:x:11:0:operator:/root:/sbin/nologin

Output lines that contain specific words

[root@localhost ~] # sed-n'/\ / p' / etc/passwd / /\

Represents the word boundary root:x:0:0:root:/root:/bin/bashoperator:x:11:0:operator:/root:/sbin/nologin

Replace eligible text

Sed's test.txt the test.txt / / replaces the first the in each line with THE

Sed's Placement LAccord 3' test.txt / / replace the third l in each line with L

Sed's test.txt the test.txt / / replaces all the in the file with THE

Sed's _

Sed's / ^ / # / 'test.txt / / insert the #: sign at the beginning of each line

Sed'/ the/s/ ^ / # / 'test.txt / / insert the # sign at the beginning of each line containing the

Sed's Universe test.txt 'test.txt / / insert the string EOF at the end of each line

Sed'3 the test.txt test.txt / / replace all the in line 3 and 5 with THE

Sed'/ the/s/o/O/g' test.txt / / replace o with O in all lines that contain the

Migrate text

Sed'/ the/ {Hutterd}; $G' test.txt / / move the line containing the to the end of the file, {;} for multiple operations

Sed'1 test.txt 5 {Hutchd}; 17G'G` / / after transferring the contents of line 1 to line 5 to line 17

Sed'/ the/w out.file' test.txt / / Save the line containing the as a file out.file

Sed'/ the/r / etc/hostname' test.txt / / after adding the contents of the file / etc/hostname to each line that contains the

Sed '3aNew' test.txt / / insert a new line after line 3 with the content New

Sed'/ the/aNew' test.txt / / insert a new line with the content New after each row that contains the

Sed '3aNew1\ nNew2' test.txt / / insert multiple lines after line 3, with\ n in the middle indicating line break

AWK command

AWK is a programming language tool for processing text. AWK is similar to the shell programming language in many ways, although AWK has its own syntax and is one of the most powerful data processing engines available in any environment.

Basic structure

Awk [options] 'pattern or condition {Edit Command}' File 1 File 2 / / filter and output matching content

Awk-f script file 1 file 2 / / call from the script and output

Basically use the example [root@localhost ~] # awk-F:'{print $1 tab 3}'/ etc/passwd// to filter out the contents of the first and third columns with ":" as the delimiter, and output (by default, the field delimiter is a space or the tab key). Special built-in variables (can be introduced directly)

FS: specifies the field separator for each line of text, which defaults to spaces or tab stops.

NF: the number of fields in the rows currently being processed.

NR: the line number (ordinal) of the currently processed row.

$0: the entire line content of the currently processed row.

$n: the nth field (nth column) of the currently processed row.

FILENAME: the name of the file being processed.

RS: data records are separated. The default is\ n, that is, one record per behavior.

Output text by line

[root@localhost ~] # awk 'NR==2,NR==4 {print}' / etc/passwd / / output the second to fourth lines of text content bin:x:1:1:bin:/bin:/sbin/nologindaemon:x:2:2:daemon:/sbin:/sbin/nologinadm:x:3:4:adm:/var/adm:/sbin/nologin [root@localhost ~] # awk-F ":" 'NR==2,NR==4 {print $1 $3}'/ etc/passwd// outputs the first and third columns of the second to fourth lines bin 1daemon 2adm 3 [root@localhost ~] # awk'(NR==1) | (NR==4) {print}'/ etc/passwd// outputs the first and fourth lines of content root:x:0:0:root:/root:/bin/bashadm:x:3:4:adm:/var/adm:/sbin/nologin

Output odd and even lines (you can use the logical operator "& &" in awk to mean "and", "|" means "or", "!" It means "not"; you can also perform simple mathematical operations, such as +, -, *, /,%, ^ for addition, subtraction, multiplication, division, remainder and multiplier, respectively.

[root@localhost ~] # awk 'NR%2==1 {print}' / etc/passwd / / output odd line root:x:0:0:root:/root:/bin/bashdaemon:x:2:2:daemon:/sbin:/sbin/nologinlp:x:4:7:lp:/var/spool/lpd:/sbin/nologinshutdown:x:6:0:shutdown:/sbin:/sbin/shutdown. [root@localhost ~] # awk 'NR%2==0 { Print}'/ etc/passwd / / outputs even-numbered lines bin:x:1:1:bin:/bin:/sbin/nologinadm:x:3:4:adm:/var/adm:/sbin/nologinsync:x:5:0:sync:/sbin:/bin/synchalt:x:7:0:halt:/sbin:/sbin/halt.

Output lines that begin with root

[root@localhost ~] # awk'/ ^ root/ {print}'/ etc/passwdroot:x:0:0:root:/root:/bin/bash

Count the number of rows ending in / bin/bash

[root@localhost ~] # awk 'BEGIN {xylene 0}; /\ / bin\ / bash$/ {xylene +}; END {print x}' / etc/passwd2

Count the number of paragraphs of text separated by blank lines:

[root@localhost opt] # vim name.txtzhangsan:lisi:wangwuzhaoliu:liuliuheiba:heihei [root@localhost opt] # awk 'BEGIN {RS= ":"}; END {print NR}' / opt/name.txt5// Statistical rules: break lines when key symbols are encountered

Call the w command and count the number of online users:

[root@localhost opt] # awk 'BEGIN {while ("w" | getline) natives; {print nmur2}}' 1

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.