Shulou(Shulou.com)06/02 Report--

Regular expressions of shell script programming (2) 1. Preface

​ this article is mainly an introduction to extended regular expressions, while continuing to introduce sed text processing tools in the style of the previous article. Sed, as one of the "three musketeers" in shell programming, plays a great role in text processing.

For examples of regular concepts and the use of grep commands in conjunction with regularities, please refer to: https://blog.51cto.com/14557673/2455588

Second, extended regular expressions

​ extended regular expressions are mainly produced to simplify instructions. For example, use the basic regular expression to query the blank line in the file and the line other than the # sign (usually used to view the configuration that takes effect), execute "grep-v'^ $'test.txt | grep-v' ^ #", while using the extended regular expression, it can be simplified to "egrep-v'^ $| ^ # 'test.txt". Because grep only supports basic regular expressions, you need to use the egrep or awk command if you want to use extended regular expressions. The next article will explain the awk command.

The ​ egrep command is a search file acquisition pattern that allows you to search for any string and symbol in a file, or to search for a string of one or more files. A prompt can be a single character, a string, a word, or a sentence.

​ extended regular expression common meta-character table

Metacharacter function and example + function: repeat one or more previous characters example: execute the "egrep-n 'wo+d' test.txt" command, you can query "wood", "woood", "woooooood" and other strings? Function: the first character of zero or one example: execute the "egrep-n 'bes?t' test.txt" command Can query "bet" and "best" these two strings | function: use or (or) to find multiple characters example: execute "egrep-n'of | is | on' test.txt" command to query "of" or "if" or "on" string () function: find "group" string example: "egrep-n't (a | e) st' test.txt". " Tast "and" test "because the" t "and" st "of these two words are repeated," a "and" e "are listed in the" () "symbol and separated by" | "to query the" tast "or" test "string () + function: identify multiple duplicate groups example:" egrep-n'A (xyz) + C 'test.txt ". The command begins with "A" and ends with "C", with more than one "xyz" string in the middle. III. Text processor-- introduction to sed tool

Sed (Stream Editor) is a powerful and simple text parsing conversion tool, which can read the text and edit the text content (delete, replace, add, move, etc.) according to the specified conditions, and finally output all lines or only some of the lines processed. Sed can also implement quite complex text processing operations without interaction, and it is widely used in Shell scripts to complete a variety of automatic processing tasks.

The workflow of ​ sed mainly includes three processes: read, execute and display.

Read: sed reads line contents and stores them in a temporary buffer (or pattern space, pattern space)

Execution: executes sequentially in the mode space by default, unless the address of the row is specified.

Display: sends the modified content to the output stream. After the data is sent again, the schema space will be emptied.

Note: by default, all sed commands are executed in the schema space, so the input file will not change unless you use redirection to store the output.

Usage: sed [option] 'Operation' parameter or sed [option]-f scriptfile parameter

Options:

-e or-- expression=: means to process the input text file with a specified command or script. -f or-- file=: means to process the input text file with the specified script file. -h or-- help: displays help. -n,-- quiet, or silent: indicates that only the processed results are displayed. -I: edit the text file directly.

Action:

A: increase by adding a specified line below the current line. C: replace, replacing the selected line with the specified content. D: delete, delete the selected row. I: insert, inserting a specified line above the selected row. P: print, if you specify a line at the same time, it prints the specified line; if you don't specify a line, it prints everything; if there are non-printed characters, it is output in ASCII code. It is usually used with the "- n" option. S: replace, replace the specified character. Y: character conversion. IV. Detailed examples of the usage of sed

1) output qualified text (p means normal output)

[root@lokott opt] # sed-n 'p'test.txt / / is equivalent to cat test.txthe was short and fat.He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online.the tongue is boneless but it breaks bones.12! Google is the best tools for search keyword.The year ahead will test our political establishment to the limit.PI=3.141592653589793238462643383249901429a wood cross!Actions speak louder than words#woood # # woooooood # AxyzxyzxyzxyzCI bet this place is really spooky late at night! Misfortunes never come alone/single.I shouldn't have lett so tast. [root@lokott opt] # cat test.txt | wc-L17 [root@lokott opt] # sed-n'p' test.txt | wc-L17 [root@lokott opt] # sed-n' 2p' test.txt / / displays the second line He was wearing a blue polo shirt with black pants. [root@lokott opt] # sed-n'2 He was wearing a blue polo shirt with black pants 5p 'test.txt / / displays 2-5 lines of He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online.the tongue is boneless but it breaks bones.12percent Google is the best tools for search keyword. [root@lokott opt] # sed-n'nscape p 'test.txt / / Loop display, skip one line, display another line, and so on He was wearing a blue polo shirt with black pants. The tongue is boneless but it breaks bones.12 the year ahead will test our political establishment to the limit.a wood crossroads woody # AxyzxyzxyzxyzCMisfortunes never come alone/single. [root@lokott opt] # sed-n'pbot n 'test.txt / / loop display, first show a line, then skip a line, and then analogy he was short and fat.The home of Football on BBC Sport online. Google is the best tools for search keyword.PI=3.141592653589793238462643383249901429Actions speak louder than words#woooooood # I bet this place is really spooky late at night! I shouldn't have lett so tast. [root@lokott opt] # nl test.txt / / displays the line number and lists the text content. For demonstration purposes, the line number is not the same as the actual line number 1 he was short and fat. 2 He was wearing a blue polo shirt with black pants. 3 The home of Football on BBC Sport online. 4 the tongue is boneless but it breaks bones.12! 5 google is the best tools for search keyword. 6 The year ahead will test our political establishment to the limit. 7 PI=3.141592653589793238462643383249901429 8 a wood cross! 9 Actions speak louder than words 10 # woood # 11 # woooooood # 12 AxyzxyzxyzxyzC 13 I bet this place is really spooky late at night! 14 Misfortunes never come alone/single. 15 I shouldn't have lett so tast. [root@lokott opt] # sed-n'2 He was wearing a blue polo shirt with black pants 5 {ptertn} 'test.txt displays 2 and 4 lines of He was wearing a blue polo shirt with black pants. The tongue is boneless but it breaks bones.12! [root@lokott opt] # sed-n'2 test.txt / / 2 to the last line interlaced display He was wearing a blue polo shirt with black pants. The tongue is boneless but it breaks bones.12!The year ahead will test our political establishment to the limit.a wood cross!#woood # AxyzxyzxyzxyzCMisfortunes never come alone/single.

When sed is used in conjunction with regular expressions, the format is slightly different. Regular expressions must be surrounded by "/". Examples are as follows:

[root@lokott opt] # sed-n'/ the/p' test.txt / / outputs the line the tongue is boneless but it breaks bones.12 that contains the! Google is the best tools for search keyword.The year ahead will test our political establishment to the limit. [root@lokott opt] # sed-n'4 google is the best tools for search keyword p' test.txt / / here it will be displayed regardless of whether there is a the in the fourth line, because the meaning of the command with the tongue is boneless but it breaks bones.12! / meaning is everything from the fourth line to the first the, the fourth line google is the best tools for search keyword. / / the does not count as [root@lokott opt] # sed-n'4 / the/p' test.txt / / changed the text presentation effect tahe tongue is boneless but it breaks bones.12 Google is the best tools for search keyword. [root@lokott opt] # sed-n'/ the/=' test.txt / / displays the line number 56 [root@lokott opt] # sed-n'/ ^ PI/p' test.txt / / showing that lines starting with PI can use grep'^ PI' test.txt instead of PI=3.141592653589793238462643383249901429 [root@lokott opt ] # sed-n'/ [0-9] $/ p 'test.txt / / displays the line ending with a number PI=3.141592653589793238462643383249901429 [root@lokott opt] # sed-n' /\ / p 'test.txt / / outputs the line containing the word wood \

Represents the word boundary a wood cross!

2) Delete eligible text (d)

[root@lokott opt] # nl test.txt | sed '3d' / / Delete the third line 1 he was short and fat. 2 He was wearing a blue polo shirt with black pants. 4 tahe tongue is boneless but it breaks bones.12! 5 google is the best tools for search keyword. 6 The year ahead will test our political establishment to the limit. 7 PI=3.141592653589793238462643383249901429 8 a wood cross! 9 Actions speak louder than words 10 # woood # 11 # woooooood # 12 AxyzxyzxyzxyzC 13 I bet this place is really spooky late at night! 14 Misfortunes never come alone/single. 15 I shouldn't have lett so tast. [root@lokott opt] # nl test.txt | sed '3pm 5d' / / Delete 3-5 lines 1 he was short and fat. 2 He was wearing a blue polo shirt with black pants. 6 The year ahead will test our political establishment to the limit. 7 PI=3.141592653589793238462643383249901429 8 a wood cross! 9 Actions speak louder than words 10 # woood # 11 # woooooood # 12 AxyzxyzxyzxyzC 13 I bet this place is really spooky late at night! 14 Misfortunes never come alone/single. 15 I shouldn't have lett so tast. [root@lokott opt] # nl test.txt | sed'/ cross/d' / / Delete the line containing cross 1 he was short and fat. 2 He was wearing a blue polo shirt with black pants. 3 The home of Football on BBC Sport online. 4 tahe tongue is boneless but it breaks bones.12! 5 google is the best tools for search keyword. 6 The year ahead will test our political establishment to the limit. 7 PI=3.141592653589793238462643383249901429 9 Actions speak louder than words 10 # woood # 11 # woooooood # 12 AxyzxyzxyzxyzC 13 I bet this place is really spooky late at night! 14 Misfortunes never come alone/single. 15 I shouldn't have lett so tast. [root@lokott opt] # sed'/ ^ [a murz] / d'test.txt | nl / / delete all lines starting with lowercase letters 1 He was wearing a blue polo shirt with black pants. 2 The home of Football on BBC Sport online. 3 The year ahead will test our political establishment to the limit. 4 PI=3.141592653589793238462643383249901429 5 Actions speak louder than words 6 # woood # 7 # woooooood # 8 AxyzxyzxyzxyzC 9 I bet this place is really spooky late at night! 10 Misfortunes never come alone/single. 11 I shouldn't have lett so tast. [root@lokott opt] # sed'/\. $/ d 'test.txt / / delete the line He was wearing a blue polo shirt with black pants that ends with a dot. Tahe tongue is boneless but it breaks bones.12 blank he was short and fat.He was wearing a blue polo shirt with black pants 3.141592653589793238462643383249901429a wood crossroads actions speak louder than words#woood # # woooooood # AxyzxyzxyzxyzCI bet this place is really spooky late at night! [root@lokott opt] # sed'/ ^ $/ d 'test.txt / delete all blank lines. The home of Football on BBC Sport online.tahe tongue is boneless but it breaks bones.12!google is the best tools for search keyword.The year ahead will test our political establishment to the limit.PI=3.141592653589793238462643383249901429a wood cross!Actions speak louder than words#woood # # woooooood # AxyzxyzxyzxyzCI bet this place is really spooky late at night! Misfortunes never come alone/single.I shouldn't have lett so tast.

3) replace qualified text

​ needs to use the s (string substitution), c (whole line / block substitution), and y (character conversion) command options when using the sed command for replacement operations. Common uses are as follows.

Sed's test.txt the test.txt / / replaces the first the in each line with THE

Sed's Placement LAccord 2' test.txt / / replace the second l in each line with L

Sed's test.txt the test.txt / / replaces all the in the file with THE

Sed's ActualPlacement 'test.txt** / / delete all o in the file (replace with an empty string)

Sed's / ^ / # / * * 'test.txt** / / insert a # sign at the beginning of each line

Sed'/ the/s/ ^ / # / 'test.txt / / insert the # sign at the beginning of each line containing the

Sed's Universe test.txt 'test.txt / / insert the string EOF at the end of each line

Sed'3 the test.txt test.txt / / replace all the in line 3 and 5 with THE

Sed'/ the/s/o/O/g**' test.txt** / / replace o with O in all lines that contain the

4) migrate eligible text

Among them, H, copy to the clipboard; g, G, overwrite / append the data in the clipboard to the specified line; w, save as a file; r, read the specified file; a, append the specified content.

Sed'/ the/ {Hutterd}; $G' test.txt / / move the line containing the to the end of the file, {;} for multiple operations

Sed'1 test.txt 5 {Hutchd}; 17G'G` / / after transferring the contents of line 1 to line 5 to line 17

Sed'/ the/w out.file' test.txt / / Save the line containing the as a file out.file

Sed'/ the/r / etc/hostname' test.txt / / after adding the contents of the file / etc/hostname to each line that contains the

Sed '3aNew' test.txt / / insert a new line after line 3 with the content New

Sed'/ the/aNew' test.txt / / insert a new line with the content New after each row that contains the

Sed '3aNew1\ nNew2' test.txt / / insert multiple lines after line 3, with\ n in the middle indicating line break

To be continued.)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.