Shulou(Shulou.com)06/02 Report--

Text processor

There are many kinds of text processors or text editors in the Linux/UNIX system, including VIM editors and grep that we have learned before. Grep, sed and awk are the text processing tools often used in shell programming, which are called the three Musketeers of Shell programming.

Sed tool

Sed (Stream EDitor) is a powerful and simple text parsing conversion tool, which can read the text and edit the text content (delete, replace, add, move, etc.) according to the specified conditions, and finally output all lines or only some of the lines processed. Sed can also implement complex text processing operations without interaction, and it is widely used in Shell scripts to complete a variety of automatic processing tasks.

The workflow of sed mainly includes three processes: reading, executing and displaying.

Read: sed reads a line from the input stream (file, pipe, standard input) and stores it in a temporary buffer (also known as pattern space, pattern space) execution: by default, all sed commands are executed sequentially in the mode space, unless the address of the line is specified, the sed command will be executed on all lines in turn to display: send the modified content to the output stream. After sending the data, the schema space will be emptied until all the contents of the file are processed, and the above process will be repeated until all the contents have been processed. By default, all sed commands are executed in the schema space, so the input file will not change unless you use redirect storage output sed command common usage.

The sed command has two formats:

Sed [option] 'Operation' parameter / / "parameter" refers to the target file of the operation, which is used when there are multiple operands, with commas between the files "," delimited or sed [option]-f scriptfile parameter / / scriptfile indicates the script file, and needs to be specified with the "- f" option.

Common sed command options

-e or-- expression=: means to process the input text file with a specified command or script-f or-- file=: means to process the input text file with the specified script file-h or-- help: show help-n,-- quiet or silent: show only the processed results-I: directly edit the text file "operation" is used to specify the action behavior on the file operation, that is, the command of sed. The format of the "[N1 [, N2]]" operation parameter is usually used. N1 and N2 are optional and do not necessarily exist, which represent the number of rows selected for operation. If the operation needs to be carried out between 5 and 20 lines, it is expressed as "5jin20 action behavior". Common operations include the following a: add, add a line below the current line to specify c: replace, replace the selected line with the specified d: delete, delete the selected line I: insert, insert a specified line above the selected line p: print, if you specify a line at the same time, it means to print the specified line; if you do not specify a line, it means to print everything; if there are non-print characters, then output with ASCII code. It usually uses s: replace with the "- n" option to replace the specified character y: character conversion example

1) output qualified text (p means normal output)

[root@localhost opt] # sed-n 'p' httpd.txt / / all the contents of the output file, equivalent to cat httpd.txt## This is the main Apache HTTP server configuration file. It contains the# configuration directives that give the server its instructions.# See for detailed information.# In particular, see # # for a discussion of each configuration directive.## Do NOT simply read the instructions in here without understanding# what they do. They're here only as hints or reminders. If you are unsure# consult the online docs. You have been warned. # # Configuration and logfile names: If the filenames you specify for many# of the server's control files begin with "/" (or "drive:/" for Win32), the# server will use that explicit path. If the filenames do * not* begin# with "/", the value of ServerRoot is prepended-- so 'log/access_log'# with ServerRoot set to' / www' will be interpreted by the# server as'/ www/log/access_log' Where as'/ log/access_log' will be...// omits part of the content. [root@localhost opt] # sed-n '3p' httpd.txt / / output Line 3 content # configuration directives that give the server its instructions. [root@localhost opt] # sed-n' 3Lines 5p' httpd.txt / / output 3 lines # configuration directives that give the server its instructions.# See for detailed information.# In particular, see [root@localhost opt] # sed-n'p N 'httpd.txt / / outputs all odd lines, n means reading the next line of data # # configuration directives that give the server its instructions.# In particular, see # for a discussion of each configuration directive.# Do NOT simply read the instructions in here without understanding# consult the online docs. You have been warned. # Configuration and logfile names: If the filenames you specify for many# server will use that explicit path. If the filenames do * not* begin# with ServerRoot set to'/ www' will be interpreted by the# interpreted as'/ log/access_log'....// omits part of the content. [root@localhost opt] # sed-n 'nposip' httpd.txt / / output all even lines # This is the main Apache HTTP server configuration file. It contains the# See for detailed information.# what they do. They're here only as hints or reminders. If you are unsure## of the server's control files begin with "/" (or "drive:/" for Win32), the...// omits part of the content. [root@localhost opt] # sed-n'1 httpd.txt 5 {ptwitn} 'httpd.txt / / output odd lines (lines 1, 3, 5) # # configuration directives that give the server its instructions.# In particular, see [root@localhost opt] # sed-n' 350 ${n P} 'httpd.txt / / the even-numbered line from line 350 to the end of the file # IncludeOptional conf.d/*.confshortwodwooooodAxyzxyzC// when executing this command, the first line read is line 350, the second line read is line 351, and so on, so the even-numbered line output is line 351, line 353 until the end of the file, including the blank line sed command combined with regular expressions, the format is slightly different The regular expression surrounds [root@localhost opt] # sed-n'/ the/p' httpd.txt / / with "/" to output the line # This is the main Apache HTTP server configuration file that contains the. It contains the# configuration directives that give the server its instructions.# Do NOT simply read the instructions in here without understanding# what they do. They're here only as hints or reminders. If you are unsure# consult the online docs. You have been warned. # Configuration and logfile names: If the filenames you specify for many# of the server's control files begin with "/" (or "drive:/" for Win32), the# server will use that explicit path. If the filenames do * not* begin# with "/", the value of ServerRoot is prepended-- so 'log/access_log'# with ServerRoot set to' / www' will be interpreted by the [root@localhost opt] # sed-n'4 See for detailed information.# In particular from line 4 to the first line containing the See # # for a discussion of each configuration directive.## Do NOT simply read the instructions in here without understanding [root@localhost opt] # sed-n'/ the/=' httpd.txt / / outputs the line number of the line containing the The equal sign (=) is used to output the line number 23910111314. Sed / omit part of the content. [root@localhost opt] # sed-n'/ ^ sh/p' httpd.txt / / output the line beginning with sh [root@localhost opt] # sed-n'/ [0-9] $/ p 'httpd.txt / / output the line # Listen 12.34.56.78:80Listen 80#ServerName www.example.com:80AddDefaultCharset UTF-8 ending with a number [ Root@localhost opt] # sed-n'/\ / p 'httpd.txt / / output the line containing the word wood \ represents word boundary wood

2) Delete eligible text (d)

The nl command in the following command is used to calculate the number of lines in the file, and combined with this command, you can see the result of the command execution more intuitively. [root@localhost opt] # nl httpd.txt | sed '3d' / / Delete line 3 1 u 2 # This is the main Apache HTTP server configuration file. It contains the 4 # See for detailed information. 5 # In particular, see 6 # 7 # for a discussion of each configuration directive....// omit part of the content. [root@localhost opt] # nl httpd.txt | sed '3Magazine 5d' / / Delete 3 times 5 lines 1 u 2 # This is the main Apache HTTP server configuration file. It contains the 6 # 7 # for a discussion of each configuration directive. . / / omit part of the content. [root@localhost opt] # nl httpd.txt | sed'/ wood/d' / / delete the line containing wood, the original line 321 is deleted 1 u / / delete the line that does not contain cross, use! The symbol denotes an inverse operation, such as'/ crossmap! D'2 # This is the main Apache HTTP server configuration file. It contains the 3 # configuration directives that give the server its instructions. . / / omit part of the content. 318 short 319 wd 320 wod 322 woooood 323 AxyzC 324 AxyzxyzC [root@localhost opt] # sed'/ ^ [Amurz] / d'httpd.txt / / delete the line # This is the main Apache HTTP server configuration file that begins with a lowercase letter. It contains the# configuration directives that give the server its instructions....// omitted part of the content. # Load config files in the "/ etc/httpd/conf.d" directory, if any.IncludeOptional conf.d/*.confAxyzCAxyzxyzC [root@localhost opt] # sed'/\. $/ d' httpd.txt / / delete to "." The line u # This is the main Apache HTTP server configuration file at the end. It contains the# In particular, see #... / / omit part of the content. [root@localhost opt] # sed'/ ^ $/ d 'httpd.txt / / delete all blank lines u # This is the main Apache HTTP server configuration file. It contains the# configuration directives that give the server its instructions.# See for detailed information.# In particular, see # # for a discussion of each configuration directive.// Note: if you delete duplicate blank lines, that is, only one consecutive blank line is left, this can be achieved by executing the command "sed-e'/ ^ $/ {nten / ^ $/ d} 'httpd.txt". The effect is the same as "cat-s test.txt", where n means to read the next row of data.

3) replace qualified text

When using the sed command to replace, you need to use s (string substitution), c (whole line / block substitution), y (character conversion) command option [root@localhost opt] # sed's go to the the 'httpd.txt / / replace the first string in each line with THEu# This is THE main Apache HTTP server configuration file. It contains the# configuration directives that give THE server its instructions.# See for detailed information.# In particular See. / / omit some content. [root@localhost opt] # vim aaa.txt / / Edit a new file llllllllllllllllll / / Edit content llllllllllllllllll~ ~ : wq / / Save and exit [root@localhost opt] # sed's UniqqLAccord 3' aaa.txt / / replace the third l in each line with LllLlllllLlll / / display the replaced content llLlllllLlllllLlllllLlll [root@localhost opt] # Sed's httpd.txt the httpd.txt / / replace the most common the in the file with THEu# This is THE main Apache HTTP server configuration file. It contains THE# configuration directives that give THE server its instructions.# See for detailed information.# In particular, see... / / omit part of the content. [root@localhost opt] # sed's httpd.txt / / delete all o in the file (replace the empty string) u # This is the main Apache HTTP server cnfiguratin file. It cntains the# cnfiguratin directives that give the server its instructins....// omits part of the content... shirtshrinking wdwdwdAxyzCAXYZC [root @ localhost opt] # sed's / ^ / # / 'aaa.txt / / insert the number # llllll#llllll#llllll#llllll#llllll#llllll [root@localhost opt] # sed's end of each line with the string eoflllllleoflllllleoflllllleoflllllleoflllllleoflllllleof [root@localhost opt] # vim aaa.txt at the end of each line / / Edit file llllllllllllllllllllllllllllllllllllaaaaaaaaaaaa / / add content aaaaaaaaaaaaaaaaaa~ ~: wq / / Save and exit [root@localhost opt] # sed'/ a/s/ ^ / # / 'aaa.txt / / insert # llllllllllllllllllllllllllllllllllll#aaaaaa#aaaaaa#aaaaaa#aaaaaa#aaaaaa [root@localhost opt] # sed' 3 at the beginning of each line containing a / replace all l in line 3 / 5 with LllllllllllllLLLLLLLLLLLLLLLLLLllllllaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa [root@localhost opt] # vim bbb.txt / / Edit a new file this isthe woodwoodwod / / Edit content the woodthis is test~ ~: wq / / Save exit [root@localhost opt] # sed'/ the/s/o/O/g' bbb.txt / / replace o with Othis isthe wOOdwoodwodthe wOOdthis is test in all lines containing the

4) migrate eligible text

H, copy to the clipboard; g, G, overwrite / append the data from the clipboard to the specified line; w, save as a file; r, read the specified file; a, append the specified content. [root@localhost opt] # sed'/ the/ {Htterd}; $G'bbb.txt / / move the line containing the to the end of the file, {;} for multiple operations this iswoodwodthis is testthe woodthe wood [root@localhost opt] # sed '1recover3 {Hteriday} 8G 'aaa.txt / / after migrating 1' 3 lines to 8 lines, llllllllllllllllllaaaaaaaaaaaallllllllllllllllllaaaaaaaaaaaaaaaaaa [root@localhost opt] # sed'/ the/w ccc.txt' bbb.txt / / Save the line containing the as a file ccc.txtthis isthe woodwoodwodthe woodthis is test [root@localhost opt] # cat ccc.txt / / View the saved file contents the woodthe wood [root@localhost opt] # sed'/ the/r / etc/hostname' bbb.txt This is / / add the contents of the file / etc/hostname to each line containing the the woodlocalhost.localdomainwoodwodthe woodlocalhost.localdomainthis is test [root@localhost opt] # sed '3aNEW' bbb.txt / / insert a new line after line 3 The content is NEWthis isthe woodwoodNEWwodthe woodthis is test [root@localhost opt] # sed'/ the/aNEW' bbb.txt / / insert a new line after each line containing the, and the content is NEWthis isthe woodNEWwoodwodthe woodNEWthis is test [root@localhost opt] # sed '3aNEW\ nNEW2' bbb.txt / / insert multiple lines after line 3, with\ n in the middle indicating newline this isthe woodwoodNEWNEW2wodthe woodthis is test

5) use scripts to edit files

Using the sed script, multiple editing instructions are stored in a file (one for each line) and called through the "- f" option. Sed'1 vim test1,3H1,3d6G~ 3 {Hutterd}; 6G 'bbb.txt / / migrate the contents of 3 lines from 1x to 6 lines after [root@localhost opt] # vim test1,3H1,3d6G~: wq [root@localhost opt] # sed-f test bbb.txt wodthe woodthis is testthis isthe woodwood

6) sed direct manipulation file example

Write a script to adjust the vsftpd service configuration: anonymous users are prohibited, but local users are allowed (and writes are also allowed). [root@localhost ~] # * * vim local_only_ftp.sh** #! / bin/bash# specify the sample file path and the configuration file path SAMPLE= "/ usr/share/doc/vsftpd-3.0.2/EXAMPLE/INTERNET_SITE/vsftpd.conf" CONFIG= "/ etc/vsftpd/vsftpd.conf" # back up the original configuration file and check whether the backup file named / etc/vsftpd/vsftpd.conf.bak exists If it does not exist, use the cp command to backup the file [!-e "$CONFIG.bak"] & & cp $CONFIG $CONFIG.bak / / adjust based on the sample configuration Overwrite the existing file sed-e'/ ^ anonymous_enable/s/YES/NO/g' $SAMPLE > $CONFIGsed-I-e'/ ^ local_enable/s/NO/YES/g'-e'/ ^ write_enable/s/NO/YES/g' $CONFIG grep "listen" $CONFIG | | sed-I'$alisten=YES' $CONFIG# start the vsftpd service and set it to run systemctl restart vsftpdsystemctl enable vsftpd [root@localhost ~] # * * chmod + x local_only_ftp.sh automatically after boot

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.