Shulou(Shulou.com)06/01 Report--

This article is to share with you about how to use sed in the Linux system. The editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article.

I. introduction to Sed

Sed is a stream editing tool for filtering and replacing text, especially when you want to make unified changes to dozens of configuration files, you will feel the charm of Sed! Sed processes some instructions and outputs them by reading only one line at a time, so Sed is more suitable for processing big data files. First of all, Sed reads the contents of the file through files or pipes, but Sed does not modify the source file directly by default, but copies the read contents into the buffer, which we call pattern space (pattern space). All instruction operations are carried out in the mode space, and then Sed processes the contents of the pattern space and outputs the results according to the corresponding instructions, and defaults to standard output (that is, on the screen). The workflow of Sed is shown in the following figure:

II. Basic grammatical format of Sed

Sed reads data from the file. If there is no input file, the standard input process data is processed by default. The script instruction is the first parameter that does not start with "-". The syntax format is as follows:

Sed [options] {script instructions} [input File] option

Meaning

Version

Show sed version

Help

Show help documentation

-nomenclature, people, etc.

Silent output, by default, the sed program will automatically print the contents of the mode space after all the script instructions have been executed. This option can mask automatic printing.

-e script

Allow multiple script instructions to be executed

-f script-file

Reading script instructions from a file is useful for writing automatic scripts.

-iPermanent in place

Use with caution, this option will modify the source file directly

-LBI N

This option directs the line length that the l instruction can output, and the l instruction outputs non-print characters.

Posix

Disable GNU sed extension functionality

-r

Using extended regular expressions in script instructions

-Scripture

By default, sed will use multiple file names entered as a long, continuous input stream, while GNU sed allows them to be treated as separate files

-uther unbuffered

Minimum cached inputs and outputs

III. Introduction to SED specification 1. Basic format specification

Sed processes files through specific script instructions. Here we briefly introduce several script instruction operations as the specification of Sed programs. A quotation append indicates an append instruction; iMagery insert indicates an insert instruction; dline delete indicates a delete instruction; and squoise indicates a replacement instruction. The basic format of Sed script instructions is [address] command (some commands can only operate on one line, some can operate on multiple lines), and commands can also be combined with curly braces, so that the command sequence can act on the same address:

Address {command1command2command3} Note: the first command can be on the same line as the left curly bracket, but the right curly bracket must be on a separate line. In addition, adding spaces after the command results in an error. The following test.txt describes the use of Sed for the operation sample source file (note that there are several blank lines):

[root@andrew Andrew] # cat-n test.txt

1 DEVICE=eno16777736

2 BOOTPROTO=static

3 IPADDR=192.168.0.1

4 NETMASK=255.255.255.0

5 GATEWAY=192.168.0.254

6 ONBOOT=yes

Append TYPE=Ethernet after the second line:

[root@andrew Andrew] # sed'2a TYPE=Ethernet' test.txt DEVICE=eno16777736BOOTPROTO=staticTYPE=EthernetIPADDR=192.168.0.1NETMASK=255.255.255.0

GATEWAY=192.168.0.254

ONBOOT=yes

Append TYPE=Network before the third line:

[root@andrew Andrew] # sed'3i TYPE=Network' test.txt DEVICE=eno16777736BOOTPROTO=staticTYPE=NetworkIPADDR=192.168.0.1NETMASK=255.255.255.0

GATEWAY=192.168.0.254

ONBOOT=yes

Replace all yes in the sample file with no:

[root@andrew Andrew] # sed's Universe no test.txt DEVICE=eno16777736BOOTPROTO=staticIPADDR=192.168.0.1NETMASK=255.255.255.0

GATEWAY=192.168.0.254

ONBOOT=no

Most of the above operation instructions locate the operation object (address) according to the line number, such as 2a, which is appended after the second line. In practice, in most cases you are not sure the line number of the object (address) you want to operate on, so you are more likely to use regular expressions to determine the object (address). The following is an example of using regular expressions to locate action lines:

Match to the line that contains ONBOOT, and add TYPE=Ethernet after that:

[root@andrew Andrew] # sed'/ ONBOOT/a TYPE=Ethernet' test.txt DEVICE=eno16777736BOOTPROTO=staticIPADDR=192.168.0.1NETMASK=255.255.255.0

GATEWAY=192.168.0.254

ONBOOT=yesTYPE=Ethernet

Match the line starting with GATEWAY and delete the line:

[root@andrew Andrew] # sed'/ ^ GATEWAY/d' test.txt DEVICE=eno16777736BOOTPROTO=staticIPADDR=192.168.0.1NETMASK=255.255.255.0

ONBOOT=yes

In addition, our operation instructions can be written to the script file and read through the-f option of sed. The comment line in the script file starts with #. If the character after # is n, the automatic output function of the Sed program is masked, which is equivalent to the command option-n. Create a sed script with the following content:

[root@andrew Andrew] # cat sed.sh#This is a test sed command# script content: after matching to a blank line, delete the line / ^ $/ d and execute the sed.sh script instruction to the test.txt file:

[root@andrew Andrew] # sed-f sed.sh test.txt DEVICE=eno16777736BOOTPROTO=staticIPADDR=192.168.0.1NETMASK=255.255.255.0GATEWAY=192.168.0.254ONBOOT=yes

When you need to execute multiple instructions, you can use the following three methods:

(1) use semicolons to separate instructions

[root@andrew Andrew] # sed's Unix no'r 'ma 'r 'ma 'ma 'r 'st 'r 'ma 'ma 'r 'r 'ma 'r 'e 'r

GATEWAY=192.168.0.254

ONBOOT=no

(2) use the-e option

[root@andrew Andrew] # sed-e's Universe no test.txt DEVICE=eno16777736BOOTPROTO=dhcpIPADDR=192.168.0.1NETMASK=255.255.255.0'- e's Grease DHCP'

GATEWAY=192.168.0.254

ONBOOT=no

(3) make use of branches

[root@andrew Andrew] # sed'

Splash, no, no.

However, it is foolish to enter excessively long instructions on the command line, so you need to use the-f option to specify the sed script file, which can contain multiple lines of instructions and is easy to modify.

two。 Example of operation address matching

From the above example, it is not difficult to find that the script instructions we write need to specify an address to determine the scope of the operation, and if not, all lines of the file will be operated by default. For example, sed d test.txt deletes all rows of test.txt, while '2d' deletes only the second row. Sed provides us with the following ways to determine the range of addresses we need to operate.

-number

Specify the unique line number of the input file

-first~step

Specify to start with first and specify the operation step as step, such as 1, 2, specify the first, third, and fifth lines. Is the operation address. 2. 5 specify the start of the second line and match the operation address every 5 lines

$

Matches the last line of the file

/ regexp/

/ / contains a regular expression that matches the operation address. If the regular expression in / / is empty, it matches the matching address of the last regular expression, followed by an example

\ cregexpc

The regular expression is matched between\ c and c, and the c character can be replaced by any character.

Addr1,addr2

Match all lines from operation address 1 to operation address 2

Addr1,+N

Match address 1 and the next N lines

Odd lines of the print file:

[root@andrew Andrew] # cat-n test.txt 1 DEVICE=eno16777736 2 BOOTPROTO=static 3 IPADDR=192.168.0.1 4 NETMASK=255.255.255.0 5 6 GATEWAY=192.168.0.254 7 8 ONBOOT=yes [root@andrew Andrew] # sed-n '1thanks 2p' test.txt DEVICE=eno16777736IPADDR=192.168.0.1

Delete all lines between 2x8:

[root@andrew Andrew] # cat-n test.txt 1 DEVICE=eno16777736 2 BOOTPROTO=static 3 IPADDR=192.168.0.1 4 NETMASK=255.255.255.0 5 6 GATEWAY=192.168.0.254 7 8 ONBOOT=yes [root@andrew Andrew] # sed'2 test.txt DEVICE=eno16777736

IV. Summary of Sed instructions and scripts 1.Sed common instructions

The following table gives a description of the common sed script instructions. Let's take a look at the detailed usage of each instruction:

Instruction function

-s replacement

-d Delete

-an append

-I insert

-c change

-l print (display non-print characters)

-y convert by character

-L print (does not display non-print characters)

-p print

-r read the contents of the file

-w Save to file

-Q exit

two。 Detailed explanation of some instructions

(1) replacement instruction (sbody substitution)

Instruction format: [address] s/pattern/replacement/flags

Address is the operation address, s is the replacement instruction, / pattern/ matches the content to be replaced, and / replacement is the new content to be replaced. The Flags tag can be:

Flags marker

Meaning

-n

A number between 1 and 512 that replaces the nth occurrence of a specified pattern in pattern space. If there are three A's in a line, you only want to replace the second A.

-g

Make global changes to all matches in the pattern space. Without g, only the first match is replaced, and if there are three An in a row, only the first An is replaced.

-p

Print the contents of the mode space

-w file

Write the contents of the schema space to the file file

Replacement is a string that replaces what matches the regular expression. In the replace section, only the following characters have a special meaning:

&

Replace with what the regular expression matches

\ n

Match the nth substring, which was previously specified in pattern with\ (\)

\

Escape (the escape replacement part contains: &,\, etc.)

(2) Delete instruction (dfocus delete)

The delete directive is used to delete matching lines, and deleting the command also changes the order in which the commands are executed in the sed script. Because once the matching line is deleted, the pattern space will become "empty" and the subsequent commands of the sed script will not be executed. Deleting the command causes the new input line (the next line) to be read, while the command in the sed script is reexecuted from scratch. It is important to note that when deleting, you delete the entire line, not just the matching content (if you delete the matching content, you can use replace)

(3) conversion instruction (y)

The syntax format of Transform is [address] y/source-chars/dest-chars, where [address] is used to locate the line that needs to be modified, source-chars is the character to be modified, and dest-chars is the character to be replaced.

Sample 3.Sed script instruction

(1) example 1

The sample files used in example 1 are:

[root@andrew Andrew] # cat test.txt

Hello the World!

Example 1: replace the second one in the sample file with

Write a sed script to replace the same content that matches the line, replacing body with / body, but only replacing the second body with / body.

[root@andrew Andrew] # cat sed.sh / body/ {SQL /\ / body/2}

The result of executing the sed program is as follows:

[root@andrew Andrew] # sed-f sed.sh test.txt

Hello the World!

(2) example 2

The sample files used in example 2 are:

[root@andrew Andrew] # cat test.txt

H2Helloh2h3Helloh3h4Helloh4

Example 2: add to all the first h 2 quotes, h 3, etc., and the second h 2 meme h 3 >.

Write a sed script as follows:

[root@andrew Andrew] # cat sed.sh / h [0-9] / {1s// & > / / 2}

Description: match h immediately follows the line of a number, replacing the same content as the matching content in the previous line, that is, h [0-9] is replaced by, where & is the content to be replaced before. The first s replacement instruction only replaces the first h _ 2p h _ 3. The second s replacement instruction is used to replace the second h _ 2 ~ H _ 3.

The result of executing the sedc program is as follows:

[root@andrew Andrew] # sed-f sed.sh test.txt

HelloHelloHello tip: another nice thing about the'sAccord 'command is that there are many alternatives to the'/ 'delimiter, and if there are many slashes in the regular expression or replacement string, you can change the delimiter by specifying a different character after 's'. Example: sed-e's usr/local usr/local sed sed will replace / usr/local with / usr.

(3) example 3

The sample file used in example 3 is (note that there are blank lines):

[root@andrew Andrew] # cat test.txt DEVICE=eno16777736ONBOOT=yesBOOTPROTO=static

IPADDR=192.168.0.1NETMASK=255.255.255.0GATEWAY=192.168.0.254

Example 3: delete blank lines in the file.

Write a sed script as follows:

[root@andrew Andrew] # cat sed.sh /. * / {/ ^ $/ d}

The result of executing the sed program is as follows:

[root@andrew Andrew] # sed-f sed.sh test.txt DEVICE=eno16777736ONBOOT=yesBOOTPROTO=staticIPADDR=192.168.0.1NETMASK=255.255.255.0GATEWAY=192.168.0.254

(4) example 4-example 6

Example 4-the sample file used in example 7 is:

[root@andrew Andrew] # cat test.txt DEVICE=eno16777736ONBOOT=yesBOOTPROTO=staticNETMASK=255.255.255.0GATEWAY=192.168.0.254

Example 4: add a line after the static line with the content IPADDR=192.168.0.1.

[root@andrew Andrew] # sed'/ static/an IPADDR=192.168.0.1' test.txt DEVICE=eno16777736ONBOOT=yesBOOTPROTO=staticIPADDR=192.168.0.1NETMASK=255.255.255.0GATEWAY=192.168.0.254

Example 5: insert the content IPADDR=192.168.0.1 before the line that matches the NETMASK.

[root@andrew Andrew] # sed'/ NETMASK/i IPADDR=192.168.0.1' test.txt DEVICE=eno16777736ONBOOT=yesBOOTPROTO=staticIPADDR=192.168.0.1NETMASK=255.255.255.0GATEWAY=192.168.0.254

Example 6: change the content that contains the ONBOOT line to ONBOOT=no

[root@andrew Andrew] # sed'/ ONBOOT/c ONBOOT=no' test.txt DEVICE=eno16777736ONBOOT=noBOOTPROTO=staticNETMASK=255.255.255.0GATEWAY=192.168.0.254

(5) examples 7 and 8

The sample files used in examples 7 and 8 are:

[root@andrew Andrew] # cat test.txt DEVICE=eno16777736ONBOOT=yesBOOTPROTO=staticnetmask=255.255.255.0GATEWAY=192.168.0.254

Example 7: convert lowercase to uppercase

Write a sed script as follows:

[root@andrew Andrew] # cat sed.sh /. * / {/ netmask/y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/}

The result of executing the sed program is as follows:

[root@andrew Andrew] # sed-f sed.sh test.txt DEVICE=eno16777736ONBOOT=yesBOOTPROTO=staticNETMASK=255.255.255.0GATEWAY=192.168.0.254

Example 8: display the contents of the first and second lines

Print (p): acts like l (print), but does not display non-display characters, usually used with-n

[root@andrew Andrew] # sed-n '1jue 2p' test.txt DEVICE=eno16777736ONBOOT=yes

(6) example 9 and example 10

The sample files used in examples 9 and 10 are:

[root@andrew Andrew] # cat name.txt JacobTomJerry [root@andrew Andrew] # cat mail.txt jacob@gmail.comtom@gmail.comjerry@gmail.com

Example 9: read the contents of the name.txt file first, then read the contents of the mail.txt file.

Write a sed script as follows:

[root@andrew Andrew] # cat sed.sh /. * / {$r mail.txt}

The result of executing the sed program is as follows:

[root@andrew Andrew] # sed-f sed.sh name.txt JacobTomJerryjacob@gmail.comtom@gmail.comjerry@gmail.com

Example 10: exit the sed directive after displaying the first two lines of name.txt content.

[root@andrew Andrew] # sed '2q' name.txt JacobTom

The normal Sed data processing flow of SED advanced applications is to read a line of the document to the schema space, then apply the corresponding Sed instruction to the line, output the line and empty the schema space when the instruction is completed, and cycle into the next line of the document until the end of the document data. However, in the real world, the data may not be so regular, and sometimes we write the data into multiple lines into documents, such as:

Name: Zhang San, mailbox: zhangsan@gmail.com name: Li Si, mailbox: lisi@gmain.com can be seen from the above template file, in fact, every two behavior of a complete record, and at this time if you need to use Sed to deal with the document, you need to human intervention in the Sed workflow.

1. Multi-line operation Next

The Next (N) instruction creates a multiline mode space by reading a new input line and appending it to the existing contents of the pattern space. The original content of the pattern space is separated from the new input line by a newline character. Newline characters inserted in pattern space can use\ nmatching.

Example 1: the sample files used are:

[root@andrew Andrew] # cat test.txt Name:TomMail:Tom@gmail.comName:JerryMail:Jerry@gmail.com writes the Sed instruction script as follows (reads the contents of the sample file to the pattern space, immediately reads the next line of content when it matches the Name, and then outputs the contents of the pattern space, # n is used to mask the automatic output).

[root@andrew Andrew] # cat sed.sh # n/Name/ {NL}

The result of running the script is as follows:

[root@andrew Andrew] # sed-f sed.sh test.txt Name:Tom Mail:Tom@gmail.comName:Jerry Mail:Jerry@gmail.com

Example 2: the sample files used are:

[root@andrew Andrew] # cat test.txt 111222222222333

Write a Sed instruction script (read the contents of the sample file to the pattern space, read the next line immediately when the reading matches 222, and then output the contents of the pattern space, lowercase l will print non-printed characters).

[root@andrew Andrew] # cat sed.sh # nBank 222 / {Nl} run the script as follows:

[root@andrew Andrew] # sed-f sed.sh test.txt 222\ n222

two。 Multi-line operation Print

Print (P) is multiline print P, which is slightly different from print p in that it outputs only the first part of the multiline mode space until the first inserted\ nnewline character. The sample files used in the comprehensive example are:

[root@andrew Andrew] # cat test.txt aaabbbcccdddeeefff: compare the differences of different printing methods through several sed commands, and the output is as follows:

[root@andrew Andrew] # sed'/. * / N' test.txt aaabbbcccdddeeefff

[root@andrew Andrew] # sed'/. * / Nten L'test.txt aaa bbbaaabbbccc dddcccdddeee fffeeefff

[root@andrew Andrew] # sed'/. * / Nipipp 'test.txt aaaaaabbbccccccdddeeeeeefff

[root@andrew Andrew] # sed'/. * / the first sed command of N 'test.txt aaabbbaaabbbcccdddcccdddeeefffeeefff uses N to read the next line, and the new read is directly separated from the original content by\ n. But sed does not have any subsequent instructions after reading the next line, so sed automatically outputs, that is, all the contents of the source file.

The second sed command uses N to read the next line, L represents the contents of the display mode space, namely aaa bbb, and the automatic output function of the sed command displays the contents of the source file, that is, aaa,bbb. And so on, sed continues to read the third line of ccc, appends ddd to the end of the line with N, uses L to display the contents of the mode space (no non-print characters), and sed automatically outputs and then displays the contents of the source file.

The third sed command appends the next line to the end of the line using N, and now the content in the mode space is aaa\ bbb, while the function of the P command is to print the first part of the mode space until the end, that is, only aaa, when sed's automatic output function outputs aaa,bbb (sed automatic output will output\ nas a newline). And so on, read the third line ccc,N and append ddd to the end of the line, P print the contents before\ n.At the same time, the sed command will automatically output.

The principle of the fourth sed command is similar to that of the third sed command, but when p prints,\ n is regarded as an enter and line feed, so what is printed is aaa carriage return bbb.

3. Multiline delete operation Delete (D)

Instruction d is a delete command, and its function is to delete the contents of the mode space and read in the new input line, but if sed has more than one instruction after the d instruction, the remaining instructions will no longer be executed, but return the first instruction to process the newly read line. The multiline delete instruction D deletes this part of the pattern space directly before the first inserted newline character (\ n). It does not read the new input line and returns the top of the sed script, so that the remaining instructions continue to apply to the rest of the pattern space.

4.Hold (hmenh), Get (gpene G)

We know that the pattern space is the buffer that holds the current input line. In addition, Sed has a buffer called hold space (hold space). The contents of the schema space can be copied to the hold space, and the contents of the hold space can also be copied to the schema space, and there is a set of Sed commands to move data between the two.

Hold (h | H)

Copy or append the contents of the schema space to the holding space

Get (g | G)

Copy or append the contents of the reserved space to the pattern space

Exchange (x)

Exchange the contents of preserving space and pattern space

The sample files used by the reserved space example are as follows:

[root@andrew Andrew] # cat test.txt aaabbbcccddd [root@andrew Andrew] # cat sed.sh / aaa/ {hd} / ccc/ {G} [root@andrew Andrew] # sed-f sed.sh test.txt bbbcccaaaddd

What is Linux system Linux is a free-to-use and free-spread UNIX-like operating system, is a POSIX-based multi-user, multi-task, multi-threaded and multi-CPU operating system, using Linux can run major Unix tools, applications and network protocols.

The above is how to use sed in the Linux system. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.