Shulou(Shulou.com)06/01 Report--

This article introduces how to use the awk command in Linux, the content is very detailed, interested friends can refer to, hope to be helpful to you.

1. Brief introduction of awk command

Awk is known as one of the three swordsmen of text processing, and its name comes from the first letter of its founders Alfred Aho, Peter Weinberger and Brian Kernighan. In fact, AWK does have its own language, the AWK programming language, which the three creators have officially defined as a "style scanning and processing language". It allows you to create short programs that read input files, sort data, process data, perform calculations on input, generate reports, and countless other functions.

So awk is a powerful text analysis tool, compared to the search of grep, the editor of sed, awk is particularly powerful when analyzing data and generating reports. To put it simply, awk is to read the file line by line, slice each line with a space as the default separator, and then analyze the cut part.

II. Format and options of awk command

Grammatical form

Awk [options] 'script' var=value file (s)

Awk [options]-f scriptfile var=value file (s)

Common command options

-F fs fs specifies the input delimiter, and fs can be a string or regular expression, such as-F:

-v var=value assigns a user-defined variable and passes the external variable to awk

-f scripfile reads awk commands from script files

The-m [fr] val sets an inherent limit on the vale value, the-mf option limits the maximum number of blocks allocated to the val, and the-mr option limits the maximum number of records. These two functions are extensions to the Bell lab version of awk and are not applicable in standard awk.

Third, the principle of awk

Awk 'BEGIN {commands} pattern {commands} END {commands}'

Step 1: execute the statements in the BEGIN {commands} statement block

Step 2: read a line from the file or standard input (stdin), and then execute the pattern {commands} statement block, which scans the file line by line, repeating the process from the first line to the last line until all the files have been read.

Step 3: when reading to the end of the input stream, execute the END {commands} statement block.

BEGIN statement blocks are executed before awk starts to read rows from the input stream. This is an optional statement block, such as variable initialization, printout table header, and so on. Statements can usually be written in the BEGIN statement block.

The END statement block is executed after awk has read all the lines from the input stream. For example, the summary of information such as printing the analysis results of all lines is done in the END statement block, which is also an optional statement block.

The general command in the pattern block is the most important part, and it is also optional. If no pattern statement block is provided, {print} is executed by default, that is, every row read is printed, and every line read by awk executes the statement block.

IV. Basic usage of awk

There are three ways to call awk

1. Command line mode

Awk [- F field-separator] 'commands' input-file (s)

Where commands is the real awk command, and [- F domain delimiter] is optional. Input-file (s) is the file to be processed.

In awk, each item separated by a domain delimiter in each line of a file is called a field. In general, when the-F domain delimiter is not named, the default field delimiter is a space.

2.shell scripting mode

Awk 'BEGIN {print "start"} pattern {commands} END {print "end"}' file

An awk script usually consists of three parts: a BEGIN statement block, a general statement block that can use pattern matching, and an END statement block, which are optional. Any part can not appear in the script, which is usually enclosed in single or double quotes, for example:

Awk 'BEGIN {ionization +} END {print I}' filenameawk "BEGIN {ionization +} END {print I}" filename

3. Insert all awk commands into a separate file, and then call the

Awk- f awk-script-file input-file (s)

Where the-f option loads the awk script in awk-script-file, input-file (s) is the same as the command line above.

Let's learn more about the use of awk through a few simple examples

[root@localhost ~] # awk'{print $0}'/ etc/passwd root:x:0:0:root:/root:/bin/bashbin:x:1:1:bin:/bin:/sbin/nologindaemon:x:2:2:daemon:/sbin:/sbin/nologinadm:x:3:4:adm:/var/adm:/sbin/nologinlp:x:4:7:lp:/var/spool/lpd:/sbin/nologinsync:x:5:0:sync:/sbin:/bin / syncshutdown:x:6:0:shutdown:/sbin:/sbin/shutdownhalt:x:7:0:halt:/sbin:/sbin/halt.... [root@localhost ~] # echo 123 | awk'{print "hello Awk "} 'hello,awk [root@localhost ~] # awk' {print" hi "}'/ etc/passwdhihihihihihihihihi....

We specify / etc/passwd as the output file, and when we execute awk, it executes the print command on each line in / etc/passwd in turn.

The awk workflow is like this: read in a record separated by the'\ n' newline character, and then divide the record into fields by the specified domain delimiter, populating the domain, $0 represents all fields, $1 represents the first domain, and $n represents the nth domain. The default domain delimiter is the blank key or the [tab] key, so $1 represents the logged-in user, $3 represents the logged-in user ip, and so on. Such as

Print all user names under / etc/passwd

[root@localhost ~] # awk-F:'{print $1}'/ etc/passwdrootbindaemonadm

....

Print all user names and UID under / etc/passwd

[root@localhost] # awk-F:'{print $1 recording 3}'/ etc/passwdroot 0bin 1daemon 2

....

Output in username: XXX uid: XXX format

[root@localhost ~] # awk-F:'{print "username:" $1 "\ t\ tuid:" $3}'/ etc/passwdusername: root uid: 0username: bin uid: 1username: daemon uid: 2. 5. Awk built-in variable description\ $nth field of the current record Fields separated by FS\ $0 number of complete input records ARGC command line parameters ARGIND command line location of the current file (starting from 0) ARGV array of command line arguments CONVFMT numeric conversion format (default is% .6g) ENVIRON environment variable associative array ERRNO last system error description of FIELDWIDTHS field width list (separated by space bar) FILENAME current file name FNR each file is counted separately Line number of the number FS field delimiter (default is any space) IGNORECASE if true The number of fields that match NF a record that ignores case, the number of records that NR has read, that is, the line number, the output format of OFMT numbers starting from 1 (default is% .6g) OFS output record delimiter (output newline character) Output with the specified symbol instead of the newline character ORS output record delimiter (default is a newline character) RLENGTH the length of the string matched by the match function RS record delimiter (default is a newline character) RSTART the first position of the string matched by the match function the subscript delimiter of the SUBSEP array (default is / 034)

Example

[root@localhost] # echo-e "line1 f2 f3\ nline2 f4 f5\ nline3 f6 f7" | awk'{print "Line No:" NR ", No of fields:" $0 = "$0," $1 = "$1," $2 = "$2," $3 = "$3} 'Line No:1, No of fields:3 $0=line1 f2 f3 $1=line1 $2=f2 $3=f3Line No:2, No of fields:3 $0=line2 f4 f5 $1=line2 $2=f4 $3=f5Line No:3, No of fields:3 $0=line3 f6 f7 $1=line3 $2=f6 $3=f7

Use print $NF to print out the last field in a line, use $(NF-1) to print the penultimate field, and so on:

[root@localhost] # echo-e "line1 f2 f3\ nline2 f4 f5" | awk'{print $NF} 'f3f5 [root@localhost ~] # echo-e "line1 f2 f3\ n line2 f4 f5" | awk' {print $(NF-1)} 'f2f4

Statistics / etc/passwd: file name, line number of each line, number of columns per line, corresponding full line content:

[root@localhost ~] # awk-F':'{print "filename:" FILENAME ", linenumber:" NR ", columns:" NF ", linecontent:" $0}'/ etc/passwdfilename:/etc/passwd,linenumber:1,columns:7,linecontent:root:x:0:0:root:/root:/bin/bashfilename:/etc/passwd,linenumber:2,columns:7,linecontent:bin:x:1:1:bin:/bin:/sbin/nologinfilename:/etc/passwd,linenumber:3,columns:7 Linecontent:daemon:x:2:2:daemon:/sbin:/sbin/nologin

Statistics / etc/passwd file command line parameter ARGC, file line number FNR, field delimiter FS, number of fields in a record NF, number of records read (default is line number) NR

[root@localhost ~] # awk-F: 'BEGIN {printf "% 4s% 4s% 4s% 4s% 4s% 4s\ n", "FILENAME", "ARGC", "FNR", "FS", "NF", "NR" Printf "-\ n"} {printf "% 4s% 4s% 4s% 4s% 4s% 4s\ n", FILENAME,ARGC,FNR,FS,NF NR}'/ etc/passwdFILENAME ARGC FNR FS NF NR-/etc/passwd 2 1: 7 1/etc/passwd 2 2: 7 2/etc/passwd 2 3: 7 36, advanced usage of awk

1.awk assignment operation

Assignment statement operator: = + =-= * / =% = ^ = * * =

For example: axiomatic 5; equivalent to a=a+5

[root@localhost ~] # awk 'BEGIN {adept 5 strategic print a}' 10

2.awk regular operation

Output the line containing root, and print the user name and UID and the original line content

[root@localhost ~] # awk-F:'/ root/ {print $1 root:x:0:0:root:/root:/bin/bashoperator'/ etc/passwdroot 0 root:x:0:0:root:/root:/bin/bashoperator 11 operator:x:11:0:operator:/root:/sbin/nologin

We found two lines. If we are looking for a line that starts with root, we should write: awk-F:'/ ^ root/' / etc/passwd

3.awk trinomial operation

[root@localhost ~] # awk 'BEGIN {a = "b"; print asides = "b"? "ok": "err"}' ok [root@localhost ~] # awk 'BEGIN {a = "b"; print asides = "c"? "ok": "err"}' err

Trinomial operation is actually a judgment operation, if it is true, then output? If it is false, then output the content after:

The Recycling of 4.awk

The use of if sentences

[root@localhost ~] # awk 'BEGIN {test=100;if (test > 90) {print "vear good";} else {print "no pass";}}' vear good

Use after each command; end.

Circular application of while

Calculate a value that adds up from 1 to 100

[root@localhost ~] # awk 'BEGIN {test=100;num=0;while (I

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.