Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to use the Awk command in Linux. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

Brief introduction

Awk is a powerful text analysis tool, compared with the search of grep and the editor of sed, awk is particularly powerful in analyzing data and generating reports. To put it simply, awk is to read the file line by line, slice each line with a space as the default separator, and then analyze the cut part.

There are three different versions of awk: awk, nawk and gawk, which are not specifically specified. Generally speaking, gawk,gawk is the GNU version of AWK.

Awk gets its name from the initials of its founders Alfred Aho, Peter Weinberger and Brian Kernighan. In fact, AWK does have its own language, the AWK programming language, which the three creators have officially defined as a "style scanning and processing language". It allows you to create short programs that read input files, sort data, process data, perform calculations on input, generate reports, and countless other functions.

Usage

Awk'{pattern + action}'{filenames}

Although the operation can be complex, the syntax is always like this, where pattern represents what AWK looks for in the data, and action is a series of commands that are executed when a match is found. Curly braces ({}) do not need to appear all the time in the program, but they are used to group a series of instructions according to a specific pattern. Pattern is the regular expression that you want to represent, surrounded by diagonal bars.

The most basic function of awk language is to browse and extract information from files or strings based on specified rules. After awk extracts information, other text operations can be carried out. A complete awk script is usually used to format information in a text file.

Typically, awk processes units as a behavior of a file. Awk receives each line of the file and then executes the appropriate command to process the text.

Call awk

There are three ways to call awk

1. Command line mode

Awk [- F field-separator] 'commands' input-file (s) where commands is the real awk command and [- F domain delimiter] is optional. Input-file (s) is the file to be processed. In awk, each item separated by a domain delimiter in each line of a file is called a field. In general, when the-F domain delimiter is not named, the default field delimiter is a space.

2.shell scripting mode

Insert all the awk commands into a file and make the awk program executable, and then the awk command interpreter, as the first line of the script, is called by typing the name of the script. Equivalent to the first line of the shell script: #! / bin/sh can be replaced by: #! / bin/awk

3. Insert all the awk commands into a separate file, and then call: awk- f awk-script-file input-file (s) where the-f option loads the awk script in awk-script-file, and input-file (s) is the same as above.

This chapter focuses on the command line approach.

Getting started example

Suppose the output of last-n 5 is as follows

# last-n 5 take out only the first five lines

Root pts/1 192.168.1.100 Tue Feb 10 11:21 still logged in

Root pts/1 192.168.1.100 Tue Feb 10 00:46-02:28 (01:41)

Root pts/1 192.168.1.100 Mon Feb 9 11:41-18:30 (06:48)

Dmtsai pts/1 192.168.1.100 Mon Feb 9 11:41-11:41 (00:00)

Root tty1 Fri Sep 5 14:09-14:10 (00:01)

If you only show the five most recently logged in accounts

# last-n 5 | awk'{print $1}'

Root

Root

Root

Dmtsai

Root

The awk workflow goes like this: read in a record separated by the'n 'newline character, and then divide the record into fields by the specified domain delimiter, populating the domain, $0 represents all fields, $1 represents the first domain, and $n represents the nth domain. The default domain separator is "blank key" or "key", so $1 represents the logged-in user, $3 represents the logged-in user ip, and so on.

If it just shows the account of / etc/passwd

# cat / etc/passwd | awk-F':'{print $1}'

Root

Daemon

Bin

Sys

This is an example of awk+action, where each line executes action {print $1}.

-F specifies that the domain delimiter is':'.

If only the account of / etc/passwd and the corresponding shell of the account are displayed, and the account and shell are separated by the tab key

# cat / etc/passwd | awk-F':'{print $1 "t" $7}'

Root / bin/bash

Daemon / bin/sh

Bin/ bin/sh

Sys / bin/sh

If you only show the account of / etc/passwd and the corresponding shell of the account, and the account and shell are separated by a comma, and you add the column name name,shell in all lines, add "blue,/bin/nosh" on the last line.

Cat / etc/passwd | awk-F': 'BEGIN {print "name,shell"} {print $1 "," $7} END {print "blue,/bin/nosh"}'

Name,shell

Root,/bin/bash

Daemon,/bin/sh

Bin,/bin/sh

Sys,/bin/sh

....

Blue,/bin/nosh

The awk workflow is like this: first execute BEGING, then read the file, read in a record separated by the / n newline character, and then divide the record into fields according to the specified domain delimiter, populate the domain, $0 represents all fields, $1 represents the first domain, $n represents the nth domain, and then starts to execute the action action corresponding to the mode. Then start reading the second record until all the records have been read, and finally perform the END operation.

Search / etc/passwd all lines with the root keyword

# awk-F:'/ root/' / etc/passwd

Root:x:0:0:root:/root:/bin/bash

This is an example of the use of pattern, where lines that match pattern (in this case, root) will execute action (no action specified, default output of each line).

Search supports rules, such as looking for awk-F:'/ ^ root/' / etc/passwd at the beginning of root

Search for all lines with the root keyword in / etc/passwd and display the corresponding shell

# awk-F:'/ root/ {print $7}'/ etc/passwd

/ bin/bash

Action {print $7} is specified here

Awk built-in variables

Awk has many built-in variables to set environment information, which can be changed. Here are some of the most commonly used variables.

Number of ARGC command line parameters

ARGV command line argument arrangement

ENVIRON supports the use of system environment variables in queues

File name browsed by FILENAME awk

The number of records FNR browsed the file

FS sets the input field separator, which is equivalent to the command line-F option

Number of fields for NF browsing records

Number of records read by NR

OFS output domain delimiter

ORS output record delimiter

RS control record delimiter

In addition, the $0 variable refers to the entire record. $1 represents the first field of the current line, and $2 represents the second field of the current row. and so on

Statistics / etc/passwd: file name, line number of each line, number of columns per line, corresponding full line content:

# awk-F':'{print "filename:" FILENAME ", linenumber:" NR ", columns:" NF ", linecontent:" $0}'/ etc/passwd

Filename:/etc/passwd,linenumber:1,columns:7,linecontent:root:x:0:0:root:/root:/bin/bash

Filename:/etc/passwd,linenumber:2,columns:7,linecontent:daemon:x:1:1:daemon:/usr/sbin:/bin/sh

Filename:/etc/passwd,linenumber:3,columns:7,linecontent:bin:x:2:2:bin:/bin:/bin/sh

Filename:/etc/passwd,linenumber:4,columns:7,linecontent:sys:x:3:3:sys:/dev:/bin/sh

Using printf instead of print can make the code more concise and easy to read

Awk-F':'{printf ("filename:s,linenumber:%s,columns:%s,linecontent:%sn", FILENAME,NR,NF,$0)}'/ etc/passwd

Print and printf

Both print and printf printout functions are provided in awk.

Where the argument to the print function can be a variable, a numeric value, or a string. Strings must be referenced in double quotes and parameters separated by commas. If there is no comma, the parameters are concatenated and cannot be distinguished. Here, the comma serves the same purpose as the delimiter of the output file, except that the latter is a space.

Printf function, whose usage is basically similar to printf in c language, can format strings. When the output is complex, printf is easier to use and the code is easier to understand.

Awk programming

Variables and assignments

In addition to awk's built-in variables, awk can also customize variables.

Count the number of accounts in / etc/passwd below

Awk'{count++;print $0;} END {print "user count is", count}'/ etc/passwd

Root:x:0:0:root:/root:/bin/bash

.

User count is 40

Count is a custom variable. There was only one print in the previous action {}. In fact, print is just a statement, while action {} can have multiple statements separated by the; sign.

Count is not initialized here. Although the default is 0, it is appropriate to initialize it to 0:

Awk 'BEGIN {count=0;print "[start] user count is", count} {count=count+1;print $0;} END {print "[end] user count is", count}' / etc/passwd

[start] user count is 0

Root:x:0:0:root:/root:/bin/bash

...

[end] user count is 40

Count the number of bytes occupied by files in a folder

Ls-l | awk 'BEGIN {size=0;} {size=size+$5;} END {print "[end] size is", size}'

[end] size is 8657198

If displayed in M units:

Ls-l | awk 'BEGIN {size=0;} {size=size+$5;} END {print "[end] size is", size/1024/1024, "M"}'

[end] size is 8.25889 M

Note that the statistics do not include subdirectories of the folder.

Conditional statement

The conditional statements in awk are borrowed from the C language, as shown in the following declaration:

If (expression) {

Statement

Statement

......

}

If (expression) {

Statement

} else {

Statement2

}

If (expression) {

Statement1

} else if (expression1) {

Statement2

} else {

Statement3

}

Count the number of bytes occupied by files in a folder and filter files with a size of 4096 (usually folders):

Ls-l | awk 'BEGIN {size=0;print "[start] size is", size} {if ($5 million) {size=size+$5;}} END {print "[end] size is", size/1024/1024, "M"}'

[end] size is 8.22339 M

Loop statement

The circular sentences in awk are also borrowed from C language and support while, do/while, for, break and continue. The semantics of these keywords are exactly the same as those in C language.

Array

Because the subscript of an array in awk can be numeric and alphabetic, the subscript of an array is often referred to as a key. The values and keywords are stored in an internal table for the key/value application hash. Because hash is not stored sequentially, when you display the contents of the array, you will find that they are not displayed in the order you expected. Arrays, like variables, are created automatically when they are used, and awk also automatically determines whether they store numbers or strings. In general, arrays in awk are used to collect information from records and can be used to calculate sums, count words, track the number of times the template has been matched, and so on.

Show / etc/passwd 's account

Awk-F': 'BEGIN {count=0;} {name [count] = $1 *

0 root

1 daemon

2 bin

3 sys

4 sync

5 games

.

This is the end of this article on "how to use Awk commands in Linux". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.