Shulou(Shulou.com)06/01 Report--

First of all, awk is a language, and secondly, compared with other languages, the advantage of awk is in text processing, numerical calculation and so on. In our daily operation and maintenance work, awk also has a very important scene with the use of sed and grep. Because of these, awk is one of the most commonly used tools for operation and maintenance engineers. Here are some of the most commonly used postures of awk in log collection and statistical scenarios.

Awk uses format

Awk [- F value] [- v var=value] 'program text' [files....]

Program text:

BEGIN {actions} / pattern/ {actions} END {actions}

Built-in variable

FS--Field Separator: the delimiter of the domain, separated by a space character by default

The delimiter of the RS--Record Separator record, which is separated by a line break by default

FILENAME-current filename

NF-Number of Feilds in current record, the number of domains

NR-the number of records entered by Number of Record, which is the same as the line number, is incremented when multiple files are used.

FNR-the current number of records entered by File Number of Record, calculated separately for each file

$0-the whole record's current entire record

$n-the nth domain of the nth field of the current record's current record

Common functions of awk

1. Split (string, array, field separator)

2. Substr returns the suffix part of the string s whose length is n starting from p

3. The length function returns the length of a string with no arguments.

4. Gsub (regular expression, subsitutionstring, target string)

Awk extension usage

Awk [options] 'script' FILES

Option

-F: field delimiter

-v: declare a variable, FS=:

Awk' {print $1 $2} 'a.txt: then welcometo is displayed

Awk' {print $1 recording 2} 'a.txt: then wlecometo is displayed

-

Built-in variable

ORS

OFS

FS

RS

NR

NF

FNR

ARGV: array to save the command line itself as a string, such as awk' {print $0} 'a.txt b.txt

In this command, ARGV [0] saves awk,ARGV [1] saves a.txt

The number of parameters of the ARGC:awk command

-

The format of the printf command

Printfformat,item1

Main points:

1. The biggest difference from the print command is that printf needs to specify format

2. Format is used to specify the output format of each subsequent item

3. The printf statement does not automatically print newline characters

Indicator in format format:

% c: ASCII that displays characters

% dline% I: decimal integer

% eForce% E: display value by scientific counting method

% f: show floating point number

% gpeng% G: displays values in scientific counting format or floating point format

% s: display string

% u: unsigned integer

%%: show% itself

Modifier:

N: display width

-: align left

+: display numeric symbol

1. Count the percentage of each status code in the squid log

# cat / usr/local/squid/var/logs/access.log | awk 'BEGIN {print "status code", "\ t"rate"} {count+=1 A [$9] + = 1} END {for (i in a) print I, "\ t", a [I] / count*100}'| sort-nk2status code rate400 0.004124484160.00412448478 0.00412448204 0.00618672401 0.02680910 0.0659916301 0.0763028302 2.19628403 2.66441206 5.02567304 17.7043200 71.4339

2. Count the domain names of the top 10 visits in the squid log

] # cat / usr/local/squid/var/logs/access.log | awk 'BEGIN {print "domain", "\ t", "rate"} {count+=1 A [$2] + = 1} END {for (I ina) {print I, "\ t", a [I]}}'| sort-nk2 | tailbsy.pp.starschinalive.com 125js1.pcfg.cache.wpscdn.cn 131portal.wsds.cn 162vas.fun.tv 173img.funshion.com 238www.duba.com 277plugin.video.51togic.com 280p1.meituan.net 286res.qxz.37wan.com 306d.ifengimg.com 481

3. Count the traffic in the last 5 minutes

Log: 27.156.95.175 uc.a.yximgs.com "p_w_picpath/webp" [25/Dec/2016:22:17:01 + 0800] "GET http://uc.a.yximgs.com/upic/2016/11/27/11/BMjAxNjExMjcxMTIxMTNfODY2MTM1NV8xMzAxODgwMDQ4XzJfMw==_low.webp?tag=1-1482675421-f-0-8a7txqz36n-ce40124930229f39 HTTP/1.1" 200

Cat / usr/local/squid/var/logs/access.log | awk'{i=int (substr ($4jor17) / 5) * 5th hobby substr ($4pr 14pag2); s [h ":" I] + = $10;} END {for (i in s) print iGrad s [I] * 8There 1024hand 1024Unip "Mbps"} '21:50 143.537Mbps

4 top10 statistics on the number of domain name requests of meme Miss

# cat / usr/local/squid/var/logs/access.log | awk'{nailed room1 If ($0~/MISS/) a [$2] + = 1} END {for (i in a) {print iQuery a [I] A [I] / nasty 100}}'| sort-nrk2 | headwww.duba.com 237 0.868291p1.meituan.net 163 0.597179pas.suning.com 66 0.241803shopping.suning.com 55 0.20150277g4l9.com5.z0.glb.qiniucdn.com 55 0.201502res.qxz.37wan.com 50 0.183184review.suning.com 41 0.150211static.1sapp.com 32 0.117238s1.vas.wpscdn.cn 30 0.10991fs.ios.kugou.com 25 0.0915919

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.