Shulou(Shulou.com)06/01 Report--

This article is about the sample analysis of the Shell text processing tool in Linux. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

The examples and parameters provided are the most commonly used and practical.

My principle for shell scripts is to write commands on a single line, and try not to exceed 2 lines.

If you have more complex task requirements, consider python.

Find file lookup

Find txt and pdf files

The code is as follows:

Find. \ (- name "* .txt"-o-name "* .pdf"\)-print

Regular way to find .txt and pdf

The code is as follows:

Find. -regex ". *\ (\ .txt |\ .pdf\) $"

-iregex: ignore case regularities

Negative parameter

Find all non-txt text

The code is as follows:

Find. !-name "* .txt"-print

Specify search depth

Print out the file of the current directory (depth 1)

The code is as follows:

Find. -maxdepth 1-type f

Custom search

Search by type:

The code is as follows:

Find. -type d-print / / lists only all directories

-type f file / l symbolic link

Search by time:

-atime access time (in days, minutes is-amin, similar below)

-mtime modification time (content modified)

-ctime change time (metadata or permission change)

All files accessed in the last 7 days:

The code is as follows:

Find. -atime 7-type f-print

Search by size:

W word k M G

Look for files greater than 2k

The code is as follows:

Find. -type f-size + 2k

Find by permission:

The code is as follows:

Find. -type f-perm 644-print / / find all files with executable permissions

Find by user:

The code is as follows:

Find. -type f-user weber-print// to find the files owned by user weber

The subsequent action after finding it.

Delete:

Delete all swp files in the current directory:

The code is as follows:

Find. -type f-name "* .swp"-delete

Perform actions (powerful exec)

The code is as follows:

Find. -type f-user root-exec chown weber {}\; / / change ownership under the current directory to weber

Note: {} is a special string. For each matching file, {} will be replaced with the corresponding file name.

Eg: copy all the files found to another directory:

The code is as follows:

Find. -type f-mtime + 10-name "* .txt"-exec cp {} OLD\

Combine multiple commands

Tips: if you need to execute multiple commands later, you can write multiple commands into a single script. Then execute the script when-exec is called

The code is as follows:

-exec. / commands.sh {}\

Delimiter of-print

Defaults to'\ n' as the delimiter of the file

-print0 uses'\ 0' as the delimiter of the file so that you can search for files that contain spaces

Grep text search

Grep match_patten file / / default access to matching lines

Common parameters

-o output only matching lines of text VS-v outputs only lines of text that do not match

-c Statistics the number of times the file contains text

The code is as follows:

Grep-c "text" filename

-n print matching line number

-I ignore case when searching

-l prints only file names

Recursive search for text in a multi-level directory (programmer's favorite search code):

The code is as follows:

Grep "class". -R-n

Match multiple patterns

The code is as follows:

Grep-e "class"-e "vitural" file

Grep outputs the file name with\ 0 as the Terminator: (- z)

The code is as follows:

Grep "test" file*-lZ | xargs-0 rm

Xargs command line argument conversion

Xargs can convert input data into command-line arguments for specific commands; this can be combined with many commands. Like grep, like find.

Convert multi-line output to single-line output

Cat file.txt | xargs

\ nThe delimiter between multiple lines of text

Convert single lines to multiple lines of output

Cat single.txt | xargs-n 3

-n: specify the number of fields displayed per row

Xargs parameter description

-d defines the delimiter (the default is the space multiline delimiter\ n)

-n specifies that the output is multiple lines

-I {} specifies the replacement string, which is replaced when the xargs is extended, for use when the command to be executed requires more than one argument

Eg:

The code is as follows:

Cat file.txt | xargs-I {}. / command.sh-p {}-1

-0: specify\ 0 as the input delimiter

Eg: counting the number of program lines

The code is as follows:

Find source_dir/-type f-name "* .cpp"-print0 | xargs-0 wc-l

Sort sorting

Field description:

-n sort by number VS-d sort by dictionary order

-r sort in reverse order

-k N specifies to sort by Nth column

Eg:

The code is as follows:

Sort-nrk 1 data.txt

Sort-bd data / / ignores leading white space characters such as spaces

Uniq eliminates duplicate lines

Eliminate duplicate lines

The code is as follows:

Sort unsort.txt | uniq

Count the number of times each line appears in the file

The code is as follows:

Sort unsort.txt | uniq-c

Find duplicate lines

The code is as follows:

Sort unsort.txt | uniq-d

You can specify the repetition to be compared in each line:-s start position-w comparison characters

Use tr for conversion

General usage

The code is as follows:

Echo 12345 | tr'0-9''9876543210' / / encryption / decryption conversion to replace the corresponding characters

Cat text | tr'\ t'/ / Tabs turn spaces

Tr delete character

The code is as follows:

Cat file | tr-d'0-9' / / Delete all digits

-c complement set

The code is as follows:

Cat file | tr-c'0-9' / / get all the numbers in the file

Cat file | tr-d-c'0-9\ n'/ / Delete non-numeric data

Tr compressed characters

Repetitive characters that occur in tr-s compressed text; most commonly used to compress excess spaces

The code is as follows:

Cat file | tr-s''

Character class

Various character classes are available in tr:

Alnum: letters and numbers

Alpha: letter

Digit: numeric

Space: White space character

Lower: lowercase

Upper: uppercase

Cntrl: control (non-printable) character

Print: printable character

Usage: tr [: class:] [: class:]

The code is as follows:

Eg: tr'[: lower:]'[: upper:]'

Cut splits text by column

Intercept columns 2 and 4 of the document:

The code is as follows:

Cut-f2 filename 4

Go to all the columns of the file except column 3:

The code is as follows:

Cut-f3-- complement filename

-d specifies the delimiter:

The code is as follows:

Cat-f2-d ";" filename

The range taken by cut

N-Nth field to the end

-M the first field is M

Nmurm M N to M fields

Units taken by cut

-b in bytes

-c in characters

-f in fields (using delimiters)

Eg:

The code is as follows:

Cut-C1-5 file / / print the first to five characters

Cut-Cmur2 file / / print the first 2 characters

Paste splices text by column

Splice two pieces of text together in columns

The code is as follows:

Cat file1

one

two

Cat file2

Colin

Book

Paste file1 file2

1 colin

2 book

The default delimiter is a tab, which can be specified with-d

Paste file1 file2-d ","

1,colin

2,book

Wc tools for counting lines and characters

Wc-l file / / count rows

Wc-w file / / count words

Wc-c file / / count the number of characters

Sharp weapon of sed text replacement

First replacement

The code is as follows:

Seg's file _ text _

Global replacement

The code is as follows:

Seg's TableText replacebound textCompact g'file

After the default replacement, output the replaced content. If you need to directly replace the original file, use-I:

The code is as follows:

Seg-I's file. Repalceeded text.

Remove blank lines:

The code is as follows:

Sed'/ ^ $/ d'file

Variable conversion

The matched string is referenced by the tag &

The code is as follows:

Echo this is en example | seg's /\ walled / [&] / g'

$> [this] [is] [en] [example]

Substring matching tag

The first matching parenthesis is referenced using the tag\ 1

The code is as follows:

Sed 's/hello\ ([0-9]\) /\ 1ax'

Double quotation mark evaluation

Sed is usually quoted in single quotation marks, or double quotation marks, which evaluate the expression after using double quotation marks:

The code is as follows:

Sed's Universe varamel HLLOE Universe

When using double quotes, we can specify variables in the sed style and the replacement string

The code is as follows:

Eg:

P=patten

R=replaced

Echo "line con a patten" | sed "s/$p/$r/g"

$> line con a replaced

Other exampl

String insert character: converts each line of text (PEKSHA) to PEK/SHA

The code is as follows:

Sed's / ^.\ {3\} / &\ / / g 'file

Awk data flow processing tool

Awk script structure

Awk 'BEGIN {statements} statements2 END {statements}'

Mode of work

1. Execute statement blocks in begin

two。 Read a line from a file or stdin, then execute statements2, and repeat the process until all the files have been read

3. Execute end statement block

Print prints the current line

When using print with no parameters, the current line is printed

The code is as follows:

Echo-e "line1\ nline2" | awk 'BEGIN {print "start"} {print} END {print "End"}'

When print is divided by commas, parameters are delimited by spaces

The code is as follows:

Echo | awk'{var1 = "v1"; var2 = "V2"; var3= "v3";\

Print var1, var2, var3;}'

$> v1 V2 v3

How to use the-splice character ("" as the splicer)

The code is as follows:

Echo | awk'{var1 = "v1"; var2 = "V2"; var3= "v3";\

Print var1 "-" var2 "-" var3;}'

$> v1-V2-v3

Special variable: NR NF $0 $1 $2

NR: indicates the number of records, corresponding to the current line number during execution

NF: indicates the number of fields, and always corresponds to the number of fields that should be in front of it during execution

$0: this variable contains the text content of the current line during execution

$1: the text content of the first field

$2: the text content of the second field

The code is as follows:

Echo-e "line1 f2 f3\ nline2\ nline 3" | awk'{print NR ":" $0 "-" $1 "-" $2}'

Print the second and third fields of each line:

The code is as follows:

Awk'{print $2, $3} 'file

Count the number of lines in the file:

The code is as follows:

Awk 'END {print NR}' file

Accumulate the first field of each line:

The code is as follows:

Echo-e "1\ n 2\ n 3\ n 4\ n" | awk 'BEGIN {num = 0

Print "begin";} {sum + = $1;} END {print "="; print sum}'

Pass external variables

The code is as follows:

Var=1000

Echo | awk'{print vara} 'vara=$var # input from stdin

Awk'{print vara} 'vara=$var file # input from a file

Use style to filter marching rows processed by awk

Awk'NR < 5'# Line number is less than 5

Print awk 'NR==1,NR==4 {print}' file # with line numbers equal to 1 and 4

Awk'/ linux/' # lines containing linux text (super powerful, which can be specified with regular expressions)

Awk'! / linux/' # Lines that do not contain linux text

Set delimiter

Use-F to set the delimiter (default is a space)

Awk-F:'{print $NF}'/ etc/passwd

Read command output

Using getline, read the output of the external shell command into the variable cmdout

The code is as follows:

Echo | awk'{"grep root / etc/passwd" | getline cmdout; print cmdout}'

Using loops in awk

For (iSuppli)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.