Shulou(Shulou.com)06/02 Report--

This article mainly introduces the method of using linux's uniq command to remove duplicate lines in the file, which has a certain reference value, and friends who need it can refer to it. Let's take a look at it with me.

The full name of the uniq command is "unique", and the Chinese definition is "unique and unique". The purpose of this command is to remove consecutive duplicate lines from a text file without other lines of text. Get rid of the repetition, what is retained is unique, that is, unique.

We should note that the difference between it and sort is that sort removes duplicate lines as long as there are duplicate lines, while uniq duplicate lines must be continuous, or you can use it to ignore duplicate lines in the file.

Syntax format: uniq [parameters] [file]

Common parameters:

-c print the number of times each line is repeated in the text-d shows only duplicate records, each duplicate record appears only once-u only shows records that are not duplicated

Reference example

Delete consecutive duplicate lines in a continuous file:

[root@linuxcool ~] # cat testfile

Test 30

Test 30

Test 30

Hello 95

Hello 95

Hello 95

Hello 95

Linux 85

Linux 85

[root@linuxcool ~] # uniq testfile

Test 30

Hello 95

Linux 85

Print the number of times each line is repeated in the file:

[root@linuxcool ~] # uniq-c testfile

3 test 30

4 Hello 95

2 Linux 85

Only duplicate records are shown, and each record appears only once:

[root@linuxcool] # uniq-d testfile

Test 30

Hello 95

Linux 85

Only records that are not duplicated are displayed:

[root@linuxcool] # uniq-u testfile

[root@linuxcool] # above are the details of the method of using linux's uniq command to remove duplicate lines in the file. have you learned anything after reading it? If you want to know more about it, welcome to the industry information!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.