Shulou(Shulou.com)06/01 Report--

This article shows you how to use fdupes to delete duplicate files in the Linux system, the content is concise and easy to understand, it will definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

What is fdupes?

Fdupes is a tool under Linux written by Adrian Lopez in the C programming language and distributed under the MIT license. The application can find duplicate files in specified directories and subdirectories. Fdupes identifies duplicates by comparing the MD5 signatures of files and comparing files byte by byte. Fdupes has a variety of options to list, delete and replace hard links with copies of files.

The file starts in the following order:

Size comparison > partial MD5 signature comparison > complete MD5 signature comparison > byte by byte comparison

Install fdupes to Linux

On Debian-based systems such as Ubuntu and Linux Mint, install the latest version of fdupes with the following command.

The code is as follows:

$sudo apt-get install fdupes

On CentOS/RHEL-and Fedora-based systems, you need to open the epel repository to install the fdupes package.

The code is as follows:

# yum install fdupes

# dnf install fdupes

[in Fedora 22 and beyond]

Note: since Fedora 22, the default package manager yum has been replaced by dnf.

How to use the fdupes command

1. For demonstration purposes, let's create some duplicate files in a directory (such as tecmint) with the following command:

The code is as follows:

$mkdir / home/ "$USER" / Desktop/tecmint & & cd / home/ "$USER" / Desktop/tecmint & & for i in {1.. 15}; do echo "I Love Tecmint. Tecmint is a very nice community of Linux Users." > tecmint$ {I} .txt; done

After executing the above command, let's use the ls command to verify that the duplicate file is created.

The code is as follows:

$ls-l

Total 60

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint10.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint11.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint12.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint13.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint14.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint15.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint1.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint2.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint3.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint4.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint5.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint6.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint7.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint8.txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint9.txt

The above script creates 15 files named tecmint1.txt,tecmint2.txt... Tecmint15.txt, and the data for each file is the same, as shown in

"I Love Tecmint. Tecmint is a very nice community of Linux Users."

2. Now search the tecmint folder for duplicate files.

$fdupes / home/$USER/Desktop/tecmint

/ home/tecmint/Desktop/tecmint/tecmint13.txt

/ home/tecmint/Desktop/tecmint/tecmint8.txt

/ home/tecmint/Desktop/tecmint/tecmint11.txt

/ home/tecmint/Desktop/tecmint/tecmint3.txt

/ home/tecmint/Desktop/tecmint/tecmint4.txt

/ home/tecmint/Desktop/tecmint/tecmint6.txt

/ home/tecmint/Desktop/tecmint/tecmint7.txt

/ home/tecmint/Desktop/tecmint/tecmint9.txt

/ home/tecmint/Desktop/tecmint/tecmint10.txt

/ home/tecmint/Desktop/tecmint/tecmint2.txt

/ home/tecmint/Desktop/tecmint/tecmint5.txt

/ home/tecmint/Desktop/tecmint/tecmint14.txt

/ home/tecmint/Desktop/tecmint/tecmint1.txt

/ home/tecmint/Desktop/tecmint/tecmint15.txt

/ home/tecmint/Desktop/tecmint/tecmint12.txt

Use the-r option to recursively search for duplicate files in each directory, including its subdirectories.

It recursively searches all files and folders and takes a moment to scan duplicate files, depending on the number of files and folders. In the meantime, the entire process is displayed in the terminal, as shown below.

The code is as follows:

$fdupes-r / home

Progress [37780 Compact 54747] 69%

4. Use the-S option to view the size of duplicate files found in a folder.

The code is as follows:

$fdupes-S / home/$USER/Desktop/tecmint

65 bytes each:

/ home/tecmint/Desktop/tecmint/tecmint13.txt

/ home/tecmint/Desktop/tecmint/tecmint8.txt

/ home/tecmint/Desktop/tecmint/tecmint11.txt

/ home/tecmint/Desktop/tecmint/tecmint3.txt

/ home/tecmint/Desktop/tecmint/tecmint4.txt

/ home/tecmint/Desktop/tecmint/tecmint6.txt

/ home/tecmint/Desktop/tecmint/tecmint7.txt

/ home/tecmint/Desktop/tecmint/tecmint9.txt

/ home/tecmint/Desktop/tecmint/tecmint10.txt

/ home/tecmint/Desktop/tecmint/tecmint2.txt

/ home/tecmint/Desktop/tecmint/tecmint5.txt

/ home/tecmint/Desktop/tecmint/tecmint14.txt

/ home/tecmint/Desktop/tecmint/tecmint1.txt

/ home/tecmint/Desktop/tecmint/tecmint15.txt

/ home/tecmint/Desktop/tecmint/tecmint12.txt

5. You can use both the-S and-r options to view the size of duplicate files in all the directories and subdirectories involved, as follows:

The code is as follows:

$fdupes-Sr / home/avi/Desktop/

65 bytes each:

/ home/tecmint/Desktop/tecmint/tecmint13.txt

/ home/tecmint/Desktop/tecmint/tecmint8.txt

/ home/tecmint/Desktop/tecmint/tecmint11.txt

/ home/tecmint/Desktop/tecmint/tecmint3.txt

/ home/tecmint/Desktop/tecmint/tecmint4.txt

/ home/tecmint/Desktop/tecmint/tecmint6.txt

/ home/tecmint/Desktop/tecmint/tecmint7.txt

/ home/tecmint/Desktop/tecmint/tecmint9.txt

/ home/tecmint/Desktop/tecmint/tecmint10.txt

/ home/tecmint/Desktop/tecmint/tecmint2.txt

/ home/tecmint/Desktop/tecmint/tecmint5.txt

/ home/tecmint/Desktop/tecmint/tecmint14.txt

/ home/tecmint/Desktop/tecmint/tecmint1.txt

/ home/tecmint/Desktop/tecmint/tecmint15.txt

/ home/tecmint/Desktop/tecmint/tecmint12.txt

107 bytes each:

/ home/tecmint/Desktop/resume_files/r-csc.html

/ home/tecmint/Desktop/resume_files/fc.html

6. Instead of searching recursively in one or all folders, you can choose to search selectively in two or three folders as required. There is no need to remind you that you can use the-S and / or-r options if necessary.

The code is as follows:

$fdupes / home/avi/Desktop/ / home/avi/Templates/

7. To delete duplicate files while keeping a copy, you can use the-d option. With this option, you must be extra careful, or the end result may be file / data loss. Solemnly remind me that this operation cannot be resumed.

The code is as follows:

$fdupes-d / home/$USER/Desktop/tecmint

[1] / home/tecmint/Desktop/tecmint/tecmint13.txt

[2] / home/tecmint/Desktop/tecmint/tecmint8.txt

[3] / home/tecmint/Desktop/tecmint/tecmint11.txt

[4] / home/tecmint/Desktop/tecmint/tecmint3.txt

[5] / home/tecmint/Desktop/tecmint/tecmint4.txt

[6] / home/tecmint/Desktop/tecmint/tecmint6.txt

[7] / home/tecmint/Desktop/tecmint/tecmint7.txt

[8] / home/tecmint/Desktop/tecmint/tecmint9.txt

[9] / home/tecmint/Desktop/tecmint/tecmint10.txt

[10] / home/tecmint/Desktop/tecmint/tecmint2.txt

[11] / home/tecmint/Desktop/tecmint/tecmint5.txt

[12] / home/tecmint/Desktop/tecmint/tecmint14.txt

[13] / home/tecmint/Desktop/tecmint/tecmint1.txt

[14] / home/tecmint/Desktop/tecmint/tecmint15.txt

[15] / home/tecmint/Desktop/tecmint/tecmint12.txt

The code is as follows:

Set 1 of 1, preserve files [1-15, all]:

You may have noticed that all duplicate files are listed and prompted to delete one by one, or specify a range, or delete them all at once. You can select a range, like below, to delete files within the specified range.

The code is as follows:

Set 1 of 1, preserve files [1-15, all]: 2-15

[-] / home/tecmint/Desktop/tecmint/tecmint13.txt

[+] / home/tecmint/Desktop/tecmint/tecmint8.txt

[-] / home/tecmint/Desktop/tecmint/tecmint11.txt

[-] / home/tecmint/Desktop/tecmint/tecmint3.txt

[-] / home/tecmint/Desktop/tecmint/tecmint4.txt

[-] / home/tecmint/Desktop/tecmint/tecmint6.txt

[-] / home/tecmint/Desktop/tecmint/tecmint7.txt

[-] / home/tecmint/Desktop/tecmint/tecmint9.txt

[-] / home/tecmint/Desktop/tecmint/tecmint10.txt

[-] / home/tecmint/Desktop/tecmint/tecmint2.txt

[-] / home/tecmint/Desktop/tecmint/tecmint5.txt

[-] / home/tecmint/Desktop/tecmint/tecmint14.txt

[-] / home/tecmint/Desktop/tecmint/tecmint1.txt

[-] / home/tecmint/Desktop/tecmint/tecmint15.txt

[-] / home/tecmint/Desktop/tecmint/tecmint12.txt

8. From a security point of view, you may want to print the output of fdupes to a file, and then check the text file to decide which file to delete. This reduces the risk of accidentally deleting files. You can do this:

The code is as follows:

$fdupes-Sr / home > / home/fdupes.txt

Note: you should replace / home for the folder you want. At the same time, if you want to search recursively and print the size, you can use the-r and-S options.

9. You can use the-f option to ignore the first file in each matching set.

First list the files in that directory.

The code is as follows:

$ls-l / home/$USER/Desktop/tecmint

Total 20

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint9 (3rd copy) .txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint9 (4th copy) .txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint9 (another copy) .txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint9 (copy) .txt

-rw-r--r-- 1 tecmint tecmint 65 Aug 8 11:22 tecmint9.txt

Then, ignore the first file in each matching set.

The code is as follows:

$fdupes-f / home/$USER/Desktop/tecmint

/ home/tecmint/Desktop/tecmint9 (copy) .txt

/ home/tecmint/Desktop/tecmint9 (3rd copy) .txt

/ home/tecmint/Desktop/tecmint9 (another copy) .txt

/ home/tecmint/Desktop/tecmint9 (4th copy) .txt

10. Check the installed version of fdupes.

The code is as follows:

$fdupes-version

Fdupes 1.51

If you need help with fdupes, you can use the-h switch.

$fdupes-h

Usage: fdupes [options] DIRECTORY...

-r-- recurse for every directory given follow subdirectories

Encountered within

-R-- recurse: for each directory given after this option follow

Subdirectories encountered within (note the':'at

The end of the option, manpage for more details)

-s-- symlinks follow symlinks

-H-hardlinks normally, when two or more files point to the same

Disk area they are treated as non-duplicates; this

Option will change this behavior

-n-- noempty exclude zero-length files from consideration

-A-- nohidden exclude hidden files from consideration

-f-- omitfirst omit the first file in each set of matches

-1-- sameline list each set of matches on a single line

-S-- size show size of duplicate files

-m-- summarize summarize dupe information

-Q-- quiet hide progress indicator

-d-- delete prompt user for files to preserve and delete all

Others; important: under particular circumstances

Data may be lost when using this option together

With-s or-- symlinks, or when specifying a

Particular directory more than once; refer to the

Fdupes documentation for additional information

-N-noprompt together with-delete, preserve the first file in

Each set of duplicates and delete the rest without

Prompting the user

-v-- version display fdupes version

-h-- help display this help message

The above is how to use fdupes to delete duplicate files in the Linux system. Have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.