Finding and removing duplicate files with fdupes

Published on 2009-08-01.

If you have a lot of duplicate files "fdupes" can help you remove them very easily.

On Debian derived systems you can install fdupes using the command:

# apt-get install fdupes

The program is run from the terminal and the options are very basic. To search a directory for duplicate files, simply run the following command:

$ fdupes -r my_dir/

The command will recursively search the directory my_dir for any duplicate files, and display the results.

You can also create a text file containing the names of you duplicate files running this command:

$ fdupes -r my_dir/ > my_duplicate_files.txt

What I needed was to remove all duplicate files except for one of each file. I also needed a list of the files that was kept and the ones that was removed. To do that I used this command:

$ fdupes -rdN my_dir/ > dubs.txt

The r option makes fdupes search for files recursively. The d option makes fdupes delete duplicates. The N option, when used together with d, preserve the first file in each set of duplicates and delete the others without prompting the user.

See man fdupes for further instructions.

If you have any comments or corrections please feel free to email them to me. Also, if you found this content useful consider supporting me on Patreon