Linux: How To Quickly Remove Millions of Files
Friday, November 9th, 2007I tend to get myself involved with projects that have a lot of small files — on the order of a few million. These files are often only good to me for a short period of time, such as when I’m importing a million xml files into a SQL Database for processing later. I’ve also worked on projects where millions of spam emails pile up in queues to be filtered by humans, which they obviously never will be, and simply need to be removed.
The annoying thing about this is the time/effort it takes to remove all of these files quickly and with as little system resource consumption as possible.
In this entry I’m going to show you a couple of ways you can set your self up for quick removal of files that you know (or are pretty sure) will be temporary, so you need to quickly remove them.