HiveBrain v1.2.0
Get Started
← Back to all entries
patterncsharpMinor

Fastest way searching specific files

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
searchingwayfilesfastestspecific

Problem

I got thousands of files with a specific file extension in thousands of sub folders. Now, what is the fastest way to search with a pattern? I tried the method DirectoryInfo.GetFiles(rootfolder) (~8 minutes) and a recursive custom method (~5 minutes).

private void WalkDirectoryTree(DirectoryInfo dr, string searchname)
    {
        System.IO.FileInfo[] files = null;
        System.IO.DirectoryInfo[] subDirs = null;
        try
        {
            files = dr.GetFiles(searchname + ".*");
        }
        catch (Exception ex)
        {
            MessageBox.Show(ex.Message);
        }

        if (files != null)
        {
            foreach (FileInfo fi in files)
            {
                allFiles.Add(fi);
            }
            subDirs = dr.GetDirectories();

            foreach (DirectoryInfo di in subDirs)
            {
                WalkDirectoryTree(di, searchname);
            }
        }
    }


Is there any faster way to do it?

Solution

Try parallelization. Instead of:

foreach (DirectoryInfo di in subDirs)
{
    WalkDirectoryTree(di, searchname);
}


Do

Parallel.ForEach(subDirs, dir => WalkDirectoryTree(dir, searchname));


Notice that by doing this allFiles will be accessed concurrently so change your collection to a ConcurrentBag.

Code Snippets

foreach (DirectoryInfo di in subDirs)
{
    WalkDirectoryTree(di, searchname);
}
Parallel.ForEach(subDirs, dir => WalkDirectoryTree(dir, searchname));

Context

StackExchange Code Review Q#74156, answer score: 6

Revisions (0)

No revisions yet.