patterncsharpMinor
Fastest way searching specific files
Viewed 0 times
searchingwayfilesfastestspecific
Problem
I got thousands of files with a specific file extension in thousands of sub folders. Now, what is the fastest way to search with a pattern? I tried the method
Is there any faster way to do it?
DirectoryInfo.GetFiles(rootfolder) (~8 minutes) and a recursive custom method (~5 minutes).private void WalkDirectoryTree(DirectoryInfo dr, string searchname)
{
System.IO.FileInfo[] files = null;
System.IO.DirectoryInfo[] subDirs = null;
try
{
files = dr.GetFiles(searchname + ".*");
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
if (files != null)
{
foreach (FileInfo fi in files)
{
allFiles.Add(fi);
}
subDirs = dr.GetDirectories();
foreach (DirectoryInfo di in subDirs)
{
WalkDirectoryTree(di, searchname);
}
}
}Is there any faster way to do it?
Solution
Try parallelization. Instead of:
Do
Notice that by doing this
foreach (DirectoryInfo di in subDirs)
{
WalkDirectoryTree(di, searchname);
}Do
Parallel.ForEach(subDirs, dir => WalkDirectoryTree(dir, searchname));Notice that by doing this
allFiles will be accessed concurrently so change your collection to a ConcurrentBag.Code Snippets
foreach (DirectoryInfo di in subDirs)
{
WalkDirectoryTree(di, searchname);
}Parallel.ForEach(subDirs, dir => WalkDirectoryTree(dir, searchname));Context
StackExchange Code Review Q#74156, answer score: 6
Revisions (0)
No revisions yet.