patterncsharpMinor
Need help to optimise my C# code
Viewed 0 times
codeoptimisehelpneed
Problem
I have a section of my code that seems to take a while during processing. Basically
rawArray contains ~1.5 million rows and tempArray has around 1000 rows. I can see why it's slow as for each of the 1.5 million rows it has to loop through the tempArray and look for a match. Any hints for optimising this would be great!foreach(string[] sArray in rawArray) {
// Loop through tempArray and look for match in SRC and/or DST
foreach (string[] sTemp in tempArray)
{
// SRC matches
if (sArray[0] == sTemp[1])
{
// Set Octets value for upload
if (sTemp[3] == "")
{
sTemp[3] = "0";
}
// Convert to double and increment
sTemp[3] = (System.Convert.ToDouble(sTemp[3]) + System.Convert.ToDouble(sArray[2])).ToString();
// Date & Time
sTemp[5] = date;
sTemp[6] = time;
}
// DST matches
if (sArray[1] == sTemp[1])
{
// Set Octets value for download
if (sTemp[2] == "")
{
sTemp[2] = "0";
}
// Convert to double and increment
sTemp[2] = (System.Convert.ToDouble(sTemp[2]) + System.Convert.ToDouble(sArray[2])).ToString();
// Date & Time
sTemp[5] = date;
sTemp[6] = time;
}
}
}Solution
Well, you could build a dictionary for the temp array but it depends how often the key (index 1 of each temp row) is repeated. If you have very few repetitions then it could reduce the time because it only has to search through a subset of the 1000 temp rows.
Something along these lines:
Then your main loop becomes something like
...
Another major speed improvement can probably be achieved by not using strings for entries which obviously aren't strings. You are doing a lot of double-to-string-and-back conversion which will be slow. If you can convert the input data into double when obtaining it, perform your calculations and then convert it back for output if necessary it should speed things up a lot as well.
Something along these lines:
Dictionary> rowCache = new Dictionary>();
foreach (var row in tempArray)
{
List keyRows;
string key = row[1];
if (!rowCache.TryGetValue(key, out keyRows))
{
keyRows = new List();
rowCache[key] = keyRows;
}
keyRows.Add(row);
}Then your main loop becomes something like
// little helper method
public List GetRows(string key)
{
List rows;
return rowCache.TryGetValue(key, out rows) ? rows : new List();
}...
foreach (var row in rawArray)
{
foreach (var srcRow in GetRows(row[0])
{
// perform your SRC logic here
}
foreach (var dstRow in GetRows(row[1])
{
// perform your DST logic here
}
}Another major speed improvement can probably be achieved by not using strings for entries which obviously aren't strings. You are doing a lot of double-to-string-and-back conversion which will be slow. If you can convert the input data into double when obtaining it, perform your calculations and then convert it back for output if necessary it should speed things up a lot as well.
Code Snippets
Dictionary<string, List<string[]>> rowCache = new Dictionary<string, List<string[]>>();
foreach (var row in tempArray)
{
List<string[]> keyRows;
string key = row[1];
if (!rowCache.TryGetValue(key, out keyRows))
{
keyRows = new List<string[]>();
rowCache[key] = keyRows;
}
keyRows.Add(row);
}// little helper method
public List<string[]> GetRows(string key)
{
List<string[]> rows;
return rowCache.TryGetValue(key, out rows) ? rows : new List<string[]>();
}foreach (var row in rawArray)
{
foreach (var srcRow in GetRows(row[0])
{
// perform your SRC logic here
}
foreach (var dstRow in GetRows(row[1])
{
// perform your DST logic here
}
}Context
StackExchange Code Review Q#61289, answer score: 3
Revisions (0)
No revisions yet.