HiveBrain v1.2.0
Get Started
← Back to all entries
patterncsharpMinor

Need help to optimise my C# code

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
codeoptimisehelpneed

Problem

I have a section of my code that seems to take a while during processing. Basically rawArray contains ~1.5 million rows and tempArray has around 1000 rows. I can see why it's slow as for each of the 1.5 million rows it has to loop through the tempArray and look for a match. Any hints for optimising this would be great!

foreach(string[] sArray in rawArray) {
    // Loop through tempArray and look for match in SRC and/or DST
    foreach (string[] sTemp in tempArray)
    {
        // SRC matches
        if (sArray[0] == sTemp[1])
        {                        
            // Set Octets value for upload
            if (sTemp[3] == "") 
            {
                sTemp[3] = "0";
            }

            // Convert to double and increment
            sTemp[3] = (System.Convert.ToDouble(sTemp[3]) + System.Convert.ToDouble(sArray[2])).ToString();
            // Date & Time
            sTemp[5] = date;
            sTemp[6] = time;
        }

        // DST matches
        if (sArray[1] == sTemp[1])
        {                        
            // Set Octets value for download
            if (sTemp[2] == "")
            {
                sTemp[2] = "0";
            }

            // Convert to double and increment
            sTemp[2] = (System.Convert.ToDouble(sTemp[2]) + System.Convert.ToDouble(sArray[2])).ToString();
            // Date & Time
            sTemp[5] = date;
            sTemp[6] = time;
        }                   
    }                
}

Solution

Well, you could build a dictionary for the temp array but it depends how often the key (index 1 of each temp row) is repeated. If you have very few repetitions then it could reduce the time because it only has to search through a subset of the 1000 temp rows.

Something along these lines:

Dictionary> rowCache = new Dictionary>();

foreach (var row in tempArray)
{
    List keyRows;
    string key = row[1];
    if (!rowCache.TryGetValue(key, out keyRows))
    {
        keyRows = new List();
        rowCache[key] = keyRows;
    }
    keyRows.Add(row);
}


Then your main loop becomes something like

// little helper method
public List GetRows(string key)
{
    List rows;
    return rowCache.TryGetValue(key, out rows) ? rows : new List();
}


...

foreach (var row in rawArray)
{
    foreach (var srcRow in GetRows(row[0])
    {
       // perform your SRC logic here
    }

    foreach (var dstRow in GetRows(row[1])
    {
       // perform your DST logic here
    }
}


Another major speed improvement can probably be achieved by not using strings for entries which obviously aren't strings. You are doing a lot of double-to-string-and-back conversion which will be slow. If you can convert the input data into double when obtaining it, perform your calculations and then convert it back for output if necessary it should speed things up a lot as well.

Code Snippets

Dictionary<string, List<string[]>> rowCache = new Dictionary<string, List<string[]>>();

foreach (var row in tempArray)
{
    List<string[]> keyRows;
    string key = row[1];
    if (!rowCache.TryGetValue(key, out keyRows))
    {
        keyRows = new List<string[]>();
        rowCache[key] = keyRows;
    }
    keyRows.Add(row);
}
// little helper method
public List<string[]> GetRows(string key)
{
    List<string[]> rows;
    return rowCache.TryGetValue(key, out rows) ? rows : new List<string[]>();
}
foreach (var row in rawArray)
{
    foreach (var srcRow in GetRows(row[0])
    {
       // perform your SRC logic here
    }

    foreach (var dstRow in GetRows(row[1])
    {
       // perform your DST logic here
    }
}

Context

StackExchange Code Review Q#61289, answer score: 3

Revisions (0)

No revisions yet.