HiveBrain v1.2.0
Get Started
← Back to all entries
patterncsharpMinor

Divide certain distinct row values into separate columns

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
dividedistinctcolumnsintoseparatevaluescertainrow

Problem

This might be difficult for me to describe clearly but I will try anyway. Also note that this is entirely speculative; it might not even be able to run any faster (that is why I am asking).

This is a question directed at performance. I have been trying to improve the performance of a method and have managed to improve it quite a bit already (around 40% faster). Right now, the entire process takes about 1 minute and 15 seconds to complete. I would like to see if maybe you guys can see some obvious faults in my code and how we could perhaps improve it even further, seeing as I am not quite satisfied yet.

The method fetches a big amount of data from a couple of databases and needs to be displayed in a grid in a certain way. Here is the heavy part of the method. I will describe it in more detail after:

```
DataClasses1DataContext dbContext = new DataClasses1DataContext(sqlConnectionString);

//Find minimum value of N in the specified timespan
var headerMin = ((from y in dbContext.Testheaders
where Convert.ToDateTime(y.StartTime) >= GlobalVariables.StartTimeMSA
select y.N).Min());

//Find maximum value of N in the specified timespan
var headerMax = ((from y in dbContext.Testheaders
where Convert.ToDateTime(y.StartTime) = headerMin
&&
x.HeaderID new { x.Step, x.LogID }).Distinct().ToList().OrderBy(x => x.Step).ThenBy(x => x.LogID);

//Add columns to datatable corresponding to each function. The number of columns is only known at runtime.
foreach (var item in functionDistinct)
{
var column = new DataColumn() { DataType = typeof(double), ColumnName = String.Format("{0}-{1}", item.Step, item.LogID), AllowDBNull = true };

msaDataTable.Columns.Add(column);
}

var distinctHeaderID = msaData.Select(x => x.HeaderID).Distinct();

foreach (var headerID in distinctHeaderID)
{
ct.ThrowIfCancellationRequested();

//Fetch the data for the current headerID

Solution

I'm thinking that the 2 calls at the beginning might be reduced to one. Which should cut down the time somewhat. Something like this:

var headers = ((from y in dbContext.Testheaders
                  let startTime = Convert.ToDateTime(y.StartTime)
                  where startTime >= GlobalVariables.StartTimeMSA &&
                        startTime <= GlobalVariables.EndTimeMSA
                  select y.N));

//Find minimum value of N in the specified timespan
var headerMin = headers.Min();
//Find maximum value of N in the specified timespan
var headerMax = headers.Max());


In a similar vein extracting the appropriate header, only once for each record your looking at in testDatas should eliminate the queries your doing in the new block. Something like this:

var msaData = from x in dbContext.Testdatas
                where x.HeaderID >= headerMin
                                    &&
                    x.HeaderID <= headerMax
                let header = (from h in dbContext.Testheaders
                            where x.HeaderID == h.N
                            select h).FirstOrDefault()  
                select new
                {
                    StartTime = Convert.ToDateTime(header.StartTime),

                    DUT_id = Convert.ToString(header.DUT_id),

                    Meas = (double)x.Meas,

                    x.Step,

                    x.LogID,

                    x.HeaderID,

                    printID = Convert.ToInt32(header.PrintID),

                    Prod_Blok = Convert.ToInt32(header.Prod_Blok),

                    Rep_Count = Convert.ToInt32(header.Rep_Count),

                    Operator = Convert.ToString(header.Operator),
                };

Code Snippets

var headers = ((from y in dbContext.Testheaders
                  let startTime = Convert.ToDateTime(y.StartTime)
                  where startTime >= GlobalVariables.StartTimeMSA &&
                        startTime <= GlobalVariables.EndTimeMSA
                  select y.N));

//Find minimum value of N in the specified timespan
var headerMin = headers.Min();
//Find maximum value of N in the specified timespan
var headerMax = headers.Max());
var msaData = from x in dbContext.Testdatas
                where x.HeaderID >= headerMin
                                    &&
                    x.HeaderID <= headerMax
                let header = (from h in dbContext.Testheaders
                            where x.HeaderID == h.N
                            select h).FirstOrDefault()  
                select new
                {
                    StartTime = Convert.ToDateTime(header.StartTime),

                    DUT_id = Convert.ToString(header.DUT_id),

                    Meas = (double)x.Meas,

                    x.Step,

                    x.LogID,

                    x.HeaderID,

                    printID = Convert.ToInt32(header.PrintID),

                    Prod_Blok = Convert.ToInt32(header.Prod_Blok),


                    Rep_Count = Convert.ToInt32(header.Rep_Count),

                    Operator = Convert.ToString(header.Operator),
                };

Context

StackExchange Code Review Q#74164, answer score: 2

Revisions (0)

No revisions yet.