patterncsharpMinor
Divide certain distinct row values into separate columns
Viewed 0 times
dividedistinctcolumnsintoseparatevaluescertainrow
Problem
This might be difficult for me to describe clearly but I will try anyway. Also note that this is entirely speculative; it might not even be able to run any faster (that is why I am asking).
This is a question directed at performance. I have been trying to improve the performance of a method and have managed to improve it quite a bit already (around 40% faster). Right now, the entire process takes about 1 minute and 15 seconds to complete. I would like to see if maybe you guys can see some obvious faults in my code and how we could perhaps improve it even further, seeing as I am not quite satisfied yet.
The method fetches a big amount of data from a couple of databases and needs to be displayed in a grid in a certain way. Here is the heavy part of the method. I will describe it in more detail after:
```
DataClasses1DataContext dbContext = new DataClasses1DataContext(sqlConnectionString);
//Find minimum value of N in the specified timespan
var headerMin = ((from y in dbContext.Testheaders
where Convert.ToDateTime(y.StartTime) >= GlobalVariables.StartTimeMSA
select y.N).Min());
//Find maximum value of N in the specified timespan
var headerMax = ((from y in dbContext.Testheaders
where Convert.ToDateTime(y.StartTime) = headerMin
&&
x.HeaderID new { x.Step, x.LogID }).Distinct().ToList().OrderBy(x => x.Step).ThenBy(x => x.LogID);
//Add columns to datatable corresponding to each function. The number of columns is only known at runtime.
foreach (var item in functionDistinct)
{
var column = new DataColumn() { DataType = typeof(double), ColumnName = String.Format("{0}-{1}", item.Step, item.LogID), AllowDBNull = true };
msaDataTable.Columns.Add(column);
}
var distinctHeaderID = msaData.Select(x => x.HeaderID).Distinct();
foreach (var headerID in distinctHeaderID)
{
ct.ThrowIfCancellationRequested();
//Fetch the data for the current headerID
This is a question directed at performance. I have been trying to improve the performance of a method and have managed to improve it quite a bit already (around 40% faster). Right now, the entire process takes about 1 minute and 15 seconds to complete. I would like to see if maybe you guys can see some obvious faults in my code and how we could perhaps improve it even further, seeing as I am not quite satisfied yet.
The method fetches a big amount of data from a couple of databases and needs to be displayed in a grid in a certain way. Here is the heavy part of the method. I will describe it in more detail after:
```
DataClasses1DataContext dbContext = new DataClasses1DataContext(sqlConnectionString);
//Find minimum value of N in the specified timespan
var headerMin = ((from y in dbContext.Testheaders
where Convert.ToDateTime(y.StartTime) >= GlobalVariables.StartTimeMSA
select y.N).Min());
//Find maximum value of N in the specified timespan
var headerMax = ((from y in dbContext.Testheaders
where Convert.ToDateTime(y.StartTime) = headerMin
&&
x.HeaderID new { x.Step, x.LogID }).Distinct().ToList().OrderBy(x => x.Step).ThenBy(x => x.LogID);
//Add columns to datatable corresponding to each function. The number of columns is only known at runtime.
foreach (var item in functionDistinct)
{
var column = new DataColumn() { DataType = typeof(double), ColumnName = String.Format("{0}-{1}", item.Step, item.LogID), AllowDBNull = true };
msaDataTable.Columns.Add(column);
}
var distinctHeaderID = msaData.Select(x => x.HeaderID).Distinct();
foreach (var headerID in distinctHeaderID)
{
ct.ThrowIfCancellationRequested();
//Fetch the data for the current headerID
Solution
I'm thinking that the 2 calls at the beginning might be reduced to one. Which should cut down the time somewhat. Something like this:
In a similar vein extracting the appropriate header, only once for each record your looking at in testDatas should eliminate the queries your doing in the new block. Something like this:
var headers = ((from y in dbContext.Testheaders
let startTime = Convert.ToDateTime(y.StartTime)
where startTime >= GlobalVariables.StartTimeMSA &&
startTime <= GlobalVariables.EndTimeMSA
select y.N));
//Find minimum value of N in the specified timespan
var headerMin = headers.Min();
//Find maximum value of N in the specified timespan
var headerMax = headers.Max());In a similar vein extracting the appropriate header, only once for each record your looking at in testDatas should eliminate the queries your doing in the new block. Something like this:
var msaData = from x in dbContext.Testdatas
where x.HeaderID >= headerMin
&&
x.HeaderID <= headerMax
let header = (from h in dbContext.Testheaders
where x.HeaderID == h.N
select h).FirstOrDefault()
select new
{
StartTime = Convert.ToDateTime(header.StartTime),
DUT_id = Convert.ToString(header.DUT_id),
Meas = (double)x.Meas,
x.Step,
x.LogID,
x.HeaderID,
printID = Convert.ToInt32(header.PrintID),
Prod_Blok = Convert.ToInt32(header.Prod_Blok),
Rep_Count = Convert.ToInt32(header.Rep_Count),
Operator = Convert.ToString(header.Operator),
};Code Snippets
var headers = ((from y in dbContext.Testheaders
let startTime = Convert.ToDateTime(y.StartTime)
where startTime >= GlobalVariables.StartTimeMSA &&
startTime <= GlobalVariables.EndTimeMSA
select y.N));
//Find minimum value of N in the specified timespan
var headerMin = headers.Min();
//Find maximum value of N in the specified timespan
var headerMax = headers.Max());var msaData = from x in dbContext.Testdatas
where x.HeaderID >= headerMin
&&
x.HeaderID <= headerMax
let header = (from h in dbContext.Testheaders
where x.HeaderID == h.N
select h).FirstOrDefault()
select new
{
StartTime = Convert.ToDateTime(header.StartTime),
DUT_id = Convert.ToString(header.DUT_id),
Meas = (double)x.Meas,
x.Step,
x.LogID,
x.HeaderID,
printID = Convert.ToInt32(header.PrintID),
Prod_Blok = Convert.ToInt32(header.Prod_Blok),
Rep_Count = Convert.ToInt32(header.Rep_Count),
Operator = Convert.ToString(header.Operator),
};Context
StackExchange Code Review Q#74164, answer score: 2
Revisions (0)
No revisions yet.