HiveBrain v1.2.0
Get Started
← Back to all entries
patternjavascriptMinor

Extracting data from database to CSV

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
csvdatabaseextractingfromdata

Problem

I have a feature for users to be able to export the database information. But if the user choose all options to export it takes 1 minute or more to download the .csv file. I'm only including 1 part of the if statement, where I'm pulling in all the data.

here it is:

```
function exportTheData() {
//get the data for data array
if(exportVolumeData == 1) {

for(j=0; j<plantData1.length; j++) {

i = plantData["MergeKey_lvl00"].indexOf(plantData1["MergeKey_lvl00"][j]);

data.push(plantData["PlantName"][i]);

if(statesExport == 1) {
countyindex = counties["CountyId"].indexOf(plantData["LocationId"][i]);
stateid = counties["StateId"][countyindex];
statename = states["StateName"][states["StateId"].indexOf(stateid)];

data.push(statename);
}
if(countyExport == 1) {
countyindex = counties["CountyId"].indexOf(plantData["LocationId"][i]);
countyname = counties["CountyName"][countyindex];

data.push(countyname);
}
if(basinsExport == 1) {
countyindex = counties["CountyId"].indexOf(plantData["LocationId"][i]);
subbasinid = counties["SubBasinId"][countyindex];
subbasinindex = basinSub["SubBasinId"].indexOf(subbasinid);
basinid = basinSub["BasinId"][subbasinindex];
basinindex = basin["BasinId"].indexOf(basinid);
basinname = basin["BasinName"][basinindex];

data.push(basinname);
}
if(subBasinsExport == 1) {
countyindex = counties["CountyId"].indexOf(plantData["LocationId"][i]);
subbasinid = counties["SubBasinId"][countyindex];
subbasinindex = basinSub["SubBasinId"].ind

Solution

Consider replacing data.push(...) with data[data.length] = ...

See this question on Stack Overflow for some of the details - the short version of it is "it's faster on some browsers".

if(statesExport == 1) {
    countyindex = counties["CountyId"].indexOf(plantData["LocationId"][i]);
    stateid = counties["StateId"][countyindex];
    statename = states["StateName"][states["StateId"].indexOf(stateid)];

    data.push(statename);
}
if(countyExport == 1) {
    countyindex = counties["CountyId"].indexOf(plantData["LocationId"][i]);
    countyname = counties["CountyName"][countyindex];

    data.push(countyname);
}
if(basinsExport == 1) {
    countyindex = counties["CountyId"].indexOf(plantData["LocationId"][i]);
    subbasinid = counties["SubBasinId"][countyindex];
    subbasinindex = basinSub["SubBasinId"].indexOf(subbasinid);
    basinid = basinSub["BasinId"][subbasinindex];
    basinindex = basin["BasinId"].indexOf(basinid);
    basinname = basin["BasinName"][basinindex];

    data.push(basinname);
}


In this section of code, you're getting the countyindex three times. Consider calling that only once and then storing that index - indexOf on a large array can be pretty expensive. The same goes for the companyindex later on.

If you really need speed, then you'll possibly have to get rid of data and work directly with a csv string. First storing everything in an array and then converting it to an array is effectively iterating over everything twice.

Code Snippets

if(statesExport == 1) {
    countyindex = counties["CountyId"].indexOf(plantData["LocationId"][i]);
    stateid = counties["StateId"][countyindex];
    statename = states["StateName"][states["StateId"].indexOf(stateid)];

    data.push(statename);
}
if(countyExport == 1) {
    countyindex = counties["CountyId"].indexOf(plantData["LocationId"][i]);
    countyname = counties["CountyName"][countyindex];

    data.push(countyname);
}
if(basinsExport == 1) {
    countyindex = counties["CountyId"].indexOf(plantData["LocationId"][i]);
    subbasinid = counties["SubBasinId"][countyindex];
    subbasinindex = basinSub["SubBasinId"].indexOf(subbasinid);
    basinid = basinSub["BasinId"][subbasinindex];
    basinindex = basin["BasinId"].indexOf(basinid);
    basinname = basin["BasinName"][basinindex];

    data.push(basinname);
}

Context

StackExchange Code Review Q#160138, answer score: 3

Revisions (0)

No revisions yet.