patternMinor
Reading CSV files pertaining to financial recorts
Viewed 0 times
readingrecortspertainingcsvfinancialfiles
Problem
I have a number of text files (they are reports from a financial system) stored in a directory. Each report has a particular piece of text to search for (eg Report Total) and then a range of characters to extract for the value (e.g. position 78 for 10 characters).
There are a number of these reports - but no more than 200 so not a huge amount of processing or work required.
The premise of my solution is to maintain a simple CSV file with the report name, text to search for, start value for extract, length of characters to extract. Then, it is a simple case of updating this text file to add or remove the reports as necessary.
The script I have developed is below, and it works, but I would be interested in some critique as to how it may be made "better".
What needs changing/improving and why?
There are a number of these reports - but no more than 200 so not a huge amount of processing or work required.
The premise of my solution is to maintain a simple CSV file with the report name, text to search for, start value for extract, length of characters to extract. Then, it is a simple case of updating this text file to add or remove the reports as necessary.
The script I have developed is below, and it works, but I would be interested in some critique as to how it may be made "better".
What needs changing/improving and why?
#!/usr/bin/perl -w
use strict;
use strict;
use Data::Dumper;
use Cwd;
use Text::CSV;
my $SrcDir="/data03/adudata/live/work/";
my $RepDir="/data03/adudata/live/archive/";
my $ReportList='pl_rec.list';
my $csv = Text::CSV->new();
my @FileToRead;
my @TextToFind;
my @RowsOffset;
my @DataStart;
my @DataEnd;
my %ReportData;
chdir $SrcDir;
open my $fh, 'getline( $fh )) {
push (@FileToRead, $row->[0]);
push (@TextToFind, $row->[1]);
push (@RowsOffset, $row->[2]);
push (@DataStart, $row->[3]);
push (@DataEnd, $row->[4]);
}
close ($fh);
for my $i(0 .. $#FileToRead) {
open $fh, ") {
if (grep /$TextToFind[$i]/, $_) {
my $extractValue = substr($_,$DataStart[$i],$DataEnd[$i]);
$extractValue =~s/(^\s+)|(s+$)//g;
$extractValue =~ s/,//g;
$ReportData{$TextToFind[$i]} = $extractValue+0;
}
}
close $fh;
}
while(my($code,$value)=each(%ReportData)){
print "$code,$value\n";
}Solution
The point of filehandles is that it allows you to open more than one file simultaneously. So you could be reading the file list and the files you're searching through simultaneously.
Also, this:
Is a a really convoluted way to write:
Overall, I'd do something like this:
Also, this:
if (grep /$TextToFind[$i]/, $_)Is a a really convoluted way to write:
if (/$TextToFind[$i]/)Overall, I'd do something like this:
#!/usr/bin/env perl
use strict;
use warnings;
use Text::CSV;
my $SrcDir = "/data03/adudata/live/work/";
my $RepDir = "/data03/adudata/live/archive/";
my $ReportList = 'pl_rec.list';
open my $list_fh, 'new();
while (my $row = $csv->getline( $list_fh ))
{
my ($filename, $text, $rows_offset, $data_start, $data_end) = @$row;
open my $fh, ')
{
next unless /\Q$text_to_find/;
my $extractValue = substr($_, $data_start, $data_end);
$extractValue =~ s/(^\s+)|(s+$)//g;
$extractValue =~ s/,//g;
$ReportData{$text} = $extractValue + 0;
}
}
for (sort keys %ReportData)
{
my $value = $ReportData{$_};
print "$_,$value\n";
}Code Snippets
if (grep /$TextToFind[$i]/, $_)if (/$TextToFind[$i]/)#!/usr/bin/env perl
use strict;
use warnings;
use Text::CSV;
my $SrcDir = "/data03/adudata/live/work/";
my $RepDir = "/data03/adudata/live/archive/";
my $ReportList = 'pl_rec.list';
open my $list_fh, '<', $SrcDir.$ReportList
or die "Could not open $ReportList: $!";
my %ReportData;
my $csv = Text::CSV->new();
while (my $row = $csv->getline( $list_fh ))
{
my ($filename, $text, $rows_offset, $data_start, $data_end) = @$row;
open my $fh, '<', $RepDir.$filename
or die "Could not open $filename: $!";
while (<$fh>)
{
next unless /\Q$text_to_find/;
my $extractValue = substr($_, $data_start, $data_end);
$extractValue =~ s/(^\s+)|(s+$)//g;
$extractValue =~ s/,//g;
$ReportData{$text} = $extractValue + 0;
}
}
for (sort keys %ReportData)
{
my $value = $ReportData{$_};
print "$_,$value\n";
}Context
StackExchange Code Review Q#36681, answer score: 5
Revisions (0)
No revisions yet.