patternpythonMinor
Reading and processing a file using Pandas
Viewed 0 times
readingfilepandasusingandprocessing
Problem
I am trying to read a file using pandas and then process it. For opening the file I use the following function:
My main concerns are the
import os
import pandas as pd
def read_base_file(data_folder, base_file):
files = map(lambda x: os.path.join(data_folder, x), os.listdir(data_folder))
if base_file in files:
try:
df = pd.read_csv(base_file, na_values=["", " ", "-"])
except Exception, e:
print "Error in reading", base_file
print e
df = pd.DataFrame()
else:
print "File Not Found."
df = pd.DataFrame()
return dfMy main concerns are the
if statement and what I should return if there is an error.Solution
Generator expression
I advice using a generator expression instead of
should become:
Also
Separation of concerns
You both print and return values, if the printing is for debugging purposes, use
Specific Exception
If you write:
any Exception will be caught, I suggest
I advice using a generator expression instead of
map:map(lambda x: os.path.join(data_folder, x), os.listdir(data_folder))should become:
(os.path.join(data_folder, x) for x in os.listdir(data_folder))Also
x should be renamed to something more expressive.Separation of concerns
You both print and return values, if the printing is for debugging purposes, use
logger.logSpecific Exception
If you write:
except Exception, e:any Exception will be caught, I suggest
IOException.Code Snippets
map(lambda x: os.path.join(data_folder, x), os.listdir(data_folder))(os.path.join(data_folder, x) for x in os.listdir(data_folder))except Exception, e:Context
StackExchange Code Review Q#97724, answer score: 4
Revisions (0)
No revisions yet.