HiveBrain v1.2.0
Get Started
← Back to all entries
patternpythonMinor

Python: Open Multiple Files

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
openfilespythonmultiple

Problem

I am trying to open two CSV files, one for reading and one for writing. The script should continue only if both files were successfully opened. My code seems to accomplish that but it seems like it could be improved further.

My questions are:

  • Should I consider a different approach?



  • My code feels a bit bloated inside the try statement (imagine 100 additional lines of code). Is there a "cleaner" approach?



  • If the files were opened successfully, but a different error occurs later on that is not an IOError, this code will not catch it, correct?



Code:

input_file = 'in_file.csv'
output_file = 'out_file.csv'

def main():

    try:
        with open(input_file, 'rb') as in_csv, open(output_file , 'wb') as out_csv:
            writer = csv.writer(out_csv, delimiter='\t')
            reader = csv.reader(in_csv, delimiter='\t')
            for row in reader:
                # many lines of code...
    except IOError as e:
        print "Error: cannot open file"
        if e.errno == errno.EACCES:
            print "\tPermission denied."
            print "\tError message: {0}".format(e)
            sys.exit()
        # Not a permission error.
        print "\tDoes file exist?"
        print "\tError message: {0}".format(e)
        sys.exit()

if __name__ == '__main__':
    main()

Solution

To answer your questions…

  • Your approach is fine.



  • Move your hundreds of lines into a function, then break down that function into reasonable-sized chunks. If it's still huge and ugly, then pose that as another code review question.



  • If an error other an IOError occurred, then the error would not get caught. However, when the with block is exited for any reason, input_file and output_file will get properly closed.



In addition to what @JimDennis said, I would like to point out that it it customary to print error messages to sys.stderr, and exit with a non-zero status when an error occurs.

def process(csv_reader, csv_writer):
    for row in csv_reader:
        # many lines of code...

def main(input_filename, output_filename):
    try:
        with open(input_filename, 'rb') as in_csv, open(output_filename , 'wb') as out_csv:
            writer = csv.writer(out_csv, delimiter='\t')
            reader = csv.reader(in_csv, delimiter='\t')
            process(reader, writer)
    except IOError as e:
        print >> sys.stderr, "Error: cannot open file"
        if e.errno == errno.EACCES:
            print >> sys.stderr, "\tPermission denied."
            print >> sys.stderr, "\tError message: {0}".format(e)
            sys.exit(1)
        # Not a permission error.
        print >> sys.stderr, "\tError message: {0}".format(e)
        sys.exit(1)
    except Exception as other_exception:
        print >> sys.stderr, "Error: " + str(other_exception)
        sys.exit(2)

if __name__ == '__main__':
    main('in_file.csv', 'out_file.csv')

Code Snippets

def process(csv_reader, csv_writer):
    for row in csv_reader:
        # many lines of code...

def main(input_filename, output_filename):
    try:
        with open(input_filename, 'rb') as in_csv, open(output_filename , 'wb') as out_csv:
            writer = csv.writer(out_csv, delimiter='\t')
            reader = csv.reader(in_csv, delimiter='\t')
            process(reader, writer)
    except IOError as e:
        print >> sys.stderr, "Error: cannot open file"
        if e.errno == errno.EACCES:
            print >> sys.stderr, "\tPermission denied."
            print >> sys.stderr, "\tError message: {0}".format(e)
            sys.exit(1)
        # Not a permission error.
        print >> sys.stderr, "\tError message: {0}".format(e)
        sys.exit(1)
    except Exception as other_exception:
        print >> sys.stderr, "Error: " + str(other_exception)
        sys.exit(2)

if __name__ == '__main__':
    main('in_file.csv', 'out_file.csv')

Context

StackExchange Code Review Q#31395, answer score: 4

Revisions (0)

No revisions yet.