HiveBrain v1.2.0
Get Started
← Back to all entries
patternpythonMinor

Python common file operations generalization

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
generalizationfileoperationspythoncommon

Problem

Functions with common and simple file operations have started to repeat in my code. Many like this one, but with subtle differences for example:

def do_something_with_file(file_name, access_mode, data):
    with open(file_name, access_mode) as f:
        f.do_something(data)


This is my solution so far:

import pickle
from functools import partial

def _open_file(func, access_mode, file_name, *args, **kwargs):
    with open(file_name, access_mode) as f:
        return func(f, *args, **kwargs)

open_file = partial(_open_file, lambda f: f, 'r')

read_lines = partial(_open_file, lambda f: list(f), 'r')

write_lines = partial(_open_file, lambda f, d: f.writelines(d), 'w')

load_pickle = partial(_open_file, lambda f: pickle.load(f), 'rb')

save_pickle = partial(_open_file, lambda f, d: pickle.dump(d, f), 'wb')


Example usage:

write_lines('file.txt', data)
pickle = load_pickle('file.pickle')


It seems readable enough for simple tasks and more complicated function definitions could also be passed to partial instead of simple anonymous functions like above.

Does anyone see any drawbacks of using this or has suggestions on how it could be improved?

Solution

The code is straighforward and reads good; there is not much to say per se. I would just advise to use f.readlines instead of list(f).

However, there exist a pattern in Python when you need to wraps functions using a common behavior: decorators.

I suggest having a decorator taking the open_mode as parameter:

import pickle
from functools import wraps

def opening(access_mode='r'):
    def decorator(func):
        @wraps(func)
        def wrapper(filename, *args, **kwargs):
            with open(filename, access_mode) as f:
                return func(f, *args, **kwargs)
        return wrapper
    return decorator

@opening()
def read_lines(file_):
    return file_.readlines()

@opening('w')
def write_lines(file_, data):
    file_.writelines(data)

@opening('rb')
def load_pickle(file_):
    pickle.load(file_)

@opening('wb')
def save_pickle(file_, data):
    pickle.dump(data, file_)


Usage is the same.

Code Snippets

import pickle
from functools import wraps

def opening(access_mode='r'):
    def decorator(func):
        @wraps(func)
        def wrapper(filename, *args, **kwargs):
            with open(filename, access_mode) as f:
                return func(f, *args, **kwargs)
        return wrapper
    return decorator

@opening()
def read_lines(file_):
    return file_.readlines()

@opening('w')
def write_lines(file_, data):
    file_.writelines(data)

@opening('rb')
def load_pickle(file_):
    pickle.load(file_)

@opening('wb')
def save_pickle(file_, data):
    pickle.dump(data, file_)

Context

StackExchange Code Review Q#126426, answer score: 2

Revisions (0)

No revisions yet.