HiveBrain v1.2.0
Get Started
← Back to all entries
patternpythonMinor

Reducing memory usage when comparing two iterables

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
iterablescomparingreducingusagetwomemorywhen

Problem

I'm changing code that writes data to a DB, so I have a dump (a text file) of an earlier run to compare against, to ensure that my changes don't screw things up. Here goes:

def dbcheck(cursor):
    dbresult = list()
    cursor.execute("SELECT COLUMN FROM TABLE")
    for item in cursor.fetchall():
        line = item[0] + "\n"
        dbresult.append(line)
    with open(dbdump) as f:
        for n, line in enumerate(f):
            if line != dbresult[n]:
                print("DB content does not match original data!")


This code runs fine, but I'm worried that dbresult can grow really large, so am looking for a less risky way of doing this. I'm also curious of what else can be improved.

[sidenote] I left out exception handling for the sake of simplicity/clarity.

Solution

This should do it:

def dbcheck(cursor):
    cursor.execute("SELECT COLUMN FROM TABLE")
    with open(dbdump) as f:
        for item in cursor:
            if f.readline() != item + '\n'
                print("DB content does not match original data!")


No need to read either the whole column nor the whole file before iterating.

Code Snippets

def dbcheck(cursor):
    cursor.execute("SELECT COLUMN FROM TABLE")
    with open(dbdump) as f:
        for item in cursor:
            if f.readline() != item + '\n'
                print("DB content does not match original data!")

Context

StackExchange Code Review Q#1669, answer score: 5

Revisions (0)

No revisions yet.