patternpythonMinor
Reducing memory usage when comparing two iterables
Viewed 0 times
iterablescomparingreducingusagetwomemorywhen
Problem
I'm changing code that writes data to a DB, so I have a dump (a text file) of an earlier run to compare against, to ensure that my changes don't screw things up. Here goes:
This code runs fine, but I'm worried that
[sidenote] I left out exception handling for the sake of simplicity/clarity.
def dbcheck(cursor):
dbresult = list()
cursor.execute("SELECT COLUMN FROM TABLE")
for item in cursor.fetchall():
line = item[0] + "\n"
dbresult.append(line)
with open(dbdump) as f:
for n, line in enumerate(f):
if line != dbresult[n]:
print("DB content does not match original data!")This code runs fine, but I'm worried that
dbresult can grow really large, so am looking for a less risky way of doing this. I'm also curious of what else can be improved.[sidenote] I left out exception handling for the sake of simplicity/clarity.
Solution
This should do it:
No need to read either the whole column nor the whole file before iterating.
def dbcheck(cursor):
cursor.execute("SELECT COLUMN FROM TABLE")
with open(dbdump) as f:
for item in cursor:
if f.readline() != item + '\n'
print("DB content does not match original data!")No need to read either the whole column nor the whole file before iterating.
Code Snippets
def dbcheck(cursor):
cursor.execute("SELECT COLUMN FROM TABLE")
with open(dbdump) as f:
for item in cursor:
if f.readline() != item + '\n'
print("DB content does not match original data!")Context
StackExchange Code Review Q#1669, answer score: 5
Revisions (0)
No revisions yet.