gotchapythonModeratepending
Gotcha: Python string concatenation in loops is O(n squared)
Viewed 0 times
string-concatenationjoinStringIOperformanceO(n^2)
Error Messages
Problem
Building a string by concatenating in a loop (s += 'text') is extremely slow for large strings because Python creates a new string object each iteration.
Solution
Use str.join() or io.StringIO for building strings:
# BAD - O(n^2) due to string immutability:
result = ''
for item in large_list:
result += f'{item}\n' # Creates new string each time
# GOOD - O(n) with join:
result = '\n'.join(str(item) for item in large_list)
# GOOD - O(n) with StringIO for complex building:
from io import StringIO
buf = StringIO()
for item in large_list:
buf.write(f'{item}\n')
result = buf.getvalue()
# GOOD - f-string or format for small fixed concatenations:
full_name = f'{first} {last}' # Fine for fixed number of parts
# BAD - O(n^2) due to string immutability:
result = ''
for item in large_list:
result += f'{item}\n' # Creates new string each time
# GOOD - O(n) with join:
result = '\n'.join(str(item) for item in large_list)
# GOOD - O(n) with StringIO for complex building:
from io import StringIO
buf = StringIO()
for item in large_list:
buf.write(f'{item}\n')
result = buf.getvalue()
# GOOD - f-string or format for small fixed concatenations:
full_name = f'{first} {last}' # Fine for fixed number of parts
Why
Python strings are immutable. Each += creates a new string object and copies all previous content. With n iterations, this is O(n^2) total work.
Revisions (0)
No revisions yet.