patternpythonMinor
condensing a list of lists with 'description' lines in Python
Viewed 0 times
withdescriptioncondensingpythonlistslistlines
Problem
I have a long list of data pulled in from a csv file that has comments / descriptions scattered throughout the document. I'd like to collapse the descriptions into the last element of each row of data, and my current solution seems slow and overly complex. Rows with data have values in the first index and description rows are empty strings. This gives me a list of lists like so:
Here's what I'd like to see:
My current solution requires creating whole new lists, one with data and one with descriptions, and then iterating over the data and searching through the descriptions each iteration:
```
# get numbered rows with data (first element is not '')
actual_data = [(i, x) for i, x in enumerate(data) if x[0] != '']
# get numbered rows with descriptions (first element is '')
descriptions = [(i, ' '.join([j for j in x if j != '']))
for i, x in enumerate(data) if x[0] == '']
# get just the indices of the rows with descriptions
description_indices = {i[0] for i in descriptions}
desired_data_attempt
data = [
['a', 'this', 'is', 'data', 1, 2, 3, ''],
['', '', '', 'this is a description', '', '', '', ''],
['', '', '', 'that carries onto two lines', '', '', '', ''],
['another', 'row', 'with', 'data', 0, 3, 1, ''],
['', '', '', 'this is a description', '', '', '', ''],
['', '', '', 'that carries onto three lines', '', '', '', ''],
['', '', '', 'like so', '', '', '', ''],
['data', 'with', 'no', 'description', 9, 2, 0, ''],
['b', 'this', 'is', 'data', 1, 2, 3, ''],
['', '', '', '', 'sometimes the description', 'is offset', '', '']
]Here's what I'd like to see:
desired_data = [
['a', 'this', 'is', 'data', 1, 2, 3,
'this is a description that carries onto two lines'],
['another', 'row', 'with', 'data', 0, 3, 1,
'this is a description that carries onto three lines like so'],
['data', 'with', 'no', 'description', 9, 2, 0, None],
['b', 'this', 'is', 'data', 1, 2, 3, 'sometimes the description is offset']
]My current solution requires creating whole new lists, one with data and one with descriptions, and then iterating over the data and searching through the descriptions each iteration:
```
# get numbered rows with data (first element is not '')
actual_data = [(i, x) for i, x in enumerate(data) if x[0] != '']
# get numbered rows with descriptions (first element is '')
descriptions = [(i, ' '.join([j for j in x if j != '']))
for i, x in enumerate(data) if x[0] == '']
# get just the indices of the rows with descriptions
description_indices = {i[0] for i in descriptions}
desired_data_attempt
Solution
You can do this in one go, just keep a data piece you are going to yield and list of comment items. If you encounter a new comment item - append to list, if you encounter a ata piece - yield one you were holding with all the comments, wipe comments and hold a new data item instead, like this:
def is_comment(item):
return item[0] == ''
def nonempty(item):
return [x for x in item if x]
def fuse_comments(data):
result = data[0]
comments = []
for element in data[1:]:
if is_comment(element):
comments += nonempty(element)
else:
yield result + [' '.join(str(comment) for comment in comments)]
comments = []
result = nonempty(element)
yield result + [' '.join(str(comment) for comment in comments)]
from pprint import pprint
pprint(list(fuse_comments(data)))Code Snippets
def is_comment(item):
return item[0] == ''
def nonempty(item):
return [x for x in item if x]
def fuse_comments(data):
result = data[0]
comments = []
for element in data[1:]:
if is_comment(element):
comments += nonempty(element)
else:
yield result + [' '.join(str(comment) for comment in comments)]
comments = []
result = nonempty(element)
yield result + [' '.join(str(comment) for comment in comments)]
from pprint import pprint
pprint(list(fuse_comments(data)))Context
StackExchange Code Review Q#134869, answer score: 3
Revisions (0)
No revisions yet.