HiveBrain v1.2.0
Get Started
← Back to all entries
patternpythonMinor

condensing a list of lists with 'description' lines in Python

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
withdescriptioncondensingpythonlistslistlines

Problem

I have a long list of data pulled in from a csv file that has comments / descriptions scattered throughout the document. I'd like to collapse the descriptions into the last element of each row of data, and my current solution seems slow and overly complex. Rows with data have values in the first index and description rows are empty strings. This gives me a list of lists like so:

data = [
    ['a', 'this', 'is', 'data', 1, 2, 3, ''],
    ['', '', '', 'this is a description', '', '', '', ''],
    ['', '', '', 'that carries onto two lines', '', '', '', ''],
    ['another', 'row', 'with', 'data', 0, 3, 1, ''],
    ['', '', '', 'this is a description', '', '', '', ''],
    ['', '', '', 'that carries onto three lines', '', '', '', ''],
    ['', '', '', 'like so', '', '', '', ''],
    ['data', 'with', 'no', 'description', 9, 2, 0, ''],
    ['b', 'this', 'is', 'data', 1, 2, 3, ''],
    ['', '', '', '', 'sometimes the description', 'is offset', '', '']
]


Here's what I'd like to see:

desired_data = [
    ['a', 'this', 'is', 'data', 1, 2, 3,
     'this is a description that carries onto two lines'],
    ['another', 'row', 'with', 'data', 0, 3, 1,
     'this is a description that carries onto three lines like so'],
    ['data', 'with', 'no', 'description', 9, 2, 0, None],
    ['b', 'this', 'is', 'data', 1, 2, 3, 'sometimes the description is offset']
]


My current solution requires creating whole new lists, one with data and one with descriptions, and then iterating over the data and searching through the descriptions each iteration:

```
# get numbered rows with data (first element is not '')
actual_data = [(i, x) for i, x in enumerate(data) if x[0] != '']

# get numbered rows with descriptions (first element is '')
descriptions = [(i, ' '.join([j for j in x if j != '']))
for i, x in enumerate(data) if x[0] == '']

# get just the indices of the rows with descriptions
description_indices = {i[0] for i in descriptions}

desired_data_attempt

Solution

You can do this in one go, just keep a data piece you are going to yield and list of comment items. If you encounter a new comment item - append to list, if you encounter a ata piece - yield one you were holding with all the comments, wipe comments and hold a new data item instead, like this:

def is_comment(item):
    return item[0] == ''

def nonempty(item):
    return [x for x in item if x]

def fuse_comments(data):
    result = data[0]
    comments = []
    for element in data[1:]:
        if is_comment(element):
            comments += nonempty(element)
        else:
            yield result + [' '.join(str(comment) for comment in comments)]
            comments = []
            result = nonempty(element)
    yield result + [' '.join(str(comment) for comment in comments)]

from pprint import pprint

pprint(list(fuse_comments(data)))

Code Snippets

def is_comment(item):
    return item[0] == ''

def nonempty(item):
    return [x for x in item if x]

def fuse_comments(data):
    result = data[0]
    comments = []
    for element in data[1:]:
        if is_comment(element):
            comments += nonempty(element)
        else:
            yield result + [' '.join(str(comment) for comment in comments)]
            comments = []
            result = nonempty(element)
    yield result + [' '.join(str(comment) for comment in comments)]

from pprint import pprint

pprint(list(fuse_comments(data)))

Context

StackExchange Code Review Q#134869, answer score: 3

Revisions (0)

No revisions yet.