HiveBrain v1.2.0
Get Started
← Back to all entries
patternpythonMinor

Optimization of average calculation

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
optimizationcalculationaverage

Problem

I have a list of dictionaries, with keys 'a', 'n', 'o', 'u'.
Is there a way to speed up this calculation, for instance with NumPy? There are tens of thousands of items in the list.

The data is drawn from a database, so I must live with that it's in the form of a list of dictionaries originally.

x = n = o = u = 0
for entry in indata:
    x += (entry['a']) * entry['n']  # n - number of data points
    n += entry['n']
    o += entry['o']
    u += entry['u']

    loops += 1

average = int(round(x / n)), n, o, u

Solution

I'm not sure if there really is a better way to do this. The best I could come up with is:

import itertools
from collections import Counter

def convDict(inDict):
    inDict['a'] = inDict['a'] * inDict['n']
    return Counter(inDict)

average = sum(itertools.imap(convDict, inData), Counter())
average['a'] = average['a'] / average['n']


But I'm still not sure if that is better than what you originally had.

Counter is a subclass of dict. You can get items from them the same way you get items from a normal dict. One of the most important differences is that the Counter will not raise an Exception if you try to select a non-existant item, it will instead return 0.

Code Snippets

import itertools
from collections import Counter

def convDict(inDict):
    inDict['a'] = inDict['a'] * inDict['n']
    return Counter(inDict)

average = sum(itertools.imap(convDict, inData), Counter())
average['a'] = average['a'] / average['n']

Context

StackExchange Code Review Q#17810, answer score: 2

Revisions (0)

No revisions yet.