patternpythonMinor
iterating over the values of a list of ordered dictionaries
Viewed 0 times
theiteratingdictionariesvalueslistoverordered
Problem
I am facing the following problem: I have a special data structure that is a dictionary whose keys are integers (dimensions). The values are also dictionaries whose keys are strings (geometric types) and whose values are numpy arrays (connectivities). Something like this:
I use this data structure quite a lot in the code, and every time I need to iterate over all rows in the numpy arrays of this data structure (all values of the dictionaries), I have to type:
And this is way too verbose. So to make things easier to other developers, I am trying to encapsulate this behavior in a custom class so that I can type:
Also, an important requirement is that all rows have to be traversed "in order".
And so I came up with a
The technique works, I an iterate as many times as I want over this data structure, and I accomplished the simplicity I wanted. But I'm a bit concerned about performance, as I don't think that calling
```
import sys
import numpy as np
from collections import OrderedDict
custom = {0: {'n1': np.array([[3]])}, 1: {'l2': np.array([[ 0, 4],
[ 4, 5],
[40, 41],
[41, 42], [57, 3],
[57, 3]])}, 2: {'t3x'
custom = {1: {'a': np.zeros(10), 'b': np.zeros(100)}, 2:{'c': np.zeros(20), 'd': np.zeros(200)}}I use this data structure quite a lot in the code, and every time I need to iterate over all rows in the numpy arrays of this data structure (all values of the dictionaries), I have to type:
for d, delem in custom.items():
for k, v in delem.items():
for row in v:
print(row)And this is way too verbose. So to make things easier to other developers, I am trying to encapsulate this behavior in a custom class so that I can type:
for row in custom:
print(row)Also, an important requirement is that all rows have to be traversed "in order".
And so I came up with a
Test class that derives from list. The class is initialized by passing a dictionary (maybe I can improve this later by just adding the elements directly to it). I have overridden the __iter__ and __next__ methods to provide the iteration type I want. By default, iteration is done through the highest dimension present. But the user can do iteration over lower-dimensional dictionaries and for that I overrode the __call__ method.The technique works, I an iterate as many times as I want over this data structure, and I accomplished the simplicity I wanted. But I'm a bit concerned about performance, as I don't think that calling
self[self.d][self.etype][self.idx-1] is very efficient. Maybe there are other improvements to be made as well that I don't see.```
import sys
import numpy as np
from collections import OrderedDict
custom = {0: {'n1': np.array([[3]])}, 1: {'l2': np.array([[ 0, 4],
[ 4, 5],
[40, 41],
[41, 42], [57, 3],
[57, 3]])}, 2: {'t3x'
Solution
yield!
You're way overthinking the problem. Python has
You just write a function that looks pretty much exactly like that:
We can even drop the last loop if you have a sufficiently recent version of Python that supports
Or you don't actually need
That's it. No need for a custom class to effectively reimplement the same. The initial loop could them be written as:
Note that if you had your own class, you could write the above in
You're way overthinking the problem. Python has
yield. This is one of the coolest things in Python (IMHO). When you want to iterate over a container like this:for d, delem in custom.items():
for k, v in delem.items():
for row in v:
print(row)You just write a function that looks pretty much exactly like that:
def iter_over_custom(custom): ## or a better name
for _, delem in custom.items():
for _, v in delem.items():
for row in v:
yield rowWe can even drop the last loop if you have a sufficiently recent version of Python that supports
yield from:def iter_over_custom(custom): ## or a better name
for _, delem in custom.items():
for _, v in delem.items():
yield from vOr you don't actually need
.items() since you just need the values:def iter_over_custom(custom):
for delem in custom.values():
for v in delem.values():
yield from vThat's it. No need for a custom class to effectively reimplement the same. The initial loop could them be written as:
for row in iter_over_custom(custom):
print(row)Note that if you had your own class, you could write the above in
__iter__ too:class Custom(object):
...
def __iter__(self):
for delem in self.whatever.values():
for v in delem.values():
yield from v
...Code Snippets
for d, delem in custom.items():
for k, v in delem.items():
for row in v:
print(row)def iter_over_custom(custom): ## or a better name
for _, delem in custom.items():
for _, v in delem.items():
for row in v:
yield rowdef iter_over_custom(custom): ## or a better name
for _, delem in custom.items():
for _, v in delem.items():
yield from vdef iter_over_custom(custom):
for delem in custom.values():
for v in delem.values():
yield from vfor row in iter_over_custom(custom):
print(row)Context
StackExchange Code Review Q#114568, answer score: 7
Revisions (0)
No revisions yet.