patternpythonCritical
Usage of __slots__?
Viewed 0 times
__slots__usagestackoverflow
Problem
What is the purpose of
__slots__ in Python — especially with respect to when I would want to use it, and when not?Solution
TLDR
The special attribute
The space savings is from:
Quick Caveats
Small caveat, you should only declare a particular slot one time in an inheritance tree. For example:
Python doesn't object when you get this wrong (it probably should), and problems might not otherwise manifest, but your objects will take up more space than they should. Python 3.8:
This is because
The biggest caveat is for multiple inheritance - multiple "parent classes with nonempty slots" cannot be combined.
To accommodate this restriction, follow best practices: create abstractions with empty
See section on multiple inheritance below for an example.
Requirements
-
To have attributes named in
-
To prevent the creation of a
There are a lot of details if you wish to keep reading.
Why use
Faster attribute access
The creator of Python, Guido van Rossum, states that he actually created
It's trivial to demonstrate measurably significant speedup:
and
The slotted access is almost 30% faster in Python 3.5 on Ubuntu.
In Python 2 on Windows I have measured it about 15% faster.
Memory Savings
Another purpose of
My own contribution to the documentation clearly states the reasons behind this:
The space saved over using
SQLAlchemy attributes a lot of memory savings to
To verify this, using the Anaconda distribution of Python 2.7 on Ubuntu Linux, with
In contrast, a class instance with
For 64 bit Python, I illustrate the memory consumption in bytes in Python 2.7 and 3.6, for
So, in spite of smaller dicts in Python 3, we see how nicely `__sl
The special attribute
__slots__ allows you to explicitly state which instance attributes you expect your object instances to have, with the expected results:- faster attribute access.
- space savings in memory.
The space savings is from:
- Storing value references in slots instead of
__dict__.
- Denying
__dict__and__weakref__creation if parent classes deny them and you declare__slots__. This has the effect of denying the creation of non-slotted attributes on its instances, including within the class body (such as in methods like__init__).
Quick Caveats
Small caveat, you should only declare a particular slot one time in an inheritance tree. For example:
class Base:
__slots__ = 'foo', 'bar'
class Right(Base):
__slots__ = 'baz',
class Wrong(Base):
__slots__ = 'foo', 'bar', 'baz' # redundant foo and bar
Python doesn't object when you get this wrong (it probably should), and problems might not otherwise manifest, but your objects will take up more space than they should. Python 3.8:
>>> from sys import getsizeof
>>> getsizeof(Right()), getsizeof(Wrong())
(56, 72)
This is because
Base's slot descriptor has a slot separate from Wrong's. This shouldn't usually come up, but it could:>>> w = Wrong()
>>> w.foo = 'foo'
>>> Base.foo.__get__(w)
Traceback (most recent call last):
File "", line 1, in
AttributeError: foo
>>> Wrong.foo.__get__(w)
'foo'
The biggest caveat is for multiple inheritance - multiple "parent classes with nonempty slots" cannot be combined.
To accommodate this restriction, follow best practices: create abstractions with empty
__slots__ for every parent class (or for every parent class but one), then inherit from these abstractions instead of their concrete versions in your new concrete class. (The original parent classes should also inherit from their respective abstractions, of course.)See section on multiple inheritance below for an example.
Requirements
-
To have attributes named in
__slots__ to actually be stored in slots instead of a __dict__, a class must inherit from object (automatic in Python 3, but must be explicit in Python 2).-
To prevent the creation of a
__dict__, you must inherit from object and all classes in the inheritance must declare __slots__ and none of them can have a '__dict__' entry.There are a lot of details if you wish to keep reading.
Why use
__slots__Faster attribute access
The creator of Python, Guido van Rossum, states that he actually created
__slots__ for faster attribute access.It's trivial to demonstrate measurably significant speedup:
import timeit
class Foo(object): __slots__ = 'foo',
class Bar(object): pass
slotted = Foo()
not_slotted = Bar()
def get_set_delete_fn(obj):
def get_set_delete():
obj.foo = 'foo'
obj.foo
del obj.foo
return get_set_delete
and
>>> min(timeit.repeat(get_set_delete_fn(slotted)))
0.2846834529991611
>>> min(timeit.repeat(get_set_delete_fn(not_slotted)))
0.3664822799983085
The slotted access is almost 30% faster in Python 3.5 on Ubuntu.
>>> 0.3664822799983085 / 0.2846834529991611
1.2873325658284342
In Python 2 on Windows I have measured it about 15% faster.
Memory Savings
Another purpose of
__slots__ is to reduce the space in memory that each object instance takes up.My own contribution to the documentation clearly states the reasons behind this:
The space saved over using
__dict__ can be significant.SQLAlchemy attributes a lot of memory savings to
__slots__.To verify this, using the Anaconda distribution of Python 2.7 on Ubuntu Linux, with
guppy.hpy (aka heapy) and sys.getsizeof, the size of a class instance without __slots__ declared, and nothing else, is 64 bytes. That does not include the __dict__. Thank you Python for lazy evaluation again, the __dict__ is apparently not called into existence until it is referenced, but classes without data are usually useless. When called into existence, the __dict__ attribute is a minimum of 280 bytes additionally.In contrast, a class instance with
__slots__ declared to be () (no data) is only 16 bytes, and 56 total bytes with one item in slots, 64 with two.For 64 bit Python, I illustrate the memory consumption in bytes in Python 2.7 and 3.6, for
__slots__ and __dict__ (no slots defined) for each point where the dict grows in 3.6 (except for 0, 1, and 2 attributes):Python 2.7 Python 3.6
attrs __slots__ __dict__* __slots__ __dict__* | *(no slots defined)
none 16 56 + 272† 16 56 + 112† | †if __dict__ referenced
one 48 56 + 272 48 56 + 112
two 56 56 + 272 56 56 + 112
six 88 56 + 1040 88 56 + 152
11 128 56 + 1040 128 56 + 240
22 216 56 + 3344 216 56 + 408
43 384 56 + 3344 384 56 + 752So, in spite of smaller dicts in Python 3, we see how nicely `__sl
Code Snippets
Python 2.7 Python 3.6
attrs __slots__ __dict__* __slots__ __dict__* | *(no slots defined)
none 16 56 + 272† 16 56 + 112† | †if __dict__ referenced
one 48 56 + 272 48 56 + 112
two 56 56 + 272 56 56 + 112
six 88 56 + 1040 88 56 + 152
11 128 56 + 1040 128 56 + 240
22 216 56 + 3344 216 56 + 408
43 384 56 + 3344 384 56 + 752Context
Stack Overflow Q#472000, score: 1890
Revisions (0)
No revisions yet.