So weakref.proxy
objects don't at all seem to work like weakref.ref
objects when it comes to checking if the reference is live, or 'dereferencing' them, or really in just about any way at all, actually. :P
They do still seem to have their place - such as maintaining a list of objects which respond to events. Since one doesn't need to deref it to call methods on the objects they are weakly referencing (unlike weakref.ref
objects which need to be called first), one does gain some time back from the use of proxy
objects over thousands of iterations. However, they seem to be harder to actually identify as 'dead' and summarily clean up once the object they are referencing disappears. Par example -
>>> mylist #note the object at mylist[1] is already dead...
[<weakproxy at 0x10ccfe050 to A at 0x10ccf9f50>, <weakproxy at 0x10ccfe1b0 to NoneType at 0x10cb53538>]
>>> for i in mylist[:]:
... try:
... getattr(i, 'stuff') #the second arg could be any sentinel;
... #I just want to hit a ReferenceError
... except AttributeError:
... pass
... except ReferenceError:
... mylist.remove(i)
...
Traceback (most recent call last):
File "<stdin>", line 7, in <module>
ReferenceError: weakly-referenced object no longer exists
So it lives there, it just can't really be referenced directly or passed around like a variable, because that apparently causes Python to 'dereference' it and work with the object it's weakly pointing at. (Or something.)
The only thing I've found so far that seems to be somewhat reliable is to route it through a try/except, but it feels a bit like a runaround.
for i, obj in enumerate(mylist):
try:
obj.__hash__
except ReferenceError:
del mylist[i]
It works, but Python tends to be a "one right answer for all the things" language, and this feels like a pretty hefty solution - I might be wrong on this, but if the list is sufficiently large, doesn't copying the list in this way lend itself to issues?
Unfortunately this is all I can seem to really think of as a potential solve that doesn't involve type-checking or other garbage, but I'm guessing I just missed something in the weakref
documentation about how to handle weakref.proxy
objects appropriately. It 'feels like' looking to ensure a weakref.proxy
isn't exceptional circumstances, so using a try/except
is a one-off utility.
So if I am right in my assumption about the use of a try/except
here, is there a better method for identifying dead weakref.proxy
objects?
EDIT: I've accepted an answer, so thank you for that - try/except seems like the only acceptable out.
As to why I'm not using WeakSets - allow me to post a simple demonstration which ultimately led me to the use of proxy objects.
>>> from weakref import WeakSet, proxy
>>> import cProfile
>>> class A(object):
... def __init__(self):
... self.x = 0
... def foo(self, v=1):
... self.x += v
...
>>> Stick = A()
>>> Dave = A()
>>> Jupiter = A()
>>> ##just a list of objects, no fancy
>>> one_group = [Stick, Dave, Jupiter]
>>> cProfile.run("for i in xrange(0, 2**16): [x.foo(i) for x in one_group]")
196610 function calls in 0.136 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
196608 0.049 0.000 0.049 0.000 <stdin>:4(foo)
1 0.087 0.087 0.136 0.136 <string>:1(<module>)
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
>>> Stick.x
2147450880
>>> Dave.x
2147450880
>>> Jupiter.x
2147450880
So we know it works, and we know that it is fairly fast for 65k iterations. Seems decent; let's take a look at WeakSets.
>>> ##now a WeakSet of objects. should be ideal; but...
>>> two_group = WeakSet((Stick, Dave, Jupiter))
>>> cProfile.run("for i in xrange(0, 2**16): [x.foo(i) for x in two_group]")
851970 function calls in 0.545 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
196608 0.055 0.000 0.055 0.000 <stdin>:4(foo)
1 0.158 0.158 0.545 0.545 <string>:1(<module>)
65536 0.026 0.000 0.026 0.000 _weakrefset.py:16(__init__)
65536 0.043 0.000 0.051 0.000 _weakrefset.py:20(__enter__)
65536 0.063 0.000 0.095 0.000 _weakrefset.py:26(__exit__)
65536 0.024 0.000 0.024 0.000 _weakrefset.py:52(_commit_removals)
262144 0.159 0.000 0.331 0.000 _weakrefset.py:58(__iter__)
65536 0.009 0.000 0.009 0.000 {method 'add' of 'set' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
65536 0.008 0.000 0.008 0.000 {method 'remove' of 'set' objects}
Premature optimization? Nah. :) That's about 4x slower. Blimey.
>>> ##now finally, a list of proxy objects
>>> three_group = [proxy(x) for x in one_group]
>>> cProfile.run("for i in xrange(0, 2**16): [x.foo(i) for x in three_group]")
196610 function calls in 0.139 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
196608 0.050 0.000 0.050 0.000 <stdin>:4(foo)
1 0.089 0.089 0.139 0.139 <string>:1(<module>)
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
Based on these numbers, I came to the idea that the simplest and fastest method for maintaining a list of objects which must correctly be noted as deallocated is to use proxy objects. I think that strategic use of the try/except
as a 'list cleaner' will ensure that dead proxies don't cause errors.
For their convenience, I may yet return to an approach which employs weakref.ref
objects, but the use of proxy objects is intriguing for their seemingly 'direct' access to the object.