Weirdness with mongoengine ReferenceField
-
29-10-2019 - |
Question
This is a puzzling problem that's difficult even to name, let alone describe. I'll start with the essential facts and then give what background information might be relevant.
Consider two mongoengine document models:
class Bar(Document):
# ...
# field definitions
# ...
def bar_func(self):
pass # ...or some arbitrary code
class Foo(Document):
bar = ReferenceField(Bar)
The following is inconsistently producing an AttributeError
on our production server:
# Assume foo_id references a valid Foo document in Mongo
# and that its 'bar' reference is to a valid Bar document.
foo = Foo.objects.with_id(foo_id)
foo.bar.bar_func() # <-- AttributeError on 'bar_func'
If I place debugging code right before the location of the error, evaluating type(foo.bar)
as a string produces <class 'bson.dbref.DBRef'>
. Obviously a DBRef
doesn't have a bar_func
attribute, but why is a DBRef
being returned instead of an instance of Bar
?
Further debugging code shows that the following condition is failing in the ReferenceField.__get__
function in mongoengine/fields.py
:
if isinstance(value, (pymongo.dbref.DBRef)):
value = _get_db().dereference(value)
But (pymongo.dbref.DBRef)
is actually bson.dbref.DBRef
, which seems to be the same as type(foo.bar)
! Why does isinstance
fail?
This is where things get really weird:
id(type(foo.bar)) == id(bson.dbref.DBRef) # <-- Evaluates to False!
In other words, type(foo.bar)
is a different bson.dbref.DBRef
than the one obtained by referencing bson.dbref.DBRef
directly. In fact, inspecting the __dict__
of these two types shows different memory locations for their functions and properties.
Note: I'll call the type returned by type(foo.bar)
fooDBRef
for convenience below, to distinguish it from the type referenced by bson.dbref.DBRef
.
To debug further, I modified the DBRef
code to add a metaclass that inspects the system modules at the time the DBRef
type is created, and stores a list of the IDs of those modules in an extra class attribute of DBRef
. The results show that the set of modules in existence when fooDBRef
is created is entirely distinct from the set of modules in existence when the bare bson.dbref.DBRef
type is created. All module IDs for one are different from all module IDs for the other.
Some possibly relevant factors:
- The server on which this error occurs runs mod_wsgi under Apache.
- The server runs two different Django sites under wsgi (call them
site_a
andsite_b
). - Foo is defined in
site_a.foo_app.models
and Bar is defined insite_b.bar_app.models
. site_a
settings.py hassite_b.bar_app
inINSTALLED_APPS
.- The requests that produce the error are handled by
site_a
. - There were
site_b.*
modules insys.modules
whenfooDBRef
was created, but nosite_a.*
modules. The reverse is true forbson.dbref.DBRef
. - After an
httpd reload
the error sometimes goes away for a little while, and returns sometime within 0-10 attempts.
Can anyone help me figure out what is causing fooDBRef
to be different from bson.dbref.DBRef
?
No correct solution