MongoDB/PyMongo won't $set attribute to document - but sets all other attributes! (bizarre error)

Question

I'm trying to write a defaultdict variable to a document in my MongoDB. Everything else sets fine, just not this one attribute, its bizarre! I'm setting a rather large defaultdict called 'domains', which has worked many times before. Check out this terminal output:

So here's my defaultdict:

>>> type(domains) 
<type 'collections.defaultdict'>

Its pretty big, about 3mb:

>>> sys.getsizeof(domains)
3146008

Here's the document we'll set it to:

>>> db.AggregateResults.find_one({'date':'20110409'}).keys()
[u'res', u'date', u'_id']

Let's grab that document's ID:

>>> myID = db.AggregateResults.find_one({'date':'20110409'})['_id']
>>> myID
ObjectId('50870847f49a00509a000000')

Great, let's set the attribute:

>>> db.AggregateResults.update({'_id':myID}, {"$set": {'domains':domains}})
>>> db.AggregateResults.find_one({'date':'20110409'}).keys()
[u'res', u'date', u'_id']

EH? It didn't save??

Hmmm...does anything save at all?

>>> db.AggregateResults.update({'_id':myID}, {"$set": {'myTest':'hello world'}})
>>> db.AggregateResults.find_one({'date':'20110409'}).keys()
[u'myTest', u'res', u'date', u'_id']

Okay...so it can save things fine...perhaps its because MongoDB doesn't like defaultdicts? Let's try:

>>> myDD = defaultdict(int)
>>> myDD['test'] = 1
>>> myDD
defaultdict(<type 'int'>, {'test': 1})
>>> db.AggregateResults.update({'_id':myID}, {"$set": {'myDD':myDD}})
>>> db.AggregateResults.find_one({'date':'20110409'}).keys()
[u'myTest', u'res', u'date', u'myDD', u'_id']

So it can save defaultdicts fine, just not this one??

So strange! Any ideas why??

EDIT with safe=True:

>>> db.AggregateResults.update({'_id':myID}, {"$set": {'domains':domains}}, safe=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/site-packages/pymongo-2.1.1_-py2.6-linux-x86_64.egg/pymongo/collection.py", line 405, in update
    _check_keys, self.__uuid_subtype), safe)
  File "/usr/lib64/python2.6/site-packages/pymongo-2.1.1_-py2.6-linux-x86_64.egg/pymongo/connection.py", line 796, in _send_message
    return self.__check_response_to_last_error(response)
  File "/usr/lib64/python2.6/site-packages/pymongo-2.1.1_-py2.6-linux-x86_64.egg/pymongo/connection.py", line 746, in __check_response_to_last_error
    raise OperationFailure(error["err"], error["code"])
pymongo.errors.OperationFailure: not okForStorage

This GoogleGroup discussion says that could be due to having fullstops in the keys, but:

>>> [x for x in domains.keys() if '.' in x]
[]

Answer 1

Aha! Found it!

Not only can keys in MongoDB not have '.', they also cannot have '$' in them.

See:

>>>[x for x in domains.keys() if '$' in x]
['$some_key_']

Answer 2

My guess is that you are trying to save too large of a document. MongoDB imposes a 16MB maximum size on all of its documents.

Try running the update command with the parameter safe=True. This will run in safe mode, which will instruct the database to send back the result of the attempted insert.