Unable to deserialize PyMongo ObjectId from JSON

Question

I'm seemingly unable to deserialize my MongoDB JSON document with the BSON json_util.

The json.loads function is choking on the ObjectId() string. I had understood json_util capable of handling MongoDB's ObjectId format and transforming into usable JSON.

Python code:

import json    
from bson import json_util

s = "{u'_id': ObjectId('4ed559abf047050c58000000')}"
u = json.loads(s, object_hook=json_util.object_hook)

I get the decoder exception:

...
    u = json.loads(s, object_hook=json_util.object_hook)
  File "\python27\lib\json\__init__.py", line 339, in loads
    return cls(encoding=encoding, **kw).decode(s)
  File "\python27\lib\json\decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "\python27\lib\json\decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 1 column 1 (char 1)

Am I missing something?

Answer 1

I think your string form actually looks like the python representation...

s = '{"_id": {"$oid": "4edebd262ae5e93b41000000"}}'
u = json.loads(s, object_hook=json_util.object_hook)

print u  # Result:  {u'_id': ObjectId('4edebd262ae5e93b41000000')}

s = json.dumps(u, default=json_util.default)

print s  # Result:  {"_id": {"$oid": "4edebd262ae5e93b41000000"}}

The bson.json_util.object_hook function does not seem to have any type of handling for there being ObjectId() in the actual json string representation.

Answer 2

There are two problems here:

1. The string you're attempting to JSON-decode is not JSON, it's the string representation of a Python dictionary. In particular, the problem is that u'_id' is not a valid JSON key (JSON keys are quoted strings; the "u" here indicates a Python unicode string, which is meaningless in JSON)

2. json_util.object_hook doesn't make ObjectId available to JSON; the json module will decode the JSON, and then call the object_hook callback with each decoded object. json_util.object_hook will look for certain patterns as defined in the strict mode of MongoDB Extended JSON.

See @jdi's answer for examples of how to properly use json_util.