why utf8 is not working under Mongodb + mongoengine + django?

Question

Ubuntu 10.04.3 LTS mongodb-1.2.2-1ubuntu1.1 django 1.3 mongoengine-0.5.2 pymongo-2.1.2

model:

class User(Document):
    email = StringField(required=True)
    first_name = StringField(max_length=50)
    last_name = StringField(max_length=50)


class Comment(EmbeddedDocument):
    content = StringField()
    name = StringField(max_length=120)

class Post(Document):
    title = StringField(max_length=120, required=True)
    author = ReferenceField(User)
    tags = ListField(StringField(max_length=30))
    comments = ListField(EmbeddedDocumentField(Comment))

class TextPost(Post):
    content = StringField()

class ImagePost(Post):
    image_path = StringField()

class LinkPost(Post):
    link_url = StringField()

trying to save a Post in which the title has the caracter "é":

 john = User(email='jdoe@example.com', first_name='John', last_name='Doe')
    john.save()


post1 = TextPost(title='Fun with MongoEnginée', author=john)
post1.content = 'Took a look at MongoEngine today, looks pretty cool.'
post1.tags = ['mongodb', 'mongoengine']
post1.save()

the following exception is thrown:

Traceback (most recent call last):
  File "/home/raton/aptana_work/test/mongo/test1/cobertura/tests.py", line 27, in create_relato
    post1.save()
  File "/home/raton/aptana_work/test/mongo/env/lib/python2.7/site-packages/mongoengine-0.5.2-py2.7.egg/mongoengine/document.py", line 149, in save
    doc = self.to_mongo()
  File "/home/raton/aptana_work/test/mongo/env/lib/python2.7/site-packages/mongoengine-0.5.2-py2.7.egg/mongoengine/base.py", line 648, in to_mongo
    data[field.db_field] = field.to_mongo(value)
  File "/home/raton/aptana_work/test/mongo/env/lib/python2.7/site-packages/mongoengine-0.5.2-py2.7.egg/mongoengine/base.py", line 127, in to_mongo
    return self.to_python(value)
  File "/home/raton/aptana_work/test/mongo/env/lib/python2.7/site-packages/mongoengine-0.5.2-py2.7.egg/mongoengine/fields.py", line 40, in to_python
    return unicode(value)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 19: ordinal not in range(128)

Any help please??

Answer 1

Try this:

post1 = TextPost(title=u'Fun with MongoEnginée', author=john)
post1.content = 'Took a look at MongoEngine today, looks pretty cool.'
post1.tags = ['mongodb', 'mongoengine']
post1.save()

The important part is declaring your string as unicode: u'Fun with MongoEnginée'

Answer 2

Is your source file encoded in UTF-8 and declared as such? You have to put this magic comment at the top:

#!/usr/bin/env python
# -*- coding: utf8 -*- 

See http://www.python.org/dev/peps/pep-0263/