Django mongoengine odm using secondary instance on replicaset

Question

I have a Django (with mongoengine) website that works on read only mode, and an offline python app that populates that MongoDB (read by Django). Facing some locks problems, I've discovered that a replicaset would be an improvement since I could make the Secondary instance the website DB.

At first I just changed my connection to the secondary instance host, and got the error:

Request Method: GET
Request URL:    http://localhost:8000/awesomeApp/
Django Version: 1.5
Exception Type: AutoReconnect
Exception Value:    
not master

After that, I've tried to solve it with a registered connection, without success and then later upgraded to mongoengine 0.8.7, that could connect with:

connect('awesomeApp', host='mongodb://rs63804-a0.mongoserviceprovider.com:27017,rs63804-a1.mongoserviceprovider.com:27017', replicaSet='rs63804', read_preference=ReadPreference.SECONDARY_PREFERRED, username='scott', password='tiger')

But now I'm getting two different error messages when I use ReadPreference.PRIMARY_PREFERRED or ReadPreference.SECONDARY_PREFERRED...

With PRIMARY_PREFERRED it returns:

Request Method: GET
Request URL:    http://localhost:8000/awesomeapp/
Django Version: 1.5
Exception Type: OperationFailure
Exception Value:    
not authorized to create index on awesomeapp.django_session

And with SECONDARY_PREFERRED:

Request Method: GET
Request URL:    http://localhost:8000/awesomeapp/
Django Version: 1.5
Exception Type: OperationFailure
Exception Value:    
database error: not authorized for query on awesomeapp.django_session

I'm stucked at this step, looking for a way to allow that operation through mongoengine and/or mongodb configs, but couldn't find a solution yet.. Any ideias?

Thanks in advance

Answer 1

As suggested by the errors you are receiving your problem appears to be with the permissions set for the user account you are using for access.

More recent versions of MongoDB from 2.4 upwards have more granular control over user roles http://docs.mongodb.org/manual/reference/user-privileges/ which may have an effect on what you are doing. In particular Mongoengine will have logic to set up indexes on the classes you are implementing and is likely trying to deploy them via ensure index. This would be a problem if the user account did not have this privilege.

Also it appears you are using a service provider for your MongoDB instances and they have likely set up some general provisions as to what the account's approved roles are. This points to your problem with querying a secondary.

Whilst you can set up your connection to read from a secondary after you have resolved your permission issues, it is probably not really what you want. The general use case is that you are sure it is absolutely okay to not get the latest data as it is likely the secondary nodes are lagging behind.

Probably what you really need to do is look into what is causing the locking issues. Considering your use of Mongoengine a likely cause is here and you might need to look at background index building. Also this is a very good reason why reading from a secondary would be a bad idea until the next version of MongoDB is released and installed by your provider. http://docs.mongodb.org/manual/tutorial/build-indexes-on-replica-sets