Django lookup types ("iexact", "icontains", "month", etc.) not working in Django nonrel (using dbindexer)

Question

I created an app using 'django nonrel' and am using 'django dbindexer' to allow for normal Django lookups. The settings file is as below

myproject/settings.py

from djangoappengine.settings_base import *

import os

DATABASES['native'] = DATABASES['default']
DATABASES['default'] = {'ENGINE': 'dbindexer', 'TARGET': 'native'}
AUTOLOAD_SITECONF = 'indexes'
SITE_ID = 1
SECRET_KEY = '=r-$b*8hglm+858&9t043hlm6-&6-3d3vfc4((7yd0dbrakhvi'
AUTH_PROFILE_MODULE = 'myapp.UserProfile'


DBINDEXER_BACKENDS = (
    'dbindexer.backends.BaseResolver',
    'dbindexer.backends.FKNullFix',
    'dbindexer.backends.ConstantFieldJOINResolver',
)

INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.sites',
    'django.contrib.contenttypes',
    'django.contrib.auth',
    'django.contrib.sessions',
    'django.contrib.staticfiles',
    'django.contrib.messages',
    'djangotoolbox',
    'autoload',
    'dbindexer',
    'myapp',

    'djangoappengine',
)

MIDDLEWARE_CLASSES = (
    'autoload.middleware.AutoloadMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
)

TEMPLATE_CONTEXT_PROCESSORS = (
    'django.contrib.auth.context_processors.auth',
    'django.core.context_processors.media',
    'django.core.context_processors.static',
    'django.core.context_processors.request',
)

TEST_RUNNER = 'djangotoolbox.test.CapturingTestSuiteRunner'

ROOT_URLCONF = 'myproject.urls'

gettext = lambda s: s
SETTINGS_PATH = os.path.abspath(os.path.dirname(__file__))
temp = os.path.join(SETTINGS_PATH, os.pardir)
ROOT = os.path.join(os.path.normpath(temp))

STATIC_ROOT = os.path.join(ROOT, 'collected_static')

STATIC_URL = '/static/'

STATICFILES_FINDERS = (
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)

AUTH_PROFILE_MODULE = 'myapp.UserProfile'

I created a 'dbindexes' file inside 'myapp' folder to add the index definitions.

myapp/dbindexes.py

from models import *
from dbindexer.lookups import StandardLookup
from dbindexer.api import register_index

register_index(myModel, {'first_name': 'icontains',
                         'last_name':  'icontains',
                           })

And the 'indexes' file in the main folder looks like this:

indexes.py

from dbindexer import autodiscover
autodiscover()

Now, when I query 'myModel' using the the registered indexes, it always returns an empty list. It seems that the indexes are being properly discovered as it does not throw any error. However, when I change 'icontains' to 'contains' in the 'myapp/dbindexes.py' file, it gives the expected result.

Could someone guide me in figuring out the issue.

Thanks!!!

Answer 1

dbindexer essentially automatically denormalizes your entity and adds additional indexable fields to help with queries. For example, GAE doesn't do case insensitive searches or substring searches, so if you need to use icontains, it generates a list of lowercase substrings and stores that in a ListField, which is indexable. This happens when you write your entity.

Note that this can end up being very expensive since it inflates your datastore, and the indexes used.

When you run a query you can only query against entities in the datastore. If your dbindexes.py specified contains at one point, then entities would have been created with search fields for contains, and queries for icontains won't find what you're looking for. If you add new entities, and they have the correct indexable data written, then they should be queryable, but the old entities won't be returned in a query.

You can use the datastore viewer to see what dbindexer was written along with your entities. If some entities don't have extra dbindexer generated attributes, then those queries won't return. You could fix this by reading and writing all your entities so that dbindexer updates the fields.