Creating custom Field Lookups in Django

Question

How do you create custom field lookups in Django?

When filtering querysets, django provides a set of lookups that you can use: __contains, __iexact, __in, and so forth. I want to be able to provide a new lookup for my manager, so for instance, someone could say:

twentysomethings = Person.objects.filter(age__within5=25)

and get back all the Person objects with an age between 20 and 30. Do I need to subclass the QuerySet or Manager class to do this? How would it be implemented?

Answer 1

As of Django 1.7, there is a simple way to implement it. Your example is actually very similar to the one from the documentation:

from django.db.models import Lookup

class AbsoluteValueLessThan(Lookup):
    lookup_name = 'lt'

    def as_sql(self, qn, connection):
        lhs, lhs_params = qn.compile(self.lhs.lhs)
        rhs, rhs_params = self.process_rhs(qn, connection)
        params = lhs_params + rhs_params + lhs_params + rhs_params
        return '%s < %s AND %s > -%s' % (lhs, rhs, lhs, rhs), params

AbsoluteValue.register_lookup(AbsoluteValueLessThan)

While registering, you can just use Field.register_lookup(AbsoluteValueLessThan) instead.

Answer 2

A more flexible way to do this is to write a custom QuerySet as well as a custom manager. Working from ozan's code:

class PersonQuerySet(models.query.QuerySet):
    def in_age_range(self, min, max):
        return self.filter(age__gte=min, age__lt=max)

class PersonManager(models.Manager):
    def get_query_set(self):
         return PersonQuerySet(self.model)

    def __getattr__(self, name):
        return getattr(self.get_query_set(), name)

class Person(models.Model):
    age = #...

    objects = PersonManager()

This allows you to chain your custom query. So both these queries would be valid:

Person.objects.in_age_range(20,30)

Person.objects.exclude(somefield = some_value).in_age_range(20, 30)

Answer 3

Rather than creating a field lookup, best practice would be to create a manager method, that might look a little bit like this:

class PersonManger(models.Manager):
    def in_age_range(self, min, max):
        return self.filter(age__gte=min, age__lt=max)

class Person(models.Model):
    age = #...

    objects = PersonManager()

then usage would be like so:

twentysomethings = Person.objects.in_age_range(20, 30)

Answer 4

First, let me say that there is no Django machinery in place that's meant to publicly facilitate what you'd like.

(Edit - actually since Django 1.7 there is: https://docs.djangoproject.com/en/1.7/howto/custom-lookups/ )

That said, if you really want to accomplish this, subclass QuerySet and override the _filter_or_exclude() method. Then create a custom manager that only returns your custom QuerySet (or monkey-patch Django's QuerySet, yuck). We do this in neo4django to reuse as much of the Django ORM queryset code as possible while building Neo4j-specific Query objects.

Try something (roughly) like this, adapted from Zach's answer. I've left actual error handling for the field lookup parsing as an exercise for the reader :)

class PersonQuerySet(models.query.QuerySet):
    def _filter_or_exclude(self, negate, *args, **kwargs):
        cust_lookups = filter(lambda s: s[0].endswith('__within5'), kwargs.items())
        for lookup in cust_lookups:
            kwargs.pop(lookup[0])
            lookup_prefix = lookup[0].rsplit('__',1)[0]
            kwargs.update({lookup_prefix + '__gte':lookup[1]-5,
                           lookup_prefix + '__lt':lookup[1]+5})
        return super(PersonQuerySet, self)._filter_or_exclude(negate, *args, **kwargs)

class PersonManager(models.Manager):
    def get_query_set(self):
         return PersonQuerySet(self.model)

class Person(models.Model):
    age = #...

    objects = PersonManager()

Final remarks - clearly, if you want to chain custom field lookups, this is going to get pretty hairy. Also, I'd normally write this a bit more functionally and use itertools for performance, but thought it was more clear to leave it out. Have fun!