Django making a query with custom collation

Question

Is it possible to make query with a collation different from database table have?

Answer 1

Here is how you can use a specific collation instead of the default collation for a given table/column. I'm assuming you always want that to be the case insensitive utf8_general_ci, but you can easily change that in the code or add it as a variable.

Note the use of the params kwarg instead of the db literal function. Params exists for the exact same purpose.

def iexact(**kw):
    fields = [['%s=%%s collate utf8_general_ci'%field,value] for (field,value) in kw.items()]
    return dict(where=[f[0] for f in fields], params=[f[1] for f in fields])

if User.objects.extra(**iexact(username='joeblow')).exists():
    status = "Found a user with this username!"

Answer 2

Using extra() is a little messy. Something similar can now be achieved with Func() expression (since Django 1.8):

username_ci = Func(
    'username',
    function='utf8_general_ci',
    template='(%(expressions)s) COLLATE "%(function)s"')

This can be used in annotate():

User.objects.annotate(uname_ci=username_ci).filter(uname_ci='joeblow').exists()

Or in order_by() to override default collation rules when sorting:

User.objects.order_by(username_ci)

Now, it still may seem messy, but if you look at the docs and code of Func(), you will discover that it is very easy to subclass it and make a reusable collation setter.

I used this trick with Postgres database.

Answer 3

I solve this using bit of a hack;

Django's extra method is just like raw method, they both using the query statetment directly;

MyModel.objects.extra(where=["name LIKE '%%" + name + "%%' COLLATE utf8_general_ci"])

But like this sql injection is possible. We need to escape name variable. I searched a lot for a function which just escapes a string for db. Found one in MySQL-python package but it can't escape unicode strings. Also package has literal method in connection but to use it we need an instance (maybe it is for db characteristic).

At last I used Django's db.connection.cursor.

from django.db import connection
cursor = connection.cursor()
name = cursor.db.connection.literal(name)[1:-1]  # [1:-1] excluding quotes

With this way we also need an instance but I suppose this not require a db connection. And I suppose this method db independent. If I am wrong please correct me.

Answer 4

This above solution works. In case of getting the reverse order the following snippet

sort_value = sort.strip()
if sort_value in ['name', '-name']:
    sort = Func('name', function='C', template='(%(expressions)s) COLLATE "%(function)s"')
if sort_value in ['-name']:
    f_res = queryset.order_by(sort).reverse()
else:
    f_res = queryset.order_by(sort)
return f_res