Django slow working. filtering by multiple params

Question

I have 3 models:

class Author(models.Model):
    title = CharField()

class Genre(models.Model):
    title = CharField()

class Book(models.Model):
    title = CharField()
    author = ManyToManyField(Author)
    genre = ManyToManyField(Genre)

And I have a checkbox form (multiple choice) with all Genres and Authors, where checkbox value is item (genre or author) id.

Purpose: Show books with selected authors or genres (without duplicates)

I can do this in two ways:

First way:

if request.POST:
    book_list = Book.objects.all() #get all books from db
    books = []  

    request_list = request.POST.getlist('genre') #select list of genres in request
        for item in request_list:
            add_book = report_list.filter(genre=r_request) #queryset of book filtered by each genre 
            books.append(add_book) 
            book_list = book_list.exclude(genre=item)

    request_list = request.POST.getlist('author') #select list of authors in request
        for item in request_list:
            add_book = report_list.filter(author=item) #queryset of book filtered by each author 
            books.append(add_book) 
            book_list = book_list.exclude(author=item)
    return ...
        'books': books

But this way is very slow when I select a lot of authors and genres, becouse excluding is very slow.

Second way:

Remove book_list = book_list.exclude(...) and apply template tag {% ifchanged book.id %}

But I think, this will be very slow too when I'll get 1000+ books in request result (books)

How can I show books with selected authors and genres fast?

Answer 1

You're looping through a list, and for each one of the items in the list you're asking for an SQL query.

Each SQL query runs over the entire table database to get the results (unless it's organized properly).

So, you can use the __in lookup to filter from a list:

"books that have authors from list and genres from list"

genre_list= request.POST.getlist('genre')
author_list = request.POST.getlist('author')
books = Book.objects.filter(genre__in=genre_list,author__in=author_list)

"books that have the genre or the author"

from django.db.models import Q
genre_list= request.POST.getlist('genre')
author_list = request.POST.getlist('author')
books = Book.objects.filter(Q(genre__in=genre_list) | Q(author__in=author_list))

You can also read Two or more __in filters in django queryset ... They use Q object to concatenate the query into one filter.

Answer 2

On my opinion:

genre_list= request.POST.getlist('genre')
author_list = request.POST.getlist('author')

query = None

if gender_list:
    gender_q = Q(genre__in=genre_list)
    query = gender_q

if author_list:
    author_q = Q(author__in=author_list)
    if gender_list:
       query|=author_q
    else:
       query = author_q

books = []
if query:
   books = Book.objects.filter(query).distinct()

http://docs.djangoproject.com/en/1.3/topics/db/queries/#complex-lookups-with-q-objects