Django optimize code for large data filtering

Question

As I am searching through 15000 results, is there any way to optimize the processing speed?

In my VIEW I'm filtering search as this:

if form.is_valid():

      results = Screening.objects.filter(
         screening_date__range(form.cleaned_data['start_date'],form.cleaned_data['end_date']))

      if form.cleaned_data['company']:
          results = results.filter(company=form.cleaned_data['company'])
      if form.cleaned_data['company_job']:
          results = results.filter(company_job__in=form.cleaned_data['company_job'])
      if form.cleaned_data['company_job_type']:
          results = results.filter(company_job_type=form.cleaned_data['company_job_type'])
      if form.cleaned_data['reason']:
          results = results.filter(reason_for_testing__in=form.cleaned_data['reason'])`

And in TEMPLATE, the passed result is used as:

{% for result in results %}

      <td>{{ result.company.name}}</td>
      <td>{{ result.first_name }} {{ result.last_name }}</td>
      <td>{{ result.company_job.job_number }}</td>
      <td>{{ result.id }}</td>
      <td>{{ result.screening_date|date}}</td></tr>

Is there any way to optimize the processing or should I use cache or sth in this case?

Answer 1

This is not an answer, but just a tip to make your code easier to read and work with:

  filters = {
      'screening_date__range': (form.cleaned_data['start_date'],form.cleaned_data['end_date'])
  }

  if form.cleaned_data['company']:
      filters['company'] = form.cleaned_data['company']
  if form.cleaned_data['company_job']:
      filters['company_job__in'] = form.cleaned_data['company_job']
  if form.cleaned_data['company_job_type']:
      filters['company_job_type'] = form.cleaned_data['company_job_type']
  if form.cleaned_data['reason']:
      filters['reason_for_testing__in'] = form.cleaned_data['reason']

  Screening.objects.filter(**filters)

Answer 2

You should definitely use select_related to make ony one DB hit (not N).

Then profile the code, find the bottleneck and try to optimize it, not "optimize in general".