Django pagination on a largeset of data

Question

I am using a template for displaying a table which has more than 2000 rows.But implementing django pagination in this makes it too slow as the data is too much.What is the best way to go about this.Or how can i make the pagination load 200 rows per page and make the page faster. How can i make the page load faster

   {% extends "base/admin_base.html" %}
   {% load pagination_tags %}




         {% autopaginate response_dict.taggeddata 100 %}
           <div align="right">{% paginate %}</div>
  <form action="/users/saveuser/" method="post">{% csrf_token %}
  <b>
        <table>

         <tr><td><font>*</font>Select Category group for tagging</td><td> 
         <select id="cgroup" onchange="getcategory('1');">
         <!--<option value="-1">Select Category group</option>   -->
         {% for group in response_dict.categorygroup %}
                  <option value="{{group.id}}">{{group.name}}</option> 
         {% endfor %}
         </select>  
         </td></tr>

        </table>
        </b>
        <table  id="box-table-a">
        <colgroup>
        <col class="vzebra-odd">
        <col class="vzebra-even">
        <col class="vzebra-odd">
        <col class="vzebra-even">
        <col class="vzebra-odd">
        <col class="vzebra-even">
        <col class="vzebra-odd">
        <col class="vzebra-even">
        </colgroup>
        <thead>
         <tr><th id="vzebra-comedy" scope="col">Field1</th><th id="vzebra-adventure" scope="col">Field2</th><th id="vzebra-comedy" scope="col">Field3</th><th id="vzebra-adventure" scope="col">Field4</th><th id="vzebra-comedy" scope="col">Field5</th><th id="vzebra-adventure" scope="col">Field6</th><th id="vzebra-comedy" scope="col">Tag</th><th id="vzebra-adventure" scope="col">Actions</th><thead></tr>
        <tbody>  
         {% for td in response_dict.taggeddata %}
           <tr id="{{td.id}}">
           <td width="20%">{{td.field1}}</td>
           <td>{{td.field2}}</td>
           <td>{{td.field3}}</td>
           <td>{{td.field4}}</td>
           <td>{{td.field5}}</td>
           <td>{{td.field6}}</td>
           <td class="tg">Select category</td>
           <td ><img src="/media/img/cancel.png" alt="delete" height="20" width="20" onclick="delete_row('{{td.id}}')"></td>
           </tr>
         {% endfor %}
        </tbody>
        </table>
         <input type="button" value="Add" id="addbtn" onclick="validate();"/>

  </form>

Answer 1

I use pagination in Django with up to 4 million data sets. works without any perf problems...

Answer 2

Use class-based views, paginate_by : see django.views.generic.list.BaseListView . Besides, if performance is important use ajax and pass data to a front end with json. Also I would recommend you to use Model.objects.values_list()

Something like this :

class MovieLandingJSON(JSONResponseMixin, BaseListView):
    """
    Movie landing page movie covers in json format.
    """
    paginate_by = 5
    context_object_name = 'movies'
    queryset = Movie.objects.all()
    values_list = ['id', 'title', 'slug']

    def get_queryset(self):
    """
    Filters  queryset on request
    """
        queryset = super(MovieLandingJSON, self).get_queryset()

        if self.values_list:
            queryset = queryset.values_list(*self.values_list)
        return queryset


from django.core.serializers import serialize
from django.utils.simplejson import dumps, loads, JSONEncoder
from django.db.models.query import QuerySet
from django.utils.functional import curry


class DjangoJSONEncoder(JSONEncoder):
    def default(self, obj):
        if isinstance(obj, QuerySet):
            # `default` must return a python serializable
            # structure, the easiest way is to load the JSON
            # string produced by `serialize` and return it
            return loads(serialize('json', obj))
        return JSONEncoder.default(self,obj)

dumps = curry(dumps, cls=DjangoJSONEncoder)


class JSONResponseMixin(object):
    """
    A mixin that can be used to render a JSON response.
    """
    response_class = HttpResponse

    def render_to_response(self, context, **response_kwargs):
        """
        Returns a JSON response, transforming QuerySet to json.
        """
        response_kwargs['content_type'] = 'application/json'
        return self.response_class(
            self.convert_queryset_to_json(context['object_list']),
            **response_kwargs
        )

    def convert_queryset_to_json(self, queryset):
        if self.values_list:
            return simplejson.dumps(list(queryset))
        else:
            return dumps(queryset)