Django dictsort not working with date

Question

I have a simple pair of models for working with time series:

class Ticker(models.Model):
    name = CharField(max_length=100)

     def latest_datapoint(self):
         return self.datapoint_set.order_by('-timestamp')[0]

     def latest_value(self):
         return self.latest_data_point().value

     def latest_timestamp(self)
         return self.latest_data_point().timestamp

class DataPoint(models.Model):
     ticker = ForeignKeyField(Ticker)
     timestamp = DateTimeField()
     value = FloatField()

I then use a generic class based view to make a ticker list:

class TickerListView(ListView):
    model = Ticker
    template_name = 'ticker-list.html'

Lastly, I implement the template:

<table>
    <thead>
        <tr>
            <th>Name</th>
            <th>Latest Value</th>
            <th>Latest Timestamp</th>
        </tr>
    </thead>
    <tbody>
        {% for object in object_list %}  <----***Let's talk about this***
            <tr>
                <td>{{ object.name }}</td>
                <td>{{ object.latest_value}}</td>
                <td>{{ object.latest_timestamp}}</td>
            </tr>
        {% endfor % }
    </tbody>
</table>

As written, this works just fine. But I would like to order my results in the template using dictsort. If I replace the highlighted line with {% for object in object_list|dictsort:"latest_value" %} everything works fine. But if I instead write{% for object in object_list|dictsort:"latest_timestamp" %} everything comes in blank , as if I had specified an invalid key to sort on. Any idea what's going on here?

(Hypothesis: dictsort doesn't know how to handle datetime.datetime data types when ordering.)

Thanks.

Added in response to Hieu Nguyen:

Thanks for your comment! So of course I'm getting burned because I have oversimplified the example above. In my actual code the method definitions for Ticker look like:

     def latest_datapoint(self):
         if self.datapoint_set:
             return self.datapoint_set.order_by('-timestamp')[0]
         else:
             return None

     def latest_value(self):
         if self.latest_datapoint():
             return self.latest_data_point().value
         else:
             return None

     def latest_timestamp(self)
         if self.latest_datapoint():             
             return self.latest_data_point().timestamp
         else:
             return None

When I try to dictsort on latest_value everything works fine, even if several of the Tickers have latest_value() = None. But when I dictsort on latest_timestamp when some of those are None then Django seems to skip the table entirely. Is the lesson here that None can be cleanly compared with Float data but not with DateTime data? If so, Would a latest_timestamp_text method, which coerces all timestamps into a YYYYMMDD HHMMSS format (with None being converted to 0) suffice as a key on which to perform in-template dictsorts?

Edited again:

This still doesn't seem to sort it out. I have implemented a new method:

def latest_timestamp_safe(self):
    if self.latest_timestamp() is None:
        return datetime.now().replace(year=1975)
    else:
        return self.latest_timestamp()

I have confirmed that this is working and that all the rows in my table display a nice tz-aware datetime in the column I made for latest_timestamp_safe. However I still cannot dictsort on this key. Any thoughts?

Edit last time:

Above doesn't work. As Hieu points out this doesn't suffice to make the "safe" datetime tz aware. So I just gave up on that and went with the stringifying idea. Works fine. Compares with None in the expected way.

Answer 1

Your hypothesis is true. It seems that dictsort can't handle to compare None value with datetime object anyhow. Even if you return a very small datetime object like:

datetime.datetime(1900, 1, 1, 1, 1)

the list will be blank too.

So the solution would be converting datetime object to text and use it, also return '' instead of None so it doesn't take time to convert None to something else when sorting:

def latest_timestamp_text(self)
    if self.latest_datapoint():             
        return self.latest_data_point().timestamp.strftime('%Y-%m-%d %H:%M:%S')
    else:
        return ''

Hope it helps.

EDIT

I didn't know that Django's DateTimeField will produce datetime object with timezone info. So of course it cannot be compared to a normal datetime object without that info:

>>> dt1 = datetime.datetime(2006, 11, 21, 16, 30, tzinfo=gmt2)
>>> dt2 = datetime.datetime(2006, 11, 22, 16, 30)
>>> dt1 > dt2
TypeError: can't compare offset-naive and offset-aware datetimes

So if you still want to use the datetime object without converting to text, you have to make a timezone info class, something like this:

from datetime import datetime, timedelta, tzinfo

class GMT2(tzinfo):
    def utcoffset(self, dt):
        return timedelta(hours=2) + self.dst(dt)

    def dst(self, dt):
        d = datetime(dt.year, 4, 1)
        self.dston = d - timedelta(days=d.weekday() + 1)
        d = datetime(dt.year, 11, 1)
        self.dstoff = d - timedelta(days=d.weekday() + 1)
        if self.dston <= dt.replace(tzinfo=None) < self.dstoff:
            return timedelta(hours=1)
        else:
            return timedelta(0)

    def tzname(self, dt):
        return "GMT +2"

Then change your function like this:

def latest_timestamp(self):
    gmt2 = GMT2()
    if self.latest_datapoint():             
        return self.latest_data_point().timestamp
    else:
        return datetime(1970, 1, 1, 1, 1, tzinfo=gmt2)

Note: 1970 is the minimum value for year.

I would prefer converting to text over this method but well it's your choice.