How to handle DST and TZ in recurring events?

Question

Does dateutil rrule support DST and TZ? Need something similar to iCalendar RRULE.

If not - how to tackle this problem (scheduling recurring events & DST offset change)

Imports

>>> from django.utils import timezone
>>> import pytz
>>> from datetime import timedelta
>>> from dateutil import rrule
>>> now = timezone.now()
>>> pl = pytz.timezone("Europe/Warsaw")

Issue with timedelta (need to have the same local hours, but different DST offsets):

>>> pl.normalize(now)
datetime.datetime(2012, 9, 20, 1, 16, 58, 226000, tzinfo=<DstTzInfo 'Europe/Warsaw' CEST+2:00:00 DST>)    
>>> pl.normalize(now+timedelta(days=180))
datetime.datetime(2013, 3, 19, 0, 16, 58, 226000, tzinfo=<DstTzInfo 'Europe/Warsaw' CET+1:00:00 STD>)

Issue with rrule (need to have the same every local hour of each occurrence):

>>> r = rrule.rrule(3,dtstart=now,interval=180,count=2)
>>> pl.normalize(r[0])
datetime.datetime(2012, 9, 20, 1, 16, 58, tzinfo=<DstTzInfo 'Europe/Warsaw' CEST+2:00:00 DST>)
>>> pl.normalize(r[1])
datetime.datetime(2013, 3, 19, 0, 16, 58, tzinfo=<DstTzInfo 'Europe/Warsaw' CET+1:00:00 STD>)

Answer 1

@asdf: I can't add code to comments so I need to post this as an answer:

I am afraid that with your solution I will always loose DST info, therefore half of the year recurrences would be 1 hour off time.

Basing on your answer I found out that this might be the correct solution:

>>> from datetime import datetime
>>> import pytz
>>> from dateutil import rrule
>>> # this is raw data I get from the DB, according to django docs I store it in UTC
>>> raw = datetime.utcnow().replace(tzinfo=pytz.UTC)
>>> # in addition I need to store the timezone so I can do dst the calculations
>>> tz = pytz.timezone("Europe/Warsaw")
>>> # this means that the actual local time would be
>>> local = raw.astimezone(tz)
>>> # but rrule doesn't take into account DST and local time, so I must convert aware datetime to naive
>>> naive = local.replace(tzinfo=None)
>>> # standard rrule
>>> r = rrule.rrule(rrule.DAILY,interval=180,count=10,dtstart=naive)
>>> for dt in r:
>>>     # now we must get back to aware datetime - since we are using naive (local) datetime, 
        # we must convert it back to local timezone
...     print tz.localize(dt)

This is why I think your solution might fail:

>>> from datetime import datetime
>>> from dateutil import rrule
>>> import pytz
>>> now = datetime.utcnow()
>>> pl = pytz.timezone("Europe/Warsaw")
>>> r = rrule.rrule(rrule.DAILY, dtstart=now, interval=180, count=2)
>>> now
datetime.datetime(2012, 9, 21, 9, 21, 57, 900000)
>>> for dt in r:
...     local_dt = dt.replace(tzinfo=pytz.UTC).astimezone(pl)
...     print local_dt - local_dt.dst()
...     
2012-09-21 10:21:57+02:00
2013-03-20 10:21:57+01:00
>>> # so what is the actual local time we store in the DB ?
>>> now.replace(tzinfo=pytz.UTC).astimezone(pl)
datetime.datetime(2012, 9, 21, 11, 21, 57, 900000, tzinfo=<DstTzInfo 'Europe/Warsaw' CEST+2:00:00 DST>)

As you can see, there is 1 hour difference between the rrule result, and the real data we store in the DB.

Answer 2

Note that what django.utils.timezone.now() returns can be either a naive or aware datetime, depending on your USE_TZ setting. What you should be using internally for calculations (eg. the now you provide to rrule.rrule) is a UTC-based datetime. It can be an offset-aware one (ie. datetime.now(pytz.UTC)), or a naive one (ie. datetime.utcnow()). The latter seems to be preferred for storing (see this blogpost).

Now, rrule.rrule handles timezones, that's why you observe the CEST-to-CET change in what your rrule yields. However, if what you want is to always get the same hour (eg. 0 AM every day, no matter DST or not), then you actually want to "ignore" the change. One way of doing so would be to do dt = dt - dt.dst(), if dt was an aware datetime.

Here's how you can do that:

from datetime import datetime
from dateutil import rrule
import pytz
now = datetime.utcnow()
pl = pytz.timezone("Europe/Warsaw")
r = rrule.rrule(rrule.DAILY, dtstart=now, interval=180, count=2)

# will yield naive datetimes, assumed UTC
for dt in r:
    # convert from naive-UTC to aware-local
    local_dt = dt.replace(tzinfo=pytz.UTC).astimezone(pl)
    # account for the dst difference
    print local_dt - local_dt.dst()

This prints two datetimes, each is in a different timezone (well, different DST setting), both represent the same wallclock hour. If you were to handle aware-UTC-datetimes instead of naive-assumed-UTC like in the example, you'd simply skip the .replace part. A quick cheat sheet about these conversions can be found here.

Answer 3

Yes, the point is you should NOT store localtime, ever. Store UTC and convert to localtime on demand (ie. on a per-request basis, using the request data, like the Accept-Language header, to know what tz you should use).

What you're doing is you're using a localized datetime for calculations (ie. rrule.rrule()). This is suboptimal, as you need to know the target timezone to do that, so this can be done per request only, as opposed to precalculating the rrule realizations. That's why you should use UTC internally (ie. to precalculate the datetimes) and then convert them prior to sending to the user. In this case, only the conversion would have to be done after receiving the request (that is, when the target timezone is known).