Inconsistent Behavior with datetime.astimezone in Python

Question

My use case is simply to store dates in the database as UTC and achieve expected results for the user scheduling in US/Central time when both scheduling and viewing a scheduled time. The inconsistent behavior below caused me some headaches and scheduled dates were off by an hour while performing certain updates.

What is causing the below inconsistent behavior? Can I count on the behavior I'm observing in my attempt to achieve sanity in the final two lines? Is there a better way, ie. am I not using datetime correctly? Frankly, I'm a bit confused so any help is appreciated!

# Instantiate a datetime in December and April and make them timezone aware
decutc = datetime.datetime(2013, 12, 12, 12, 12, 12).replace(tzinfo=pytz.UTC)
aprutc = datetime.datetime(2013, 4, 12, 12, 12, 12,).replace(tzinfo=pytz.UTC)

# Convert both to US/Central, April is STD and December is DST as expected
# NOTE is STD
decutc.astimezone(pytz.timezone('US/Central'))
Out[164]: datetime.datetime(2013, 12, 12, 6, 12, 12, tzinfo=<DstTzInfo 'US/Central' CST-1 day, 18:00:00 STD>)
# NOTE is DST
aprutc.astimezone(pytz.timezone('US/Central'))
Out[165]: datetime.datetime(2013, 4, 12, 7, 12, 12, tzinfo=<DstTzInfo 'US/Central' CDT-1 day, 19:00:00 DST>)
# Move an aware datetime to another month with a different daylight savings time
# NOTE This one DOES NOT change from STD to DST
decutc.astimezone(pytz.timezone('US/Central')).replace(month=4).astimezone(
    pytz.timezone('US/Central'))
Out[166]: datetime.datetime(2013, 4, 12, 6, 12, 12, tzinfo=<DstTzInfo 'US/Central' CST-1 day, 18:00:00 STD>)
# NOTE This one DOES change from DST to STD
aprutc.astimezone(pytz.timezone('US/Central')).replace(month=12).astimezone(
    pytz.timezone('US/Central'))
Out[167]: datetime.datetime(2013, 12, 12, 6, 12, 12, tzinfo=<DstTzInfo 'US/Central' CST-1 day, 18:00:00 STD>)

In order to achieve consistent behavior I ended up doing the following:

# NOTE correctly goes from STD to DST
decutc.astimezone(pytz.timezone('US/Central')).replace(month=4).astimezone(
    pytz.timezone('US/Central')).astimezone(pytz.UTC).astimezone(pytz.timezone('US/Central'))
Out[172]: datetime.datetime(2013, 4, 12, 7, 12, 12, tzinfo=<DstTzInfo 'US/Central' CDT-1 day, 19:00:00 DST>)

# NOTE correctly goes from DST to STD
aprutc.astimezone(pytz.timezone('US/Central')).replace(month=12).astimezone(
    pytz.timezone('US/Central')).astimezone(pytz.UTC).astimezone(pytz.timezone('US/Central'))
Out[170]: datetime.datetime(2013, 12, 12, 6, 12, 12, tzinfo=<DstTzInfo 'US/Central' CST-1 day, 18:00:00 STD>)

Answer 1

Usually I can count on Python and Python libraries being consistent out of the box :), but instead pytz has opted to provide a "normalize" function, that we as developers need to be aware of when doing time zone conversions. The problem in that the problem is inconsistent and it forces us (the developers) to make the decision on how to handle all the craziness ourselves, cause the library author cannot decide for us.

http://pytz.sourceforge.net/#localized-times-and-date-arithmetic seems like a MUST READ.

Note that this library differs from the documented Python API for tzinfo implementations; if you want to create local wallclock times you need to use the localize() method documented in this document. In addition, if you perform date arithmetic on local times that cross DST boundaries, the result may be in an incorrect timezone (ie. subtract 1 minute from 2002-10-27 1:00 EST and you get 2002-10-27 0:59 EST instead of the correct 2002-10-27 1:59 EDT). A normalize() method is provided to correct this. Unfortunately these issues cannot be resolved without modifying the Python datetime implementation.

This is very unpythonic in that there is more then one way to do things that work. ie. For UTC normalize and localize are not necessary, and we can observe things working until we actually cross DST - STD boundaries when working with other time zones, but in this case I'm not sure the alternative would have been any better (breaking code that uses the standard datetime API). I certainly would have preferred a stacktrace instead of a silently bad astimezone conversion.

I really put the fault on the datetime documentation:

http://docs.python.org/2/library/datetime.html#datetime.datetime.astimezone

Which mentions pytz and that this is where to get some tzinfo objects, but doesn't mention that you better read their documentation for caveats.

Answer 2

I was able to reproduce your results and get the inconsistent behavior. To make it consistent all I had to do was remove the.astimezone(pytz.timezone('US/Central')) calls you tacked on to the conversion calls. After doing that, the results are consistent in the sense that is does not change from STD TO DST or vice-versa.

In other words, change the:

decutc.astimezone(pytz.timezone('US/Central')).replace(month=4).astimezone(
    pytz.timezone('US/Central'))
aprutc.astimezone(pytz.timezone('US/Central')).replace(month=12).astimezone(
    pytz.timezone('US/Central'))

calls to just:

decutc.astimezone(pytz.timezone('US/Central')).replace(month=4)
aprutc.astimezone(pytz.timezone('US/Central')).replace(month=12)

Update
To get proper and consistent STD <--> DST conversion when making date changes, just put the .astimezone(pytz.timezone('US/Central') call after the .replace() calls, like this:

decutc.replace(month=4).astimezone(pytz.timezone('US/Central'))
aprutc.replace(month=12).astimezone(pytz.timezone('US/Central'))

This makes sense if you think about it because what's happening is that first a change to the month of an UTC datetime (which doesn't observe daylight saving time) is made, then that intermediate result is converted to a timezone which does.

So in conclusion I would suggest that you store all your datetimes in UTC, do all your manipulations in that timezone, and only convert to a specific local time when absolutely necessary.