Losing DST information using pytz and UTC conversions

Question

Maybe this is a 4am bug, but I think I'm doing everything right, but it doesn't appear as though DST is translating from the UTC timestamp to the localized datetime.

>>> from datetime import datetime
>>> import pytz
>>> eastern = pytz.timezone("US/Eastern")
>>> utc = pytz.utc
>>> local_now = eastern.localize(datetime.now())
>>> utc_now = local_now.astimezone(utc)
>>> seconds = int(utc_now.strftime("%s"))
>>> utc_then = utc.localize(datetime.fromtimestamp(seconds))
>>> local_then = utc_then.astimezone(eastern)
>>> print utc_now, utc_then
2013-06-16 10:05:27.893005+00:00 2013-06-16 11:05:27+00:00
>>> print local_now, local_then
2013-06-16 06:05:27.893005-04:00 2013-06-16 07:05:27-04:00

Answer 1

             o------------o
             |            |  DT.datetime.utcfromtimestamp (*)
             |            |<-----------------------------------o
             |            |                                    |
             |  datetime  |                                    |
             |            |  DT.datetime.fromtimestamp         |
             |            |<----------------------------o      |
             |            |                             |      |
             o------------o                             |      |
                |   ^                                   |      |
     .timetuple |   |                                   |      |
  .utctimetuple |   | DT.datetime(*tup[:6])             |      |
                v   |                                   |      |
             o------------o                          o------------o
             |            |-- calendar.timegm (*) -->|            |
             |            |                          |            |
             |            |---------- time.mktime -->|            |
             |  timetuple |                          |  timestamp |
             |            |<-- time.localtime -------|            |
             |            |                          |            |
             |            |<-- time.gmtime (*)-------|            |
             o------------o                          o------------o

(*) Interprets its input as being in UTC and returns output in UTC

As the diagram shows, when you have a datetime in UTC such as utc_now, to get its timestamp, use

seconds = calendar.timegm(utc_date.utctimetuple())

When you have a timestamp, to get to the datetime in UTC, use

DT.datetime.utcfromtimestamp(seconds)

---

import datetime as DT
import pytz
import calendar
eastern = pytz.timezone("US/Eastern")
utc = pytz.utc
now = DT.datetime(2013, 6, 16, 10, 0, 0)
local_now = eastern.localize(now)
utc_now = local_now.astimezone(utc)
seconds = calendar.timegm(utc_now.utctimetuple())

print(seconds)
# 1371391200

utc_then = utc.localize(DT.datetime.utcfromtimestamp(seconds))
local_then = utc_then.astimezone(eastern)

print utc_now, utc_then
# 2013-06-16 14:00:00+00:00 2013-06-16 14:00:00+00:00
print local_now, local_then
# 2013-06-16 10:00:00-04:00 2013-06-16 10:00:00-04:00

PS. Note that the timetuple() and utctimetuple() methods drop microseconds off the datetime. To convert a datetime to a timestamp in a way that preserves microseconds, use mata's solution.

Answer 2

You should avoid datetime.now if you want to write portable code, as it always uses the local timezone, so local_now = eastern.localize(datetime.now()) will only work if the timezone on the local machine is eastern. Always try to use utcnow, and for the same reason utcfromtimestamp.

Also, using strftime("%s") to convert a datetime to a timestamp doesn't work.

from datetime import datetime
import pytz

utc_now = pytz.utc.localize(datetime.utcnow())
eastern = pytz.timezone("US/Eastern")
local_now = utc_now.astimezone(eastern)

# seconds = utc_now.timestamp()  python3
seconds = (utc_now - pytz.utc.localize(datetime.utcfromtimestamp(0))).total_seconds()
utc_then = pytz.utc.localize(datetime.utcfromtimestamp(seconds))

local_then = utc_then.astimezone(eastern)

print("%s - %s" % (utc_now, utc_then))
print("%s - %s" % (local_now, local_then))

Answer 3

To get your local timezone as pytz.timezone object, you could use tzlocal module:

#!/usr/bin/env python
from datetime import datetime
import pytz # pip install pytz
from tzlocal import get_localzone # pip install tzlocal

local_tz = get_localzone()
local_now = datetime.now(local_tz)
utc_now = local_now.astimezone(pytz.utc)
seconds = (utc_now - datetime(1970, 1, 1, tzinfo=pytz.utc)).total_seconds()

utc_then = datetime.fromtimestamp(seconds, pytz.utc)
local_then = utc_then.astimezone(local_tz)
print("%s %s" % (utc_now, utc_then))
print("%s %s" % (local_now, local_then))

• don't use datetime.now() -- it may be ambiguous e.g., during DST change. Either pass tzinfo explicitly as in my example or use datetime.utcnow()
• don't use utc_now.strftime('%s') -- it ignores timezone info (it uses the current local timezone) and it is not portable. Use datetime.timestamp() method or its analogs instead
• don't use utc.localize(datetime.fromtimestamp(seconds)) -- .fromtimestamp() returns a naive datetime object in local timezone that may be different from UTC. Either pass tzinfo explicitly as in my example or use datetime.utcfromtimestamp() to get a naive datetime object that represents UTC time
• don't use datetime.utctimetuple() with naive datetime objects -- it doesn't convert them to UTC. If the object is already in UTC: utc_now.timetuple() returns the same time.

To raise an exception for ambiguous local time, use localize(is_dst=None):

aware_dt = tz.localize(naive_dt, is_dst=None)

Without is_dst=None, there is 50% chance that tz.localize() returns wrong result during DST change. If it is preferable to get a possibly wrong result instead of an exception in your particular case then you could pass the explicit is_dst=False to remind yourself about it.

In general, pytz documentation recommends tz.normalize() after .astimezone() call if neither source nor destination timezones are UTC.