How to compare dates with psycopg2 (PostgreSQL)?

Question

By using the PostgreSQL Python module psycopg2, I want to find rows that have an end time that falls before some timestamp. I tried to do this with a cursor.execute(…) on a SQL query with … where end_time < %s, where %s replaced by a datetime.datetime object with a timezone. The SQL generated by psycopg2 is:

select * from my_table where end_time < '2014-05-01T13:00:00+00:00'::timestamptz;

which returns a row like:

id  |     start_time      |      end_time       
331 | 2014-05-01 14:55:00 | 2014-05-01 15:05:00

The end time is not earlier than 2014-05-01 at 13:00 UTC, contrary to what I was expecting.

What is the proper way of performing the desired time selection in psycopg2?

Answer 1

Alright, using PostgreSQL timestamps with time zones in the table solved the problem.

Here are details about where the problem was:

• I put timezone-aware timestamps in columns that are not timezone-aware. I assumed that the timestamp without time zone column would contain the UTC time (like for timestamp with time zone), but it actually contains the timestamp in local time, as described in the documentation:

  Conversions between timestamp without time zone and timestamp with time zone normally assume that the timestamp without time zone value should be taken or given as timezone local time. A different time zone can be specified for the conversion using AT TIME ZONE.

  Thus, 00:00 UTC is stored as 08:00 in a column timestamp without time zone when in China.

• When doing a comparison like "timestamp without time zone" < "timestamp with time zone", I can't find in the documentation what PostgreSQL does. However, what was observed in the question is consistent with the idea that the timezone is first removed in the same way, and the timestamps without timezone are compared. I should not have been surprised by the result: the late time "15:00" is indeed earlier than 13:00 UTC, in China (UTC+8). The key was to know that displayed (and stored) times are in the local time zone.

The moral I get from this is: conversions between timestamps with and without timezones are better handled explicitly.

Answer 2

Specify the time zone at which the times should be displayed. If start_time and end_time are timestamp without time zone as your output suggests then you also need to tell postgresql at what time zone they should be in

select 
    id,
    start_time at time zone 'UTC' at time zone 'UTC',
    end_time at time zone 'UTC' at time zone 'UTC'
from my_table 
where end_time at time zone 'UTC' < %s