How to convert pandas timezone aware timestamps to UNIX epoche?

Question

I need to convert a timezone aware date_range (TimeStamps) to UNIX epoch values for use in an external Javascript library.

My approach is:

# Create localized test data for one day
rng = pd.date_range('1.1.2014', freq='H', periods=24, tz="Europe/Berlin")
val = np.random.randn(24)
df = pd.DataFrame(data=val, index=rng, columns=['values'])

# Reset index as df column
df = df.reset_index()

# Convert the index column to the desired UNIX epoch format
df['index'] = df['index'].apply(lambda x: x.value // 10**6 )

df['index'] contains the UNIX epoch values as expected but they are are stored in UTC(!).

I suppose this is because pandas stores timestamps in numpy UTC datetime64 values under the hood.

Is there a smart way to get "right" epoch values in the requested time zone?

This proposal doesn't work with DST

Answer 1

In [17]: df
Out[17]: 
                             values
2014-01-01 00:00:00+01:00  1.027799
2014-01-01 01:00:00+01:00  1.579586
2014-01-01 02:00:00+01:00  0.202947
2014-01-01 03:00:00+01:00 -0.214921
2014-01-01 04:00:00+01:00  0.021499
2014-01-01 05:00:00+01:00 -1.368302
2014-01-01 06:00:00+01:00 -0.261738

2014-01-01 22:00:00+01:00  0.808506
2014-01-01 23:00:00+01:00  0.459895

[24 rows x 1 columns]

Use the index method asi8 to convert to int64 (which is already in ns since epoch) These are the UTC times!

In [18]: df.index.asi8//10**6
Out[18]: 
array([1388530800000, 1388534400000, 1388538000000, 1388541600000,
       1388545200000, 1388548800000, 1388552400000, 1388556000000,
       1388559600000, 1388563200000, 1388566800000, 1388570400000,
       1388574000000, 1388577600000, 1388581200000, 1388584800000,
       1388588400000, 1388592000000, 1388595600000, 1388599200000,
       1388602800000, 1388606400000, 1388610000000, 1388613600000])

These are the local timezone since epoch. Note that this is NOT a public method for normally, I would always exchange UTC data (and the timezone if you need).

In [7]: df.index._local_timestamps()//10**6
Out[7]: 
array([1388534400000, 1388538000000, 1388541600000, 1388545200000,
       1388548800000, 1388552400000, 1388556000000, 1388559600000,
       1388563200000, 1388566800000, 1388570400000, 1388574000000,
       1388577600000, 1388581200000, 1388584800000, 1388588400000,
       1388592000000, 1388595600000, 1388599200000, 1388602800000,
       1388606400000, 1388610000000, 1388613600000, 1388617200000])