R-Python: Getting Monthly, weekly index points

Question

In R's xts package there is a function called endpoints which given a xts object will return a index for when a given month, week, or any user specified frequency back. How can one do this in pandas with python?

R:

endpoints(xts.object, "frequency")

Python:

from matplotlib.pylab import *
from pandas.io.data import DataReader
from datetime import datetime
symbols = ["SPY","IEF"]
data_holder = DataReader(symbols,  "yahoo",datetime(2001,1,1))
adj_close = data_holder["Adj Close"] #adjusted close data
adj_close = adj_close.dropna() #drop NAs
adj_close.head() #inspect elements

I understand that resampling function in python with "M" as parameter will get me the monthly data. But is there a way to get an array of index such that each of these indexes reference a row in the dataframe which is a month end date?

So a concrete example and I am using pseudo-code:

month_ends = adj_close.someFunction("months") #gives me the index of each month ends
month_ends.head()

[22,41,62..etc]

adj_close[month_ends,] #should give me the same thing as resampled("M")

Answer 1

Create a series with [0, 1, ...] as the values and then call resample:

s = pd.Series(np.arange(adj_close.shape[0]), index=adj_close.index)
locs = s.resample("M", how="max")
print locs

the output is:

Date
2002-07-31      0
2002-08-31     22
2002-09-30     42
2002-10-31     65
2002-11-30     85
2002-12-31    106
2003-01-31    127
2003-02-28    146
2003-03-31    167
2003-04-30    188
2003-05-31    209
2003-06-30    230
2003-07-31    252
2003-08-31    273
2003-09-30    294
...
2012-09-30    2561
2012-10-31    2582
2012-11-30    2603
2012-12-31    2623
2013-01-31    2644
2013-02-28    2663
2013-03-31    2683
2013-04-30    2705
2013-05-31    2727
2013-06-30    2747
2013-07-31    2769
2013-08-31    2791
2013-09-30    2811
2013-10-31    2834
2013-11-30    2844
Freq: M, Length: 137, dtype: int64

to get the rows:

print adj_close.iloc[locs, :].head(10)

output:

             IEF    SPY
Date                    
2002-07-31  55.49  73.01
2002-08-30  56.89  73.51
2002-09-30  59.08  65.80
2002-10-31  58.34  71.22
2002-11-29  56.93  75.61
2002-12-31  58.95  71.33
2003-01-31  58.50  69.58
2003-02-28  59.79  68.64
2003-03-31  59.56  68.79
2003-04-30  59.64  74.61

Answer 2

If I understand you correctly, you are looking for panda's DateOffset:

http://pandas.pydata.org/pandas-docs/dev/timeseries.html#dateoffset-objects

There are some great examples there, but to give you an idea of how you can use it:

import datetime
from pandas.tseries.offsets import *

a=datetime.datetime(2013,11,5)
print a + BMonthEnd() #Last Business day of the month, 
OUT:datetime.datetime(2013, 11, 29, 0, 0)

print a + MonthEnd()
OUT: datetime.datetime(2013, 11, 30, 0, 0)

print a.weekday() # is 1, i.e. Tuesday
print a + Week(weekday=4) # Looking for Friday
OUT: 2013-11-08 00:00:00

The above should give you the correct datetime object which you can then use to query your data.

EDIT: there might be easier ways to do this, but after a few beers I got the index in this way with my 'df' DataFrame:

a=list(df.index.values) # This copies the index into a list and allows you to do:
print a.index(np.datetime64(dt.datetime(2013,11,5) + Week(weekday=4))
OUT: The row number of the end of the week

df.index.values returns a ndarray which doesn't have an index() method so you need to convert it to a list which does have this method.

Note, I generated my index with pd.data_range which uses numpy.datetime64 objects.

So after you found this last day of the end of the week with dt.datetime(yyyy,mm,dd)+Week(weekday=4) you can convert it to a numpy.datetime64 object and then search for its index in your list.