How to correctly use egen mean function for computing average differences of the time series?

Question

I have the panel data in the format: | transaction_id | city_id | week | tr_cw | , where tr_cw is some metric computed for each city for each week.

I'm trying to compute the mean growth of the metric for each city using egen:

egen tag_cw = tag(city_id week)
sort tag_cw city_id week
by tag_cw city_id: egen tr_speed = mean(tr_cw[_n]-(tr_cw[_n-1])) if tag_cw

However, the result is different from a bit more complicated computation (which numerically looks more valid):

by tag_cw city_id: g _tr_n = tr_cw[_n] if tag_cw
by tag_cw city_id: g _tr_n1 = tr_cw[_n-1] if tag_cw
by tag_cw city_id: egen tr_speed2 = mean(_tr_n-_tr_n1)

What am I missing in using egen? Is there a good tutorial to find out about the correct usage?

Answer 1

The help for egen is quite explicit:

"Explicit subscripting (using _N and _n), which is commonly used with generate, should not be used with egen"

The reason is that egen often changes the sort order temporarily to do what it does, so subscripts will not necessarily mean what you think.

So you need to create your own growth variable first and then use egen.

Simplifying your problem to one transaction for each city and week, and assuming no gaps in time within each panel and no missings, this should suffice:

bysort city_id (week) : gen tr_diff = tr_cw - tr_cw[_n-1] 
by city_id: egen tr_speed = mean(tr_diff) 
egen tag = tag(city_id) 
l city tr_speed if tag 

You will need to complicate that for multiple transactions. You don't make clear how they are to be combined.

Answer 2

Nick's answer should give you what you need,

To answer your follow up question of

why week is in parentheses in "bysort city_id (week)"?

Is explained in the manual for bysort

to quote the manual...

It verifies that the data are sorted by varlist1 varlist2 and then performs a by as if only varlist1 were specified