Last cell in recurrent network always the most accurate

Question

I am using a recurrent network for time series forecasting.

The prediction from the last cell in the network always seems to be the most accurate. For example if I have 20 cells (so my input samples are of length 20), the prediction from the latter cells (e.g. cells 11-20) seem better than the predictions from the early cells (e.g. cells 1-10).

Is this normal?

Lets assume my network only has 3 cells. At present I would input data x(t=0, t+1, t+2) to obtain predictions for t=0, t+1, t+2.Is this the wrong approach? If I want good predictions at all timesteps, do I need to using a "moving window" i.e. should I use a rolling sequence of inputs:

For prediction at t=0, input x(t-2, t-1, t=0)
For prediction at t=1, input x(t-1, t=0, t=1)
For prediction at t=2, input x(t=0, t+1, t+2)