Kalman Filter and sudden measurements jumps

Question

Ok here is what i need to do:

I want to do some tracking using Kalman filter(possibly adaptive).My measurements(when they are available) are very good with very small error from the real measurements. In some cases though the measurements jump to a value,completely off from the correct position i am looking for, and then after few frames the come back to their correct position.

The problem is that if my filter(not adaptive) has specific values for Measurement Noise Covariance(R) and State Error Covariance(Q) matrices the results are not very accurate,because even for these 1% of cases i have to do a compromise between R and Q.

So i decided to use an adaptive Kalman filter as they do in here: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.367.1747&rep=rep1&type=pdf

They estimate the measurement noise covariance matrix based on the innovation sequences. Basically, they are using a moving window on previous samples and the calculate the covariance of the error between the previous measurements-prior estimations. For eg 5 past measurements and the 5 prior estimations.When a faulty measurement comes under the window, the covariance increases and thus the R increases also.

But in practice the R increases(but not enough) so in the next step the estimation is still good but just a bit towards the the faulty measurement.In the next step(because now the the previous estimation has moved a bit towards the measurement) the R becomes smaller with result the new estimation to go even closer to the measurements, and so on and so forth.

In the end after a few frames the estimations follow the faulty measurements. Here is a plot to understand better what i mean.

https://www.dropbox.com/s/rkv0tjcm4s54kv3/untitled.tif

Maybe what i am trying to do is completely wrong and can't be done with the adaptive Kalman filter.Maybe someone who has worked extensively with Kalman Filter in the past and he has faced this problem before can help.

Any idea is welcome!

Answer 1

Before the answer, I want to be sure I got the problem you have right.

You have measurements, some of them are good (Low measurement noise) yet others are outliers.
The problem you're having is tuning the measurement noise covariance matrix.

Practically, you tune for the good measurements.
Outliers measurements are rejected by using the Error Covariance. If the innovation falls outside an ellipse you define using the Error Covariance Matrix the measurement is rejected.
Whenever a measurement is rejected you just apply the prediction step again and wait for another measurement.

Answer 2

Yes the problem is exactly this.

However i manage to solve it without the need to define any ellipse.What i was doing was correct except the fact that was not working if i had a lot of(lets say fifty) consecutive outliers.

This is normal if you think the size of your window.If it is for example only 10 samples and you have 20 outliers obviously it won't work.But for 5 consecutive outliers work perfectly.Generally i haven't used any threshold as you propose("if the innovation falls outside an ellipse") reject the measurements.I keep the measurements but in the same time when i start to have outliers the Error measurement covariance becomes very large.So the estimation is based more in previous estimation than in current measurement.

If i used your method which is indeed more logical(reject the current measurement,if it is an outlier based on a threshold) i have the problem that i have to define this threshold a priori,right?Maybe i am missing something..