how to prevent series of integers to have the same value to often [closed]

Question

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.

Closed 8 years ago.

I have "online" series of integers number, so every several milliseconds I have new number. Frequency is not specified - sometimes I have many numbers, sometimes I have no numbers.

In reality this numbers are so called "true price" of the stock. I use it in my trading application.

Now I just use last number as a "true price" so I do not track series at all. But there are some problems with such approach. Let's look at this series:

1. 98; last = 98
2. 100; last = 100
3. 101; last = 101
4. 100; last = 100
5. 101; last = 101
6. 100; last = 100
7. 101; last = 101
8. 100; last = 100
9. 101; last = 101
10. 100; last = 100
11. 99; last = 99
12. 98; last = 98

The problem is that in very short period of time my "true price" is changed from 100 to 101 and back too many times. Every change of true price implies a lot of work (recalculations, orders placement etc) so i do not need to change true price like that. This change (100-101-100-101) is due "measurement problems" and i need to filter it. I can not fix "measurement" so I have to fix it here.

One of the approach would be to use "3LastAverage" algorithm:

1. 98; 3lastAverage = 98
2. 100; 3lastAverage = 99
3. 101; 3lastAverage = 99.67 => 100
4. 100; 3lastAverage = 100.33 => 100
5. 101; 3lastAverage = 101
6. 100; 3lastAverage = 100
7. 101; 3lastAverage = 101
8. 100; 3lastAverage = 100
9. 101; 3lastAverage = 101
10. 100; 3lastAverage = 100
11. 99; 3lastAverage = 100
12. 98; 3lastAverage = 99

As you can see such approach doesn't always work, i still have 100-101-100 problem. Probably i can use more numbers to calculate "average"... but such approach will not work for me because of such example:

1. 99; 3lastAverage = 99 (i want 99)
2. 100; 3lastAverage = 100 (i want 100)
3. 101; 3lastAverage = 100 (i want 101)
4. 102; 3lastAverage = 101 (i want 102)
5. 103; 3lastAverage = 103 (i want 103)

In general, when things go "OK" i need truePrice to be just the last number!

So, having "online" series of integers numbers, i need to calculate so called "true value" of this series, which defined this way:

• if everything is OK then this is just the last number
• if there are too much the same number in series (because measurement problems) then "true value" shouldn't change too often. most adequate value should be used

My suggestion would be:

• just prohibit "true value" to have the same price more than once per second.

For example:

1. 0.000 : 98; true value = 98
2. 0.100 : 100; true value = 100
3. 0.200 : 101; true value = 101
4. 0.300 : 100; true value = 101 (100 was already used at step 2 0.2 seconds ago)
5. 0.400 : 98; true value = 101 (98 was already used at step 1 0.4 seconds ago)
6. 0.500 : 99; true value = 99
7. 0.600 : 100; true value = 99 (100 was used at step 2 0.5 seconds ago)
8. 1.500 : 101; true value = 101 (more than one second is elapsed since 101 was used).

However such approach also has such "bug":

1. 99 tp = 99
2. 100 tp = 100
3. 101 tp = 101
4. 102 tp = 102
5. 103 tp = 103
6. 102 tp = 103
7. 101 tp = 103
8. 100 tp = 103
9. 99 tp = 103
10. 98 tp = 98
11. 97 tp = 97

The problem here is that "tp" was "frozen" at level 103 for too long time.

I'm really sorry for such a big question. But probably someone was solving close problem and can share experience. My primary problem is that I do need to solve two opposite problems at the same time:

• "true price" must just be the "last" value under general conditions
• "true price" shouldn't change to often (so I need to use previous values one way or another)
• also it's hard to say when we have "general" conditions and when we have "measurement problems"

So my question is really vague and I just hope that someone was tring to solve something like that. As always "common sense" should be used solving this problem.

Answer 1

You want to respond to long-term (low frequency) changes in price, but ignore small short-term (high frequency) fluctuations. So what you need is a low-pass filter.

There are lots of different kinds of low-pass filters with different characteristics. A moving average is one kind of low-pass filter, but it has the problem (which you observe) that it trades smoothing for lag (in order to smooth out high-frequency fluctuations you need to average over quite a long sequence, but then it takes a long time for low-frequency changes to become apparent).

So you might try out the other low-pass filters, for example the exponentially-weighted moving average, and see if they meet your requirements.

The other thing to say about this task is that basing your decision about when you do a recalculation on the nearest integer to your moving average is a bad thing to do, because it amplifies small fluctuations when the moving average is close to halfway between two integers. (If the moving average goes from 99.49 to 99.51 the rounding operation amplifies this small fluctuation to a big one from 99 to 100.)

What you should do instead is use hysteresis to avoid this kind of response. When you do a recalculation, record the current value a₀ of your moving average a, and don't do another recalculation until the moving average a has moved by at least ε, that is, |a − a₀| ≥ ε.