StreamInsight complex query with filter on HoppingWindow

Question

I'm trying out StreamInsight and I came across a problem with a query I need.

I'm trying to throw a warning if there are several changes in my measured values (of up to 20% change) in the last 30 minutes.

This is the query I came up with for now but it isn't working and it's not even correct I think.

Apparently I can't filter on a window...?

var deviationQuery = from s in wcfStream
                     group s by s.SensorId into grouped
                     from window in grouped.HoppingWindow(TimeSpan.FromMinutes(30),TimeSpan.FromMinutes(1))
                     where window.StdDev(e => e.Value) > measurableValue * 1.2
                     select new OutputEvent
                     {
                         Error = "Deviation"
                     };

Thanks in advance!

Answer 1

I found a working query for my problem. At first I thought it didn't work but I misinterpreted the results. It may not be the shortest and best query, so if you have a better answer, please tell me!

var deviationQuery = from s in wcfStream
                     where s.Value > measurableValue * (1 + deviationThreshold) || s.Value < measurableValue * (1 - deviationThreshold)
                     group s by s.SensorId into grouped
                     from window in grouped.HoppingWindow(TimeSpan.FromSeconds(180), TimeSpan.FromSeconds(120))
                     select window.Count();
var deviation = from c in deviationQuery
                where c > maxIncorrectValues
                select new OutputEvent
                    {
                        M = new Measurement() { SensorId = "354354", Value = 53, Time = DateTime.Now },
                        Deflection = c,
                        Error = "Deviation"
                    };

Answer 2

If I understand correctly, this is what you want to do:

1. Group the stream by SensorId.
2. Divide each group into 30-minute windows.
3. Write an error message for the windows that have too many incorrect values.

This should do it, hopefully.

var deviationQuery = from s in wcfStream
                     group s by s.SensorId into grouped
                     from window in grouped.HoppingWindow(TimeSpan.FromMinutes(30),TimeSpan.FromMinutes(1))
                     where window.Count(event => event.Value > maxValue) > maxIncorrectValues                        
                     select new OutputEvent
                     {
                         Error = "Deviation"
                     };