Question

The problem is pretty simple: extract only the not exists records from 2 different streams using Esper engine.
ID exists in streamA but NOT EXITS in streamB.

In SQL it would look like this:

SELECT *
FROM   tableA
WHERE  NOT EXISTS (SELECT *
                   FROM   tableB
                   WHERE  tableA.Id = tableB.Id)

I've tried it Esper style but it doesn't work:

SELECT *
FROM  streamA.win:ext_timed(timestamp, 5 seconds) as stream_A
WHERE NOT EXSITS
    (SELECT stream_B.Id
     FROM streamB.win:ext_timed(timestamp, 5 seconds) as stream_B
     WHERE stream_A.Id = stream_B.Id)

Sadly if stream_A.Id inserted before stream_B.id than it will answer the query conditions and the query won't work.

Any suggestions on how to identify "ID exists in streamA but NOT EXITS in streamB" using Esper?

Was it helpful?

Solution

One simple way is to time-order the stream, so that A and B are timestamp ordered before sending events in.

Or you could delay A such as this query:

select * from pattern [every a=streamA -> timer:interval(1 sec)] as delayed_a where not exists (... where delayed_a.a.id = b.id)

There is no need for an externally timed window for streamA. For externally timed behavior in general use external timer events.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top