StreamInsight 2.1 against a SQL Database and ... How does it work?

Question

I've installed StreamInsight 2.1. I've been able to turn it on and Server.Create(), expose a WCF endpoint, and connect to it with the Event Flow Debugger. I've found plenty of examples of creating fake event sources.

I'd like to know how to create what I think to be possible: a standing query against a SQL 2012 database. I haven't found any sample code for connecting to an RDB - just statements that "This is just for show, for real you'll maybe connect to an RDB".

Can it only use LINQ-to-SQL? Entity Framework 4? plainer ADO.NET? Some MS SQL-specific protocol?

I guess the part that really confuses me is how these standing queries are supposed to work. It would seem odd to me, these days, for Microsoft to have an exposed capability in SQL Server that is ONLY accessible by a certain independent SKU (StreamInsight). Therefore, it must be one of these two:

Is SQL aware of the standing-ness of the query or does StreamInsight just modify a LINQ statement to poll efficiently (like by adding AND @LastExecution <= InsertedOn and proper indexes).

If SQL is aware of it as a special kind of query, what mechanism does StreamInsight use to create such a standing query? Is it something that a programmer can utilize without StreamInsight Services and so on?

Answer 1

The standing queries you mentioned are the ones written using LINQ that run inside the StreamInsight engine and not SQL Server itself. With StreamInsight, we the developers are on the hook for writing the adapter/source/sink code to get data in and out.

Can it only use LINQ-to-SQL? Entity Framework 4? plainer ADO.NET? Some MS SQL-specific protocol?

Yes. I have written this code before and I've had good luck with just plain ADO.NET. For what our needs were, it was way more flexible than LINQ-to-SQL and Entity Framework.

Do you have control over the schema of the table(s) you are querying? If you do, I would suggest adding some kind of LastUpdated datetime column that gets updated when the data in your table changes. That way you can write the query in such a way that only pulls back the changes without returning the whole table itself. Your adapter/source will need to have some kind of timer that executes your SQL statement periodically. You'll also need to maintain the last executed datetime so you can plug that into your SQL statement.

Answer 2

You could create a StreamInsight input adapter for SSSB to generate events based on messages from Server Broker queues.

Answer 3

I've not worked through this solution, but it seems to me that it would be possible to implement Service Broker or a SQL query notification in SQL Server to message the application on data change, then use Streaminsight to process and respond to this altered state.

I think this would give the effect of a "standing query" and avoid the (potentially wasteful) overhead of polling.

Answer 4

I'd like to know how to create what I think to be possible: a standing query against a SQL 2012 databas

And here we go. Not possible. Period. Why you think that is possible? Need to poll, the best is the ever waiting poll for the message queue, but no, standing query NOT POSSIBLE. Too bad.

Where did you get the funny idea it is possible?