How do you create a reactive dataframe (populated with streaming data) in Deedle?

Question

I want to create a dataframe which would contain streaming data. How is this done in Deedle?

The amount of data rows is not known in advance hence I want to create something like an Rx stream.

Answer 1

The short answer is that Deedle does not currently have support for streaming data.

You can definitely use something like Rx or F# asynchronous sequences to collect data reactively and then pass them to Deedle (once you have a chunk you want to process) but that's not going to be very optimal. Another interesting project to look at is Naiad LINQ which sounds like exactly what you need.

There is definitely some interest in using Deedle with streaming data, and we've been thinking about how to do that - the design of Deedle is fairly extensible so there might be a way to do this. If you're interested in contributing, check out the design notes document.