Is there a DSL or declarative system for TPL Dataflow?

Question

Is there any DSL or other fully- or partially-declarative mechanism for declaring TPL Dataflow flows? Or is the best (only) practice to just wire them up in code?

Failing that, is there any DSL or other fully- or partially-declarative mechanism for using any dataflow library that I could use as a model and/or source of ideas?

(I've searched without success so maybe one doesn't exist ... or maybe I didn't find it.)

Update: To answer @svick below as to why I want this and what do I gain by it:

First, I just like a sparser syntax that more clearly shows the flow rather than the details. I think

downloadString => createWordList => filterWordList => findPalindromes;

is preferable to

downloadString.LinkTo(createWordList);
createWordList.LinkTo(filterWordList);
filterWordList.LinkTo(findPalindromes);
findPalindromes.LinkTo(printPalindrome);

with its repeated names and extra punctuation. Similar to the way you'd rather use the dot DSL to describe a DAG than a bunch of calls to the Visio DOM API. You can imagine a syntax for network flows, as well as pipelines, such that network flows in particular would be very clear. That may not seem compelling, of course, but I like it.

Second, I think that with a DSL you might be able to persist the DSL description, e.g., as a field in a row in a database, and then instantiate it later. Though perhaps that's a different capability entirely.

Answer 1

Let's start with the relevant facts and work from there:

1. There isn't anything like this for TPL Dataflow yet.
2. There isn't a good way of embedding a DSL into C#. The common compilers are not extensible and it would be hard to access local variables from a string-based DSL.
3. The are several limitations to operators in C#, but the most significant here is that operators can't be generic. This means that the sparser syntax either wouldn't be type-safe (which is unacceptable to me), or it can't use overloaded operators.
4. The IDisposable returned from LinkTo() that can be used to break the created link isn't used that often, so it doesn't have to be supported. (Or maybe the expression that sets up the flow could return a single IDisposable that breaks the whole flow?)

Because of this, I think the best that can be done is something like:

downloadString.Link(createWordList).Link(filterWordList).Link(findPalindromes);

This avoids the repetition of LinkTo(), but is not much better.

The implementation of the simple form of this is mostly trivial:

public static class DataflowLinkExtensions
{
    public static ISourceBlock<TTarget> Link<TSource, TTarget>(
        this ISourceBlock<TSource> source,
        IPropagatorBlock<TSource, TTarget> target)
    {
        source.LinkTo(
            target,
            new DataflowLinkOptions { PropagateCompletion = true });
        return target;
    }

    public static void Link<TSource>(
        this ISourceBlock<TSource> source, ITargetBlock<TSource> target)
    {
        source.LinkTo(
            target,
            new DataflowLinkOptions { PropagateCompletion = true });
    }
}

I chose to set PropagateCompletion to true, because I think that makes the most sense here. But it could also be an option of Link().

---

I think most of the alternative linking operators of Axum are not relevant to TPL Dataflow, but linking multiple blocks to or from the same block could be done by taking a collection or array as one of the parameters of Link():

new[] { source1, source2 }.Link(target);
source.Link(target1, target2);

If Link() actually returned something that represents the whole flow (similar to Encapsulate()), you could combine this to create more complicated flows, like:

source.Link(propagator1.Link(target1), target2);