Graceful iteration over an IronPython generator from C#

Question

Fairly well versed in C# and Python (but ignorant of the new dynamic features in .NET 4.x), I've recently decided to add IronPython scripting support to one of my C# apps. I have all the basic communications working such as passing an Action<>s in and calling out from Python into my C# code.

But I'm having trouble finding documentation or examples using python generators from C# code. I have a solution but it's far from elegant.

I define a generator in my python code ("test.py"):

def do_yield():
    yield 1
    yield 2
    yield 3

In C#, I set up my IronPython environment:

// set up IronPython
var ipy = Python.CreateRuntime();
test = ipy.UseFile("test.py");

I've defined a helper function to convert from the returned PythonGenerator to a C# iterator, so my foreach looks like:

foreach (int n in PythonGeneratorToCSharpIterator<int>(test.do_yield()))
{
    Log("py gen returned: " + n);
}

And my helper function:

IEnumerable<T> PythonGeneratorToCSharpIterator<T>(IronPython.Runtime.PythonGenerator pyGenerator)
{
    while (true) {
        T t = default(T);

        try {
            // get the next value
            object o = pyGenerator.next();

            // will throw an exception if it's not the correct type (or no more values in the generator)
            t = (T)o; 
        }
        catch (Exception ex) {
            break; // break out of the while loop and return from the iterator
        }

        yield return t; // this can't be inside try/catch
    }
}

The python generator returns a LightException when there are no more values to return, so the line t = (T)o will throw an exception since it is trying to cast this to an int.

One problem with this is that I'm not catching and properly handling any exceptions from the python code. I'm just throwing them away and exiting the loop. Another is that I'm throwing an exception when I'm out of values from the generator, and I'd prefer a boolean check to test that the value returned is invalid.

Maybe it's my ignorance of the new dynamic aspects of .NET that's preventing me from understanding how to code this properly. Is there a better/more standard way to code this in C#/IronPython?

EDIT

From vcsjones' comment, I now have this code which works well for me, using Linq's Cast extension method:

var gen = (IronPython.Runtime.PythonGenerator)test.do_yield();
foreach(int n in gen.Cast<int>()) {
    Log("py gen returned (with Cast): " + n);
}

Answer 1

This is easy to work with using dynamic:

var ipy = Python.CreateRuntime();
dynamic test = ipy.UseFile("test.py");
foreach (int n in test.do_yield())
{
    Console.WriteLine("py gen returned: " + n);
}

// outputs
py gen returned: 1
py gen returned: 2
py gen returned: 3

The one big thing you should note is that extension methods (like LINQ) don't work with dynamic, since extension methods are just static method calls with some syntactic sugar applied. And the object in this case implements IEnumerable but not IEnumerable<int>, limiting your options for how to deal with this.

Let's say you want to filter the items to even ones. This is one way to do it:

var ipy = Python.CreateRuntime();
dynamic test = ipy.UseFile("test.py");
foreach (int n in test.do_yield())
{
    if (n % 2 == 0)
        Console.WriteLine("py gen returned: " + n);
}

Or you can cast and then use LINQ extension methods (this works because the IEnumerable cast lets the rest of the code be evaluated at compile time, like non-dynamic code):

var ipy = Python.CreateRuntime();
dynamic test = ipy.UseFile("test.py");
foreach (int n in ((IEnumerable)test.do_yield()).Cast<int>().Where(n => n % 2 == 0))
{
    Console.WriteLine("py gen returned: " + n);
}

In some instances, you might wish to use extension methods without their syntactic sugar, e.g.

// these lines are generally equivalent, unless someList is dynamic
var filteredList = Enumerable.Where(someList, x => x % 2 == 0);
var filteredList = someList.Where(x => x % 2 == 0);