Why is there a different runtime behaviour with deferred execution using the “yield” keyword in c#?

Question

If you call the IgnoreNullItems extension method in the sammple code below the deferred execution works as expected however when using the IgnoreNullItemsHavingDifferentBehaviour the exception is raised immediately. Why?

List<string> testList = null;
testList.IgnoreNullItems(); //nothing happens as expected

testList.IgnoreNullItems().FirstOrDefault();
//raises ArgumentNullException as expected

testList.IgnoreNullItemsHavingDifferentBehaviour(); 
//raises ArgumentNullException immediately. not expected behaviour -> 
//  why is deferred execution not working here?

Thanks for sharing your ideas!

Raffael Zaghet

public static class EnumerableOfTExtension
{
    public static IEnumerable<T> IgnoreNullItems<T>(this IEnumerable<T> source)
        where T: class
    {
        if (source == null) throw new ArgumentNullException("source");

        foreach (var item in source)
        {
            if (item != null)
            {
                yield return item;
            }
        }
        yield break;
    }

    public static IEnumerable<T> IgnoreNullItemsHavingDifferentBehaviour<T>(
        this IEnumerable<T> source) 
        where T : class
    {
        if (source == null) throw new ArgumentNullException("source");

        return IgnoreNulls(source);
    }

    private static IEnumerable<T> IgnoreNulls<T>(IEnumerable<T> source)
        where T : class
    {
        foreach (var item in source)
        {
            if (item != null)
            {
                yield return item;
            }
        }
        yield break;
    }
}

Here a version with the same behaviour:

Here a version that shows the same behaviour. Don't let resharper "improve" your foreach statement in this case ;) --> resharper changes the foreach to the "IgnoreNullItemsHavingDifferentBehaviour" version with a return statement.

public static IEnumerable<T> IgnoreNullItemsHavingSameBehaviour<T>(this IEnumerable<T> source) where T : class
            {
                if (source == null) throw new ArgumentNullException("source");

                foreach (var item in IgnoreNulls(source))
                {
                    yield return item;
                }
                yield break;
            }

Answer 1

The exception is raised immediately because IgnoreNullItemsHavingDifferentBehaviour doesn't contain any "yield" itself.

Rather, it's IgnoreNulls which gets converted into an iterator block and thus uses deferred execution.

This is actually the approach that Jon Skeet used in his EduLinq series to force immediate null checks for source sequences. See this post for a more detailed explanation (specifically the "Let's implement it" section).

Answer 2

I haven't tested, but I can guess...

With the IgnoreNullItems method, the whole method is deferred until you being the enumeration. With your alternate method, only the execution of IgnoreNulls is deferred - the null check in IgnoreNullItemsHavingDifferentBehaviour happens immediately.

Answer 3

Deferred execution comes from how yield return works. It will create the state machine inside a method, that will not start or do any code until you try to enumerate first item.

But when there is no yield return it will behave like normal method.

Perfectly explained and shown in Jon Skeet's Edulinq.