The wonders of the yield keyword

StackOverflow https://stackoverflow.com/questions/4455796

Question

Ok, as I was poking around with building a custom enumerator, I had noticed this behavior that concerns the yield

Say you have something like this:

  public class EnumeratorExample 
  {

        public static IEnumerable<int> GetSource(int startPoint) 
        {
                int[] values = new int[]{1,2,3,4,5,6,7};
                Contract.Invariant(startPoint < values.Length);
                bool keepSearching = true;
                int index = startPoint;

                while(keepSearching) 
                {
                      yield return values[index];
                      //The mind reels here
                      index ++ 
                      keepSearching = index < values.Length;
                }
        }

  }

What makes it possible underneath the compiler's hood to execute the index ++ and the rest of the code in the while loop after you technically do a return from the function?

Was it helpful?

Solution

The compiler rewrites the code into a state machine. The single method you wrote is split up into different parts. Each time you call MoveNext (either implicity or explicitly) the state is advanced and the correct block of code is executed.

Suggested reading if you want to know more details:

OTHER TIPS

The compiler generates a state-machine on your behalf.

From the language specification:

10.14 Iterators

10.14.4 Enumerator objects

When a function member returning an enumerator interface type is implemented using an iterator block, invoking the function member does not immediately execute the code in the iterator block. Instead, an enumerator object is created and returned. This object encapsulates the code specified in the iterator block, and execution of the code in the iterator block occurs when the enumerator object’s MoveNext method is invoked. An enumerator object has the following characteristics:

• It implements IEnumerator and IEnumerator, where T is the yield type of the iterator.

• It implements System.IDisposable.

• It is initialized with a copy of the argument values (if any) and instance value passed to the function member.

• It has four potential states, before, running, suspended, and after, and is initially in the before state.

An enumerator object is typically an instance of a compiler-generated enumerator class that encapsulates the code in the iterator block and implements the enumerator interfaces, but other methods of implementation are possible. If an enumerator class is generated by the compiler, that class will be nested, directly or indirectly, in the class containing the function member, it will have private accessibility, and it will have a name reserved for compiler use (§2.4.2).

To get an idea of this, here's how Reflector decompiles your class:

public class EnumeratorExample
{
    // Methods
    public static IEnumerable<int> GetSource(int startPoint)
    {
        return new <GetSource>d__0(-2) { <>3__startPoint = startPoint };
    }

    // Nested Types
    [CompilerGenerated]
    private sealed class <GetSource>d__0 : IEnumerable<int>, IEnumerable, IEnumerator<int>, IEnumerator, IDisposable
    {
        // Fields
        private int <>1__state;
        private int <>2__current;
        public int <>3__startPoint;
        private int <>l__initialThreadId;
        public int <index>5__3;
        public bool <keepSearching>5__2;
        public int[] <values>5__1;
        public int startPoint;

        // Methods
        [DebuggerHidden]
        public <GetSource>d__0(int <>1__state)
        {
            this.<>1__state = <>1__state;
            this.<>l__initialThreadId = Thread.CurrentThread.ManagedThreadId;
        }

        private bool MoveNext()
        {
            switch (this.<>1__state)
            {
                case 0:
                    this.<>1__state = -1;
                    this.<values>5__1 = new int[] { 1, 2, 3, 4, 5, 6, 7 };
                    this.<keepSearching>5__2 = true;
                    this.<index>5__3 = this.startPoint;
                    while (this.<keepSearching>5__2)
                    {
                        this.<>2__current = this.<values>5__1[this.<index>5__3];
                        this.<>1__state = 1;
                        return true;
                    Label_0073:
                        this.<>1__state = -1;
                        this.<index>5__3++;
                        this.<keepSearching>5__2 = this.<index>5__3 < this.<values>5__1.Length;
                    }
                    break;

                case 1:
                    goto Label_0073;
            }
            return false;
        }

        [DebuggerHidden]
        IEnumerator<int> IEnumerable<int>.GetEnumerator()
        {
            EnumeratorExample.<GetSource>d__0 d__;
            if ((Thread.CurrentThread.ManagedThreadId == this.<>l__initialThreadId) && (this.<>1__state == -2))
            {
                this.<>1__state = 0;
                d__ = this;
            }
            else
            {
                d__ = new EnumeratorExample.<GetSource>d__0(0);
            }
            d__.startPoint = this.<>3__startPoint;
            return d__;
        }

        [DebuggerHidden]
        IEnumerator IEnumerable.GetEnumerator()
        {
            return this.System.Collections.Generic.IEnumerable<System.Int32>.GetEnumerator();
        }

        [DebuggerHidden]
        void IEnumerator.Reset()
        {
            throw new NotSupportedException();
        }

        void IDisposable.Dispose()
        {
        }

        // Properties
        int IEnumerator<int>.Current
        {
            [DebuggerHidden]
            get
            {
                return this.<>2__current;
            }
        }

        object IEnumerator.Current
        {
            [DebuggerHidden]
            get
            {
                return this.<>2__current;
            }
        }
    }
}

Yield is magic.

Well, not really. The compiler generates a full class to generate the enumeration that you're doing. It's basically sugar to make your life simpler.

Read this for an intro.

EDIT: Wrong this. Link changed, check again if you have once.

That's one of the most complex parts of the C# compiler. Best read the free sample chapter of Jon Skeet's C# in Depth (or better, get the book and read it :-)

Implementing iterators the easy way

For further explanations see Marc Gravell's answer here:

Can someone demystify the yield keyword?

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top