Javascript Generators: Understanding them

Question

I'm pretty sure my understanding of generators is inherently broken. All online resources seem to conflict and it makes for an incredibly difficult and confusing learning experience.

From what I understand, the yield keyword enables a currently executing block of code to wait for a value instead of throwing remaining code to be executed inside a callback. So, as most tutorials have pointed out, you can use this:

(function *() {
     // Wait until users have be got and put into value of `results`
     var results = yield db.get("users");
     // And continue
     view.display(results);
})();

Instead of:

db.get("user", function(results) {
    view.display(results);
});

Right, that's all well and good until I try to write my own generators. I've run into several hitches:

• The first example code I above will not run because there is nothing to iterate over the generator, correct? Some higher being has to call the .next somewhere, right?
• The entire API will have to be rewritten right down to the I/O calls to support generators, correct?
• From what I gather, yield seems to stand for wait for the value most general use cases whereas in the implementation part (read: return value to/inside db.get) yield seems to stand for send this value back to currently waiting block to resume execution.

Take for example:

function *fn() {
    yield 1;
    yield "a";
}

var gen = fn();
gen.next(); // 1
gen.next(); // "a";

yield in that context is sending values back down instead of waiting for the results. In the first example above, it waits for the results from the db.get and the resumes execution instead of "returning" or sending back a value. If the db.get case is true, is this not inherently synchronous? I mean, isn't it exactly the same as:

(function() {
     //Wait for the results
    var results = fs.readFileSync("users.txt");
    // Use results
    view.display(results);
})();

Unfortunately, if it's any way clear from this question (probably the only thing clear) is that I don't understand generators. Hopefully, I might get some insight here.

Answer 1

TL;DR: the essence of generator is controlling the suspension of code execution.

For generator itself, you can refer to this.

To sum up, there is three components you should distinguish: 1. generator function 2. generator 3. generated result

Generator function is simply the function with star in its head and (optional) yield in its body.

function *generator() {
  console.log('Start!');
  var i = 0;
  while (true) {
    if (i < 3)
      yield i++;
  }
}

var gen = generator();
// nothing happens here!!

Generator function itself does not do anything but return a generator, in the case above, gen. No console output here because only after the returned generator's next method is called the body of generator function will run. Generator has several methods, of which the most important one is next. next runs the code and returns the generator result.

var ret = gen.next();
// Start!
console.log(ret);
// {value: 0, done: false}

ret above is generator result. It has two property: value, the value you yield in generator function, and done, a flag indicating whether the generator function return.

console.log(gen.next());
// {value: 1, done: false}
console.log(gen.next());
// {value: 2, done: false}
console.log(gen.next());
// {value: undefined, done: true}

At this point, no one will expect you to understand generator, at least not the async power of generator.

To put it simple, generator has two features:

• one can choose to jump out of a function and let outer code to determine when to jump back into the function.
• the control of asynchronous call can be done outside of your code

In code, yield jumps outside of function, and next(val) jumps back to the function and pass value back into the function. Outside code can handle asynchronous call and decide proper time to switch to your own code.

Look the sample again:

var gen = generator();
console.log('generated generator');
console.log(gen.next().value);
// mock long long processing
setTimeout(function() {
  console.log(gen.next().value);
  console.log('Execute after timer fire');
}, 1000);
console.log('Execute after timer set');

/* result:
    generated generator
    start
    0
    Execute after timer set
    1
    Execute after timer fire
*/

See? The generator function itself does not handle callback. The outside code does.

The base is here. You can elaborate this code to support full asynchronousity while keeping generator function like sync one.

For example, suppose geturl is an asynchronous call that returns a promise object. you can write var html = yield getUrl('www.stackoverflow.com'); This jumps outside your code. And outside code will do stuff like:

var ret = gen.next();
ret.then(function (fetchedHTML) {
  // jumps back to your generator function
  // and assign fetchHTML to html in your code
  gen.next(fetchedHTML);
});

For more complete guide, refer to this. And repository like co, galaxy, suspend and etc.

Answer 2

none of the async stuff if part of generators. generators simply pause and resume blocks of code. all the async magic happens when you use what i call a "generator engine" like https://github.com/visionmedia/co.

basically, what gen.next() does is return the last yielded value and allow you to return a value if the yield is being assigned to something, ex. var something = yield 1. so if you have the block of code:

function* genfun() {
  var a = yield 1
  var b = yield 2
}

var gen = genfun()

gen.next() // returns the first yielded value via `{value: 1}`
gen.next(1) // sets `a` as 1, returns the next yielded value via `{value: 2}`
gen.next(2) // sets `b` as 2, the generator is done, so it returns `{done: true}`

gen.throw(err) is the same as next, except the error is thrown instead of assigned to a variable.

this is how control flow engines work - you get the next value which is probably a callback or something. execute the callback and don't gen.next() until the callback is finished.

Answer 3

The two examples are not the same. When you yield, the function now effectively becomes a callback, waiting to be executed when db.get("users") finishes. This way, the function does not block other function's execution. Think of it as a way to turn synchronous functions into asynchronous functions by letting the system know that you can pause at certain points.