JavaScript Generator Functions

  • Generators are functions which can be paused (yielded) and resumed midway of execution.
  • The state of the function is retained between pauses.
  • Generators can also send out or take in values at pauses.

Generator as a function with pauses

  • Below is a simple example for a generator.
  • Generators are marked with a * after the function keyword.
  • Execution gets paused whenever yield is encountered.
  • On pause, the control is passed back to the caller.
function * demo(){
    console.log('-----------1----------')
    console.log('---------------pausing');
    yield;
    console.log('-----------2----------')
    console.log('---------------pausing');
    yield
    console.log('-----------3----------')
    console.log('-------------returning');
}

Below is an example of how the above generator could be run. Invoking the generator, does not run it. Instead it returns an object which has a method next through which the function could be executed through the pauses.

const iterator = demo();
console.log('starting--------------');
iterator.next();
console.log('resuming--------------');
iterator.next();
console.log('resuming--------------');
iterator.next();

First call to iterator.next starts the execution and continues to the first yield. Each subsequent call to iterator.next resumes execution from the last yield to the next yield. Below is how the result would look.

starting--------------
-----------1----------
---------------pausing
resuming--------------
-----------2----------
---------------pausing
resuming--------------
-----------3----------
-------------returning

Generator as a producer / stream of values

function * producer(){
    //random code here
    yield 1;
    //random code here
    yield 2;
}

Generators can send out values when yielding. These values can be accessed by the caller from the object returned by iterator.next.

let result = null;
const iterator = producer();
result = iterator.next();
console.log(`${result.value} received by caller`);
result = iterator.next();
console.log(`${result.value} received by caller`);

// output:
// 1 received by caller
// 2 received by caller

Object return by iterator.next() has two properties:

  • value - the actual value yielded by generator
  • done - a boolean to let the caller know if the generator has finished execution to completion.

The producer function above can be run to completion with this api and a while loop.

const iterator = producer();
let result = iterator.next();
while(!result.done){
    console.log(`${result.value} received by caller`);
    result = iterator.next();
}

// output:
// 1 received by caller
// 2 received by caller

Looping over generator iterables can be further simplified like looping over arrays using for...of construct. for...of automatically handles dealing with the iterator api.

for(let i of producer()){
    console.log(`${i} received by caller`);
}

// output:
// 1 received by caller
// 2 received by caller

Lazily evaluated, memory efficient, on-demand stream of data

function * stream(){
    for(let i=0;i<10;i++){
        yield i;
    }
}

//prints all values (0-9) yielded by the generator
for(let i of stream()){
    console.log(i);
}
  • Generator functions can act as a lazy-evaluated version of an array.
  • Unlike arrays, values are not pre-determined. It is evalutated and pulled on demand.
  • Unlike arrays, values don’t require memory allocated and does not require memory growing with growing length of data.

Infinite stream of data

function * fibonacciStream(){
    let secondLast = 0;
    let last = 1;
    yield secondLast;
    yield last;
    while(true){
        value = secondLast + last;
        secondLast = last;
        last = value;
        yield value;
    }
}

const iterator = fibonacciStream();

//get first 100 values from the Fibonacci Series
for(let i=0;i<100;i++){
    console.log(iterator.next().value);
}

An infinite stream of data can be easily created with yield inside an endless loop. Lazily evaluated, memory efficient and on-demand attributes apply.

Lazy stream-processing pipeline

//sends stream
function * range(limit){
    for(let i=0;i<limit;i++){
        yield i;
    }
}

//receives and sends stream
function * square(numbers){
    for(let number of numbers){
        yield number * number;
    }
}

//receives and sends stream
function * add5(numbers){
    for(let number of numbers){
        yield number + 5;
    }
}

//declarations
const numbersLessThan10 = range(10);
const squaresOfNumbersLessThan10 = square(numbersLessThan10);
const squaresOfNumbersLessThan10Added5 = add5(squaresOfNumbersLessThan10);

//evalution
for(let i of squaresOfNumbersLessThan10Added5){
    //prints (x^2+5) where x starts from 0 and continues to 9
    console.log(i);
}
  • With generators linked to each other, a stream-processing pipeline can be created.
  • In the above example, range, square and add5 are generators. range receives a number and sends out a stream of values. square and add5 receive streams, process them and send out new streams. These are like UNIX commands which allow piping in and piping out of streams.
  • Relationships between the generators can be declared, without any actual evaluation. If these were UNIX commands, it would have looked like range 10 | square | add5.
  • In the above example, evaluation actually starts with the for...of loop and continues sequentially for each number in the stream generated by range. for...of loop only refers the last generator in pipeline. The other generators are iterated on demand through the pipeline.

Generator as a consumer / observer of values

When yield is used on the RHS of an assignment, the generator can accept values from the caller.

function * consumer(){
    let input = null;
    input = yield;
    console.log(`${input} received by generator`);
    input = yield;
    console.log(`${input} received by generator`);
}

const iterator = consumer();
iterator.next();
iterator.next(3);
iterator.next(4);

//output:
//3 received by generator
//4 received by generator
  • iterator.next() takes the execution to the first yield inside consumer.
  • iterator.next(3) passes 3 to the first yield, resumes execution, and continues execution till second yield.
  • iterator.next(4)passes 4 to the second yield, resumes execution, and continues execution till the end.

Generator as a coroutine (producer-consumer)

yield keyword can be used as in the below example, to send and receive values simultaneously, like a coroutine. A coroutine is a multi-tasker program which can communicate bi-directionally with a main program.

function * coroutine(){
    let input = null;
    input = yield 1;
    console.log(`${input} received by generator`);
    input = yield 3;
    console.log(`${input} received by generator`);
}

//main program
let result = null;
const iterator = coroutine();
result = iterator.next();
console.log(`${result.value} received by caller`);
result = iterator.next(2);
console.log(`${result.value} received by caller`);
result = iterator.next(4);

//output:
//1 received by caller
//2 received by generator
//3 received by caller
//4 received by generator
  • iterator.next() takes the execution to the first yield inside coroutine. Since the first yield sends out 1, the main program receives 1.
  • iterator.next(2) passes 2 to the first yield, resumes execution, and continues execution till second yield. Since the second yield sends out 3, the main program receives 3.
  • iterator.next(4) passes 4 to the second yield, resumes execution, and continues execution till the end.

Synchronous code for asynchronous actions

  • Coroutines can be used as a pattern to write synchronous code for asynchronous calls.
  • This can be achieved by ‘yielding’ the promise returned from asynchronous calls.
  • The main program will receive the promise and once the promise is resolved, the main program can insert the result of the resolved promise back to the coroutine.
  • Coroutine remains in paused state while asynchronous action is running.
const asynchronousCall1 = () => {
    return new Promise(resolve => {
        setTimeout(() => resolve(1000),1000);
    });
}

const asynchronousCall2 = () => {
    return new Promise(resolve => {
        setTimeout(() => resolve(2000),2000);
    });
}

function * coroutine(){
    const resultOfAsynchronousCall1 = yield asynchronousCall1();//1000
    const resultOfAsynchronousCall2 = yield asynchronousCall2();//2000
    console.log(resultOfAsynchronousCall1 + resultOfAsynchronousCall2);//3000
}

//main program or library
const iterator = coroutine();
const promise1 = iterator.next().value;
promise1.then(value1 => {
    const promise2 = iterator.next(value1).value;
    return promise2;
}).then(value2 => {
    iterator.next(value2);
});
  • Coroutine remains synchronous while the main program handles sending results of asynchronous calls back to the coroutine and resuming it.
  • The functions of the main program can be automated and abstracted.
  • Coroutines run under the hood for async/await. The main program’s functions are abstracted by the environment.
  • There are libraries which let us write coroutines for asynchronous calls in a similar way. Such libraries abstract the grunt work and let us write synchronous code. Examples would be co.js and redux-saga.