Kieran Barker

Iterators and generators in JavaScript

Last week, we learned about my NumRange class. It's a simple class for creating numeric ranges. Instead of creating an entire array in memory, it returns an iterable NumRange object. To help us understand how it works, we're going to learn about iterators and generators in JavaScript.

Iteration protocols

There are two iteration protocols we need to understand: the iterable protocol and the iterator protocol. The MDN Web Docs explain that these aren't new built-ins or syntax, but protocols. These protocols can be implemented by any object by following some conventions.

The iterable protocol

Here's the definition of the iterable protocol from the MDN Web Docs:

The iterable protocol allows JavaScript objects to define or customize their iteration behavior, such as what values are looped over in a for...of construct. Some built-in types are built-in iterables with a default iteration behavior, such as Array or Map, while other types (such as Object) are not.

In order to be iterable, an object has to implement an [@@iterator]() method. This can either be directly or through one of the objects in its prototype chain. The @@iterator key is accessible through the Symbol.iterator symbol.

The MDN Web Docs explain further:

Whenever an object needs to be iterated (such as at the beginning of a for...of loop), its @@iterator method is called with no arguments, and the returned iterator is used to obtain the values to be iterated.

Let's add an [@@iterator]() method to our NumRange class. We can't implement the body of the method until we learn about the iterator protocol, but we can at least add the method to the class:

class NumRange {
  constructor(start, stop, step) {
    this.start = start;
    this.stop = stop;
    this.step = step;
  }

  [Symbol.iterator]() {
    // We need to return an object that conforms to the iterator protocol...
  }
}

The iterator protocol

Here's the definition of the iterator protocol from the MDN Web Docs:

The iterator protocol defines a standard way to produce a sequence of values (either finite or infinite), and potentially a return value when all values have been generated.

In order to be an iterator, an object needs to have a next() method which returns an object that conforms to the IteratorResult interface. The iterator can have return() and throw() methods too, but these are optional. The returned object conforms to the IteratorResult interface if it has value and done properties. The value of the value property can be anything; the value of the done property is a boolean that reflects whether the iterator has completed its sequence.

Now we can implement the body of the [@@iterator]() method. We'll declare a variable called num and assign it the value of the this.start property. Then we'll return an object with a next() method. Because we need to access properties of the NumRange instance, we'll use an arrow function. Otherwise this would refer to the iterator object being returned, not the NumRange instance.

In the body of the next() method, we'll check if the value of num is greater than or equal to the value of this.stop. If so, that means we've completed our sequence, so we'll return an object with its value property set to undefined and its done property set to true. Otherwise, we'll declare a variable called value and assign it the value of num. Then we'll increment the value of num by the value of this.step. Finally, we'll return an object with its value property set to the value of the value variable and its done property set to false.

class NumRange {
  constructor(start, stop, step) {
    this.start = start;
    this.stop = stop;
    this.step = step;
  }

  [Symbol.iterator]() {
    let num = this.start;

    return {
      next: () => {
        if (num >= this.stop) return { value: undefined, done: true };
        let value = num;
        num += this.step;
        return { value, done: false };
      },
    };
  }
}

Instances of our NumRange class are now iterable! Syntaxes that expect an iterable, such as spread syntax (...) and for...of loops, will invoke the [@@iterator]() method automatically. Here's a couple of examples:

const range = new NumRange(1, 6, 1);

// spread syntax
console.log(...range);

// for...of loop
for (const num of range) {
  console.log(num);
}

We can also invoke the [@@iterator]() method manually:

const range = new NumRange(1, 6, 1);
const iterator = range[Symbol.iterator]();

let done = false;

while (!done) {
  const result = iterator.next();

  if (typeof result.value !== "undefined") {
    console.log(result.value);
  }

  done = result.done;
}

Iterable iterators

To create an iterator that's also iterable, we need to add an [@@iterator]() method to the iterator that returns this, i.e. it returns the iterator itself. In the following code snippet, pay attention to the body of the NumRange class' [@@iterator]() method. In addition to the next() method, the returned iterator object has an [@@iterator]() method of its own that returns this. Note that this method is not an arrow function, because we want this to refer to the object itself, not the NumRange instance.

class NumRange {
  constructor(start, stop, step) {
    this.start = start;
    this.stop = stop;
    this.step = step;
  }

  [Symbol.iterator]() {
    let num = this.start;

    return {
      next: () => {
        if (num >= this.stop) return { value: undefined, done: true };
        let value = num;
        num += this.step;
        return { value, done: false };
      },

      [Symbol.iterator]() {
        return this;
      },
    };
  }
}

The MDN Web Docs explain why this is useful:

Such [an] object is called an iterable iterator. Doing so allows an iterator to be consumed by the various syntaxes expecting iterables—therefore, it is seldom useful to implement the Iterator Protocol without also implementing Iterable (in fact, almost all syntaxes and APIs expect iterables, not iterators). The generator object is an example.

For example, now we can use spread syntax to iterate over the iterator itself. I'm not sure when we'd ever need to do this, but it works!

const range = new NumRange(1, 6, 1);
const iterator = range[Symbol.iterator]();

console.log(...iterator); // 1 2 3 4 5

If we hadn't implemented an [@@iterator]() method on the iterator that returns this, the preceding code would have thrown a TypeError:

TypeError: Found non-callable @@iterator

Generator functions

Generator objects are a native type of iterable iterator. To create a Generator object, we use a generator function. Generators are functions that can be exited and later re-entered. Their context (variable bindings) will be saved across re-entrances, as the MDN Web Docs explain. We use the yield keyword to do this; it lets us pause and resume the function's execution.

Using a generator function, we can implement our NumRange class' [@@iterator]() method much more cleanly. We'll prepend the method name with an asterisk (*) to indicate that it's a generator method. Then we'll use a regular for loop, yielding the value of i on each invocation.

class NumRange {
  constructor(start, stop, step) {
    this.start = start;
    this.stop = stop;
    this.step = step;
  }

  *[Symbol.iterator]() {
    for (let i = this.start; i < this.stop; i += this.step) {
      yield i;
    }
  }
}

I explained how this works in last week's post:

The first time the generator function is called, it returns a Generator object. When a value is consumed by calling the Generator object's next() method, the generator function executes until it encounters the yield keyword. This is repeated each time the next() method is called until the end of the generator function is reached.

Summary

  • The iterable protocol lets an object customize its iteration behaviour.
  • The iterator protocol defines a standard way to produce a sequence of values.
  • Any object can implement these protocols by following the proper conventions.
  • An object that implements both protocols is called an iterable iterator.
  • Generator objects are a native type of iterable iterator.
  • We use generator functions to create Generator objects.
  • We use the yield keyword to pause/resume a generator function's execution.