JavaScript Generators and Iterators: function* and yield

By Engr. Henry Akpan

How for...of Really Works

In JavaScript you can loop over an array, a string, a Map or a Set with the same for...of loop, and spread any of them into an array with [...x]. This works because they all follow one shared rule, the iteration protocol. Once you understand it, you can make your own objects loopable, and you can use JavaScript generators, functions that pause and resume, to produce sequences lazily, including infinite ones. This tutorial explains iterables and iterators, writing a custom iterator, generator functions with function* and yield, lazy pipelines, passing values into generators, yield*, and async generators. Each code block runs on its own in Node.js, and the outputs shown are real.

Iterables and Iterators

Two definitions make up the protocol:

  • An iterator is an object with a next() method. Each call returns an object { value, done }. When the sequence is finished, done is true.
  • An iterable is an object with a method under the special key Symbol.iterator that returns a fresh iterator. Arrays, strings, Maps, Sets and the arguments object are all built-in iterables.

You can drive an iterator by hand to see the protocol at work:

const subjects = ["Maths", "Physics"];
const it = subjects[Symbol.iterator]();
console.log(it.next());
console.log(it.next());
console.log(it.next());

Output:

{ value: 'Maths', done: false }
{ value: 'Physics', done: false }
{ value: undefined, done: true }

A for...of loop simply calls [Symbol.iterator]() once and then next() repeatedly until done is true. Spread syntax, destructuring, Array.from, Promise.all and the Map and Set constructors all use the same protocol.

Plain objects are not iterable, which is why for (const x of {a: 1}) throws a TypeError. Use Object.keys, Object.values or Object.entries to get iterable arrays from them.

Writing a Custom Iterable

Here is a range object that counts from start to end in steps, written with the protocol directly:

const range = {
  start: 1, end: 10, step: 3,
  [Symbol.iterator]() {
    let current = this.start;
    const { end, step } = this;
    return {
      next() {
        if (current > end) return { value: undefined, done: true };
        const value = current;
        current += step;
        return { value, done: false };
      }
    };
  }
};

for (const n of range) console.log(n);
console.log([...range]);

Output:

1
4
7
10
[ 1, 4, 7, 10 ]

That works, but it is a lot of boilerplate. Generators make it much shorter.

JavaScript Generators: function* and yield

A generator function is written with function*. Calling it does not run its body; it returns a generator object, which is both an iterator and an iterable. Each time next() is called, the function runs until the next yield, hands out that value, and pauses, keeping all its local variables, until it is asked for the next value.

function* countTo(n) {
  console.log("starting");
  for (let i = 1; i <= n; i++) {
    yield i;
  }
  console.log("finished");
}

const gen = countTo(2);
console.log("created");
console.log(gen.next());
console.log(gen.next());
console.log(gen.next());

Output:

created
starting
{ value: 1, done: false }
{ value: 2, done: false }
finished
{ value: undefined, done: true }

Notice that "starting" is only printed when the first value is requested. With a generator, the range object above becomes a few lines:

function* range(start, end, step = 1) {
  for (let i = start; i <= end; i += step) yield i;
}
console.log([...range(1, 10, 3)]);

class Playlist {
  constructor(...songs) { this.songs = songs; }
  *[Symbol.iterator]() { yield* this.songs; }
}
for (const s of new Playlist("Essence", "Last Last")) console.log(s);

Output:

[ 1, 4, 7, 10 ]
Essence
Last Last

yield* delegates to another iterable, yielding each of its values in turn, so the Playlist class becomes iterable with a single line.

Infinite Sequences and Lazy Evaluation

Because a generator only computes values when they are asked for, it can describe an infinite sequence safely, as long as the consumer stops. Small helper generators then form lazy pipelines:

function* naturals() {
  let n = 1;
  while (true) yield n++;
}
function* filter(iter, pred) {
  for (const x of iter) if (pred(x)) yield x;
}
function* map(iter, fn) {
  for (const x of iter) yield fn(x);
}
function* take(iter, count) {
  if (count <= 0) return;
  for (const x of iter) {
    yield x;
    if (--count === 0) return;
  }
}

const evensSquared = take(map(filter(naturals(), n => n % 2 === 0), n => n * n), 5);
console.log([...evensSquared]);

Output:

[ 4, 16, 36, 64, 100 ]

Only the first ten natural numbers are ever generated. With arrays you would have to guess in advance how many numbers to create. Lazy pipelines are also memory-friendly when processing large files or data streams one item at a time. Recent versions of JavaScript add similar built-in helpers directly on iterators (such as .map, .filter and .take), but the generator versions work everywhere and show how they are built.

A Practical Example: Paging Through Results

Generators are handy for walking through data in chunks, such as pages of results from an API or rows of a large table:

const students = Array.from({ length: 7 }, (_, i) => `Student ${i + 1}`);

function* pages(items, size) {
  for (let i = 0; i < items.length; i += size) {
    yield items.slice(i, i + size);
  }
}

let page = 1;
for (const chunk of pages(students, 3)) {
  console.log(`Page ${page++}:`, chunk.join(", "));
}

Output:

Page 1: Student 1, Student 2, Student 3
Page 2: Student 4, Student 5, Student 6
Page 3: Student 7

Sending Values into a Generator

next(value) resumes the generator and makes the paused yield expression evaluate to value. This turns a generator into a two-way conversation, here a running average:

function* runningAverage() {
  let total = 0, count = 0, average = null;
  while (true) {
    const score = yield average;
    total += score;
    count++;
    average = total / count;
  }
}

const avg = runningAverage();
avg.next();                      // run to the first yield
console.log(avg.next(70).value);
console.log(avg.next(80).value);
console.log(avg.next(96).value);

Output:

70
75
82

The first next() only starts the generator; the value passed to it would be ignored. Generators also have return(value), which finishes them early (running any finally blocks), and throw(error), which raises an error at the paused yield.

Async Generators

An async function* can await inside and yields promises; consume it with for await...of. This suits data that arrives over time, such as network pages or file chunks:

const wait = (ms) => new Promise(resolve => setTimeout(resolve, ms));

async function* fetchPages(total) {
  for (let p = 1; p <= total; p++) {
    await wait(10);              // pretend to fetch from a server
    yield { page: p, items: p * 10 };
  }
}

(async () => {
  let sum = 0;
  for await (const result of fetchPages(3)) {
    console.log("got page", result.page);
    sum += result.items;
  }
  console.log("total items:", sum);
})();

Output:

got page 1
got page 2
got page 3
total items: 60

Node.js streams are async iterables too, so for await (const chunk of stream) is a clean way to read a file piece by piece.

Summary

  • An iterable has a [Symbol.iterator]() method returning an iterator; an iterator's next() returns { value, done }.
  • for...of, spread and destructuring all use this protocol; plain objects are not iterable.
  • JavaScript generators (function*) pause at each yield and resume on next(), making custom iterables short and readable.
  • Generators are lazy, so they can represent infinite sequences and memory-friendly pipelines.
  • next(value) sends data in, yield* delegates, and async generators with for await handle data that arrives over time.

Related Tutorials