javascript 39 lines · 7 steps

Paginated APIs with async generators

Stream items from a paginated API lazily using async generators and for-await-of.

Explained by highlit
Advanced async-iterators generators pagination lazy-evaluation promises
1async function* fetchPages(baseUrl, maxPages) {
2 let page = 1;
3 while (page <= maxPages) {
4 const data = await mockFetch(`${baseUrl}?page=${page}`);
5 if (!data.items.length) return;
6 yield* data.items;
7 if (!data.hasMore) return;
8 page++;
9 }
10}
11 
12function mockFetch(url) {
13 return new Promise((resolve) => {
14 const match = url.match(/page=(\d+)/);
15 const page = Number(match[1]);
16 setTimeout(() => {
17 resolve({
18 items: page <= 3 ? [`item-${page}a`, `item-${page}b`] : [],
19 hasMore: page < 3,
20 });
21 }, 10);
22 });
23}
24 
25async function collectAll(baseUrl) {
26 const results = [];
27 for await (const item of fetchPages(baseUrl, 10)) {
28 results.push(item);
29 }
30 return results;
31}
32 
33async function* take(asyncIterable, n) {
34 let count = 0;
35 for await (const value of asyncIterable) {
36 if (count++ >= n) return;
37 yield value;
38 }
39}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Async generators let you model an unbounded paginated source as a flat stream of items, fetching pages only as they're consumed.
  2. 2yield* delegates to an iterable, flattening a page's items into the generator's own output without a manual loop.
  3. 3Because consumption is lazy, a wrapper like take can stop early and the producer simply never fetches the pages it doesn't need.

Related explainers

javascript 
const ROLE_PERMISSIONS = {
  admin: ['users:read', 'users:write', 'billing:read', 'billing:write'],
  manager: ['users:read', 'billing:read'],
  member: ['users:read'],

Role-based permissions middleware in Express

authorization middleware rbac
Intermediate 9 steps
javascript 
function attachThousandSeparators(input, { locale = 'en-US' } = {}) {
  const formatter = new Intl.NumberFormat(locale);
  const groupSep = formatter.format(11111).replace(/\d/g, '')[0] || ',';
  const decimalSep = formatter.format(1.1).replace(/\d/g, '')[0] || '.';

Live thousand separators without losing the caret

dom intl caret-preservation
Advanced 8 steps
javascript 
import { useReducer, useEffect } from "react";
 
const initialState = { status: "idle", data: null, error: null };
 

Building a data-fetching hook in React

custom-hooks usereducer data-fetching
Intermediate 9 steps
ruby 
module Paginatable
  extend ActiveSupport::Concern
 
  private

A reusable pagination concern in Rails

pagination concerns http-headers
Intermediate 8 steps
javascript 
const express = require('express');
const app = express();
 
app.get('/health', (req, res) => res.json({ status: 'ok' }));

Graceful shutdown in an Express server

graceful-shutdown signal-handling connection-tracking
Advanced 9 steps
javascript 
import { useState, useEffect, useCallback } from 'react';
 
function getColumnCount(width) {
  if (width < 640) return 1;

A responsive column hook in React

custom-hooks debouncing responsive-design
Intermediate 7 steps

Share this explainer

Here's the card — post it anywhere.

Paginated APIs with async generators — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code