typescript 51 lines · 8 steps

Streaming cursor pagination with async generators

An async generator walks every page of a cursor-based API and yields items one at a time, hiding the pagination loop from callers.

Explained by highlit

Intermediate async-generators pagination generics fetch cursors

1interface Page<T> {
2  items: T[];
3  nextCursor: string | null;
4}
5 
6interface FetchOptions {
7  pageSize?: number;
8  signal?: AbortSignal;
9}
10 
11async function fetchPage<T>(
12  endpoint: string,
13  cursor: string | null,
14  pageSize: number,
15  signal?: AbortSignal,
16): Promise<Page<T>> {
17  const url = new URL(endpoint);
18  url.searchParams.set("limit", String(pageSize));
19  if (cursor) url.searchParams.set("cursor", cursor);
20 
21  const res = await fetch(url, {
22    headers: { Accept: "application/json" },
23    signal,
24  });
25 
26  if (!res.ok) {
27    throw new Error(`Request failed: ${res.status} ${res.statusText}`);
28  }
29 
30  const body = (await res.json()) as {
31    data: T[];
32    meta: { next_cursor: string | null };
33  };
34 
35  return { items: body.data, nextCursor: body.meta.next_cursor };
36}
37 
38export async function* paginate<T>(
39  endpoint: string,
40  { pageSize = 100, signal }: FetchOptions = {},
41): AsyncGenerator<T, void, unknown> {
42  let cursor: string | null = null;
43 
44  do {
45    const page = await fetchPage<T>(endpoint, cursor, pageSize, signal);
46    for (const item of page.items) {
47      yield item;
48    }
49    cursor = page.nextCursor;
50  } while (cursor !== null);
51}

01 / 01

STEP 01

Walkthrough

Space play ←→ step click any line

Three takeaways

1Async generators let you expose an endless paginated resource as a simple item-by-item stream.
2Cursor-based pagination loops until the server stops handing back a next cursor.
3Generic type parameters carry the item shape through fetch, parse, and yield without casting at each call site.

Related explainers

typescript

import { registerLocaleData } from '@angular/common';
import localeFr from '@angular/common/locales/fr';
import localeFrExtra from '@angular/common/locales/extra/fr';
import localeDe from '@angular/common/locales/de';

Locale-aware bootstrapping in Angular

i18n localization dependency-injection

Intermediate 8 steps

typescript

import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import * as Joi from 'joi';

Validating env config at boot in NestJS

configuration schema-validation environment-variables

Intermediate 8 steps

typescript

import { Inject, Injectable, Logger } from '@nestjs/common';
import { CACHE_MANAGER } from '@nestjs/cache-manager';
import { Cache } from 'cache-manager';
import { InjectRepository } from '@nestjs/typeorm';

A cache-aside country lookup in NestJS

cache-aside dependency-injection batch-lookup

Intermediate 8 steps

typescript

import { Injectable, effect, signal, computed } from '@angular/core';
 
interface Preferences {
  theme: 'light' | 'dark';

A signal-based preferences store in Angular

signals state-management persistence

Intermediate 7 steps

typescript

import { useEffect, useState } from "react";
 
interface Section {
  id: string;

Building a scroll-spy hook in React

custom-hooks intersectionobserver dom-observation

Intermediate 8 steps

typescript

import { Injectable, Scope, Inject, NotFoundException } from '@nestjs/common';
import { REQUEST } from '@nestjs/core';
import { Request } from 'express';
import { DataSource } from 'typeorm';