typescript 51 lines · 8 steps

Streaming cursor pagination with async generators

An async generator walks every page of a cursor-based API and yields items one at a time, hiding the pagination loop from callers.

Explained by highlit

Intermediate async-generators pagination generics fetch cursors

1interface Page<T> {
2  items: T[];
3  nextCursor: string | null;
4}
5 
6interface FetchOptions {
7  pageSize?: number;
8  signal?: AbortSignal;
9}
10 
11async function fetchPage<T>(
12  endpoint: string,
13  cursor: string | null,
14  pageSize: number,
15  signal?: AbortSignal,
16): Promise<Page<T>> {
17  const url = new URL(endpoint);
18  url.searchParams.set("limit", String(pageSize));
19  if (cursor) url.searchParams.set("cursor", cursor);
20 
21  const res = await fetch(url, {
22    headers: { Accept: "application/json" },
23    signal,
24  });
25 
26  if (!res.ok) {
27    throw new Error(`Request failed: ${res.status} ${res.statusText}`);
28  }
29 
30  const body = (await res.json()) as {
31    data: T[];
32    meta: { next_cursor: string | null };
33  };
34 
35  return { items: body.data, nextCursor: body.meta.next_cursor };
36}
37 
38export async function* paginate<T>(
39  endpoint: string,
40  { pageSize = 100, signal }: FetchOptions = {},
41): AsyncGenerator<T, void, unknown> {
42  let cursor: string | null = null;
43 
44  do {
45    const page = await fetchPage<T>(endpoint, cursor, pageSize, signal);
46    for (const item of page.items) {
47      yield item;
48    }
49    cursor = page.nextCursor;
50  } while (cursor !== null);
51}

01 / 01

STEP 01

Walkthrough

Space play ←→ step click any line

Three takeaways

1Async generators let you expose an endless paginated resource as a simple item-by-item stream.
2Cursor-based pagination loops until the server stops handing back a next cursor.
3Generic type parameters carry the item shape through fetch, parse, and yield without casting at each call site.

Related explainers

typescript

import {
  Injectable,
  PipeTransform,
  ArgumentMetadata,

A custom validation pipe in NestJS

validation dto recursion

Intermediate 10 steps

typescript

import {
  CallHandler,
  ExecutionContext,
  Injectable,

Recording HTTP metrics with a NestJS interceptor

interceptor observability prometheus

Intermediate 5 steps

rust

#[derive(Debug, Default)]
pub struct RequestBuilder {
    url: String,
    method: String,

The builder pattern in Rust

builder-pattern method-chaining ergonomic-api

Intermediate 8 steps

java

package com.example.lb;
 
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;

A thread-safe round-robin load balancer in Java

concurrency load-balancing round-robin

Intermediate 9 steps

typescript

import { Component, inject, effect } from '@angular/core';
import { FormControl, ReactiveFormsModule } from '@angular/forms';
import { ActivatedRoute, Router } from '@angular/router';
import { toSignal } from '@angular/core/rxjs-interop';

Debounced search that syncs to the URL in Angular

signals reactive-forms debouncing

Advanced 8 steps

typescript

import { NestFactory } from '@nestjs/core';
import { DocumentBuilder, SwaggerModule } from '@nestjs/swagger';
import { ValidationPipe } from '@nestjs/common';
import { ApiProperty } from '@nestjs/swagger';