ruby 50 lines · 7 steps

Paginating an HTTP API with a Ruby enumerator

A client walks every page of a cursor-paginated REST API and yields each record lazily.

Explained by highlit
Intermediate pagination http enumerator rest-api authentication
1require "net/http"
2require "json"
3require "uri"
4require "base64"
5 
6class GitHubStarsClient
7 BASE = "https://api.example.com/v1"
8 
9 def initialize(token:, page_size: 100)
10 @token = token
11 @page_size = page_size
12 end
13 
14 def each_repository
15 return enum_for(:each_repository) unless block_given?
16 
17 cursor = nil
18 loop do
19 page = fetch_page(cursor)
20 page.fetch("data").each { |repo| yield repo }
21 
22 cursor = page.dig("page_info", "next_cursor")
23 break unless page.dig("page_info", "has_next") && cursor
24 end
25 end
26 
27 private
28 
29 def fetch_page(cursor)
30 query = { "limit" => @page_size }
31 query["cursor"] = cursor if cursor
32 
33 uri = URI("#{BASE}/repositories")
34 uri.query = URI.encode_www_form(query)
35 
36 request = Net::HTTP::Get.new(uri)
37 request["Authorization"] = "Bearer #{@token}"
38 request["Accept"] = "application/json"
39 
40 response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: uri.scheme == "https") do |http|
41 http.request(request)
42 end
43 
44 unless response.is_a?(Net::HTTPSuccess)
45 raise "API request failed: #{response.code} #{response.message}"
46 end
47 
48 JSON.parse(response.body)
49 end
50end
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Returning enum_for when no block is given lets a method act as both an iterator and a lazy enumerator.
  2. 2Cursor-based pagination loops until the API signals there is no next page, carrying the cursor between requests.
  3. 3Checking the response type before parsing turns silent HTTP failures into explicit, actionable errors.

Related explainers

rust 
use axum::{
    body::Body,
    http::{Method, StatusCode, Uri},
    response::{IntoResponse, Response},

Nesting routers and JSON fallbacks in Axum

routing http json-responses
Intermediate 7 steps
typescript 
import { useCallback, useEffect, useRef, useState } from "react";
 
interface Page<T> {
  items: T[];

A cursor-based infinite scroll hook in React

custom-hooks pagination intersection-observer
Intermediate 9 steps
ruby 
class DateRangeMerger
  def initialize(ranges)
    @ranges = ranges
  end

Merging overlapping date ranges in Ruby

sorting interval-merging enumerable
Intermediate 5 steps
go 
package admin
 
import (
	"net/http"

Building a protected admin area in Gin

routing middleware authentication
Intermediate 6 steps
ruby 
module SlowQueryLogger
  SLOW_QUERY_THRESHOLD_MS = 200.0
  IGNORED_PAYLOAD_NAMES = %w[SCHEMA TRANSACTION].freeze
 

Logging slow SQL queries in Rails

instrumentation logging pub-sub
Intermediate 7 steps
ruby 
require "phonelib"
 
class PhoneNumber
  class InvalidNumber < StandardError; end

Wrapping phone parsing in a Ruby value object

value-object memoization validation
Intermediate 7 steps

Share this explainer

Here's the card — post it anywhere.

Paginating an HTTP API with a Ruby enumerator — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code