javascript 49 lines · 9 steps

Streaming OpenAI tokens from a Next.js edge route

A Next.js edge route handler pipes OpenAI's streamed completion straight to the client as it arrives.

Explained by highlit
Next.js Advanced streaming edge-runtime readablestream async-iteration llm-api
1import OpenAI from 'openai';
2 
3const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
4 
5export const runtime = 'edge';
6 
7export async function POST(req) {
8 const { messages } = await req.json();
9 
10 if (!Array.isArray(messages) || messages.length === 0) {
11 return Response.json({ error: 'messages required' }, { status: 400 });
12 }
13 
14 const completion = await openai.chat.completions.create({
15 model: 'gpt-4o-mini',
16 messages,
17 stream: true,
18 });
19 
20 const encoder = new TextEncoder();
21 
22 const stream = new ReadableStream({
23 async start(controller) {
24 try {
25 for await (const chunk of completion) {
26 const token = chunk.choices[0]?.delta?.content;
27 if (token) {
28 controller.enqueue(encoder.encode(token));
29 }
30 }
31 } catch (err) {
32 controller.error(err);
33 return;
34 }
35 controller.close();
36 },
37 async cancel() {
38 await completion.controller.abort();
39 },
40 });
41 
42 return new Response(stream, {
43 headers: {
44 'Content-Type': 'text/plain; charset=utf-8',
45 'Cache-Control': 'no-cache, no-transform',
46 Connection: 'keep-alive',
47 },
48 });
49}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Wrapping an async iterator in a ReadableStream lets you forward data to the client as fast as it arrives, without buffering the whole response.
  2. 2Edge runtime keeps latency low for streaming workloads but restricts you to web-standard APIs like Response and TextEncoder.
  3. 3Handling cancel by aborting the upstream request prevents wasted tokens and cost when a client disconnects mid-stream.

Related explainers

javascript 
function evaluate(expression) {
  const tokens = tokenize(expression);
  let pos = 0;
 

Building a recursive descent calculator

parsing recursion operator-precedence
Intermediate 8 steps
javascript 
import { useState, useEffect, useCallback, useRef } from 'react';
 
const cache = new Map();
const inflight = new Map();

Building a stale-while-revalidate hook in React

caching request-deduplication custom-hooks
Advanced 10 steps
python 
import random
from typing import Iterator, List
 
 

How reservoir sampling picks k items

reservoir-sampling streaming randomness
Intermediate 5 steps
javascript 
import { useEffect, useRef, useState } from 'react';
 
export function useDelayedFlag(active, delay = 300) {
  const [visible, setVisible] = useState(false);

Delaying a loading spinner with a React hook

custom-hooks debouncing cleanup
Intermediate 8 steps
javascript 
const SWIPE_THRESHOLD = 80;
const MAX_TRANSLATE = 120;
 
export function attachSwipeToDismiss(element, onDismiss) {

Building a swipe-to-dismiss gesture in JS

touch-events gesture-detection dom-manipulation
Intermediate 10 steps
javascript 
const { pool } = require('./db');
 
function withTransaction() {
  return async (req, res, next) => {

A per-request transaction middleware in Express

middleware database-transactions connection-pooling
Advanced 7 steps

Share this explainer

Here's the card — post it anywhere.

Streaming OpenAI tokens from a Next.js edge route — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code