typescript 64 lines · 9 steps

Streaming JSON parsing with a depth counter

A ReadableStream transform that emits complete JSON values as they arrive, tracking brace depth to find object boundaries.

Explained by highlit
Advanced streams parsing state-machine backpressure generics
1type JsonValue = Record<string, unknown> | unknown[];
2 
3export function parseJsonStream<T = JsonValue>(
4 stream: ReadableStream<Uint8Array>,
5): ReadableStream<T> {
6 const decoder = new TextDecoder();
7 let buffer = "";
8 
9 return new ReadableStream<T>({
10 async start(controller) {
11 const reader = stream.getReader();
12 try {
13 while (true) {
14 const { done, value } = await reader.read();
15 if (done) break;
16 buffer += decoder.decode(value, { stream: true });
17 buffer = drain(buffer, controller);
18 }
19 buffer += decoder.decode();
20 drain(buffer, controller, true);
21 controller.close();
22 } catch (err) {
23 controller.error(err);
24 } finally {
25 reader.releaseLock();
26 }
27 },
28 });
29 
30 function drain(input: string, controller: ReadableStreamDefaultController<T>, final = false): string {
31 let depth = 0;
32 let inString = false;
33 let escaped = false;
34 let start = -1;
35 
36 for (let i = 0; i < input.length; i++) {
37 const ch = input[i];
38 if (inString) {
39 if (escaped) escaped = false;
40 else if (ch === "\\") escaped = true;
41 else if (ch === '"') inString = false;
42 continue;
43 }
44 if (ch === '"') inString = true;
45 else if (ch === "{" || ch === "[") {
46 if (depth === 0) start = i;
47 depth++;
48 } else if (ch === "}" || ch === "]") {
49 depth--;
50 if (depth === 0 && start !== -1) {
51 controller.enqueue(JSON.parse(input.slice(start, i + 1)) as T);
52 input = input.slice(i + 1);
53 i = -1;
54 start = -1;
55 }
56 }
57 }
58 
59 if (final && input.trim().length > 0) {
60 throw new SyntaxError("Trailing incomplete JSON in stream");
61 }
62 return input;
63 }
64}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Tracking nesting depth lets you split a byte stream into complete values without buffering everything.
  2. 2String and escape state must be tracked separately so braces inside quoted strings don't corrupt the count.
  3. 3Passing { stream: true } to TextDecoder handles multi-byte characters split across chunk boundaries.

Related explainers

typescript 
import { registerLocaleData } from '@angular/common';
import localeFr from '@angular/common/locales/fr';
import localeFrExtra from '@angular/common/locales/extra/fr';
import localeDe from '@angular/common/locales/de';

Locale-aware bootstrapping in Angular

i18n localization dependency-injection
Intermediate 8 steps
typescript 
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import * as Joi from 'joi';
 

Validating env config at boot in NestJS

configuration schema-validation environment-variables
Intermediate 8 steps
ruby 
class UserAgentParser
  BROWSERS = [
    [/Edg\/([\d.]+)/, "Edge"],
    [/OPR\/([\d.]+)/, "Opera"],

Parsing user-agent strings in Ruby

regex pattern-matching lookup-tables
Intermediate 8 steps
javascript 
function evaluate(expression) {
  const tokens = tokenize(expression);
  let pos = 0;
 

Building a recursive descent calculator

parsing recursion operator-precedence
Intermediate 8 steps
typescript 
import { Inject, Injectable, Logger } from '@nestjs/common';
import { CACHE_MANAGER } from '@nestjs/cache-manager';
import { Cache } from 'cache-manager';
import { InjectRepository } from '@nestjs/typeorm';

A cache-aside country lookup in NestJS

cache-aside dependency-injection batch-lookup
Intermediate 8 steps
rust 
use axum::{
    extract::{Path, State},
    response::sse::{Event, KeepAlive, Sse},
};

Streaming import progress with SSE in Axum

server-sent-events streams watch-channel
Advanced 7 steps

Share this explainer

Here's the card — post it anywhere.

Streaming JSON parsing with a depth counter — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code