go 35 lines · 8 steps

Streaming NDJSON progress with Gin

A Gin handler runs an export in a goroutine and streams progress updates to the client as newline-delimited JSON.

Explained by highlit
Gin Advanced streaming goroutines channels context-cancellation server-sent-updates
1func (h *ExportHandler) StreamExport(c *gin.Context) {
2 datasetID := c.Param("id")
3 
4 ctx := c.Request.Context()
5 progress := make(chan ExportProgress, 8)
6 
7 go func() {
8 defer close(progress)
9 if err := h.exporter.Run(ctx, datasetID, progress); err != nil {
10 progress <- ExportProgress{Stage: "failed", Error: err.Error()}
11 }
12 }()
13 
14 c.Header("Content-Type", "application/x-ndjson")
15 c.Header("Cache-Control", "no-cache")
16 c.Header("X-Accel-Buffering", "no")
17 
18 enc := json.NewEncoder(c.Writer)
19 
20 c.Stream(func(w io.Writer) bool {
21 select {
22 case update, ok := <-progress:
23 if !ok {
24 return false
25 }
26 if err := enc.Encode(update); err != nil {
27 return false
28 }
29 c.Writer.Flush()
30 return update.Stage != "failed"
31 case <-ctx.Done():
32 return false
33 }
34 })
35}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Running work in a goroutine and reporting through a buffered channel decouples production of progress from its delivery to the client.
  2. 2Closing the channel with defer gives the streaming loop a clean, unambiguous end-of-stream signal.
  3. 3Selecting on both the progress channel and ctx.Done lets a stream stop promptly when the client disconnects.

Related explainers

go 
package streaming
 
import (
	"bufio"

Streaming NDJSON logs over HTTP in Go

http-streaming channels select
Advanced 10 steps
go 
package api
 
import (
	"crypto/sha256"

ETag conditional requests in Gin

http-caching etag conditional-requests
Intermediate 6 steps
go 
func (w *Watcher) resetDebounce(d time.Duration) {
	if !w.timer.Stop() {
		select {
		case <-w.timer.C:

Debouncing a stream of events in Go

debounce timers channels
Advanced 7 steps
python 
import random
from typing import Iterator, List
 
 

How reservoir sampling picks k items

reservoir-sampling streaming randomness
Intermediate 5 steps
go 
package logging
 
import (
	"context"

Deduplicating log attributes in Go's slog

decorator-pattern structured-logging immutability
Intermediate 8 steps
go 
package middleware
 
import (
	"net/http"

Per-plan export limits in Gin middleware

middleware rate-limiting authorization
Intermediate 7 steps

Share this explainer

Here's the card — post it anywhere.

Streaming NDJSON progress with Gin — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code