rust 37 lines · 6 steps

Streaming lines from a gzip file in Rust

A newtype wraps a layered reader so you can iterate decompressed lines lazily without loading the whole file.

Explained by highlit
Intermediate iterators io-streaming decompression newtype-pattern error-handling
1use std::fs::File;
2use std::io::{self, BufRead, BufReader};
3use std::path::Path;
4 
5use flate2::read::MultiGzDecoder;
6 
7pub struct GzLines {
8 inner: io::Lines<BufReader<MultiGzDecoder<File>>>,
9}
10 
11impl GzLines {
12 pub fn open(path: impl AsRef<Path>) -> io::Result<Self> {
13 let file = File::open(path)?;
14 let decoder = MultiGzDecoder::new(file);
15 let reader = BufReader::with_capacity(64 * 1024, decoder);
16 Ok(Self { inner: reader.lines() })
17 }
18}
19 
20impl Iterator for GzLines {
21 type Item = io::Result<String>;
22 
23 fn next(&mut self) -> Option<Self::Item> {
24 self.inner.next()
25 }
26}
27 
28pub fn count_matching_lines(path: impl AsRef<Path>, needle: &str) -> io::Result<usize> {
29 let mut hits = 0;
30 for line in GzLines::open(path)? {
31 let line = line?;
32 if line.contains(needle) {
33 hits += 1;
34 }
35 }
36 Ok(hits)
37}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Wrapping a reader chain in a struct lets you expose a clean iterator while hiding the layered types.
  2. 2Delegating next to an inner iterator turns any buffered reader into a lazy, memory-efficient line stream.
  3. 3Propagating io::Result through both open and each line keeps errors visible instead of silently swallowed.

Related explainers

rust 
use serde::Deserialize;
 
#[derive(Debug, Deserialize)]
#[serde(untagged)]

Parsing flexible JSON shapes with serde

deserialization enums json
Intermediate 6 steps
ruby 
require "shellwords"
require "open3"
 
module Backup

Building safe shell commands in Ruby

shell-out subprocess command-injection
Intermediate 7 steps
rust 
use std::cmp::{Ordering, Reverse};
use std::collections::BinaryHeap;
use std::fs::File;
use std::io::{self, BufRead, BufReader, BufWriter, Lines, Write};

K-way merge of sorted logs in Rust

binary-heap k-way-merge streaming-io
Intermediate 8 steps
rust 
use axum::{
    extract::{Path, State},
    response::sse::{Event, KeepAlive, Sse},
};

Streaming import progress with SSE in Axum

server-sent-events streams watch-channel
Advanced 7 steps
rust 
use std::f64::consts::PI;
 
#[derive(Debug, Clone, Copy)]
pub struct LatLng {

Building geographic bounding boxes in Rust

geospatial value-types option
Intermediate 7 steps
rust 
use chrono::{Duration, NaiveDate};
 
#[derive(Debug)]
pub struct DateRange {

Parsing and iterating date ranges in Rust

error-handling iterators parsing
Intermediate 7 steps

Share this explainer

Here's the card — post it anywhere.

Streaming lines from a gzip file in Rust — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code