rust 29 lines · 6 steps

Deduplicating and sorting words in Rust

A BTreeSet collects unique, cleaned words from a file and writes them back out in sorted order.

Explained by highlit
Intermediate file-io btreeset string-processing error-handling deduplication
1use std::collections::BTreeSet;
2use std::fs::File;
3use std::io::{self, BufRead, BufReader, BufWriter, Write};
4use std::path::Path;
5 
6pub fn dedup_sort_words(input: &Path, output: &Path) -> io::Result<usize> {
7 let reader = BufReader::new(File::open(input)?);
8 let mut words: BTreeSet<String> = BTreeSet::new();
9 
10 for line in reader.lines() {
11 let line = line?;
12 for token in line.split_whitespace() {
13 let cleaned: String = token
14 .trim_matches(|c: char| !c.is_alphanumeric())
15 .to_lowercase();
16 if !cleaned.is_empty() {
17 words.insert(cleaned);
18 }
19 }
20 }
21 
22 let mut writer = BufWriter::new(File::create(output)?);
23 for word in &words {
24 writeln!(writer, "{word}")?;
25 }
26 writer.flush()?;
27 
28 Ok(words.len())
29}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1A BTreeSet gives you deduplication and sorted order in one data structure, for free.
  2. 2The `?` operator lets each fallible IO call bubble errors up cleanly to the caller.
  3. 3Buffered readers and writers avoid a syscall per line, which matters for large files.

Related explainers

rust 
use axum::{extract::{Path, State}, http::StatusCode, Json};
use dashmap::DashMap;
use serde::Serialize;
use std::sync::Arc;

Request coalescing in an Axum handler

caching concurrency request-coalescing
Advanced 8 steps
ruby 
class TemplateInterpolator
  PLACEHOLDER = /\{\{\s*([\w.]+)\s*\}\}/
 
  def initialize(strict: false)

Interpolating templates with dotted keys in Ruby

regex string-interpolation hash-traversal
Intermediate 6 steps
go 
package config
 
import (
	"fmt"

A thread-safe config singleton in Go

singleton concurrency environment-variables
Intermediate 7 steps
rust 
use std::net::Ipv4Addr;
use std::str::FromStr;
 
#[derive(Debug, Clone, Copy)]

Parsing and matching IPv4 CIDR ranges in Rust

bitwise-operations parsing error-handling
Intermediate 8 steps
rust 
use std::sync::OnceLock;
 
static CRC_TABLE: OnceLock<[u32; 256]> = OnceLock::new();
 

Table-driven CRC32 with lazy init in Rust

checksums lazy-initialization lookup-tables
Intermediate 8 steps
typescript 
import { parsePhoneNumberFromString, CountryCode } from 'libphonenumber-js';
 
export interface NormalizedPhone {
  e164: string;

Normalizing phone numbers to E.164 in TypeScript

validation normalization error-handling
Intermediate 7 steps

Share this explainer

Here's the card — post it anywhere.

Deduplicating and sorting words in Rust — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code