go 32 lines · 5 steps

Generic deduplication in Go

Two generic functions strip duplicates from a slice while preserving order, using a set tracked by an empty-struct map.

Explained by highlit
Intermediate generics deduplication maps slices order-preserving
1package collection
2 
3func Deduplicate[T comparable](items []T) []T {
4 seen := make(map[T]struct{}, len(items))
5 result := make([]T, 0, len(items))
6 
7 for _, item := range items {
8 if _, ok := seen[item]; ok {
9 continue
10 }
11 seen[item] = struct{}{}
12 result = append(result, item)
13 }
14 
15 return result
16}
17 
18func DeduplicateBy[T any, K comparable](items []T, key func(T) K) []T {
19 seen := make(map[K]struct{}, len(items))
20 result := make([]T, 0, len(items))
21 
22 for _, item := range items {
23 k := key(item)
24 if _, ok := seen[k]; ok {
25 continue
26 }
27 seen[k] = struct{}{}
28 result = append(result, item)
29 }
30 
31 return result
32}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1A map with struct{} values is Go's idiomatic zero-memory set for membership checks.
  2. 2Pre-sizing maps and slices with the input length avoids repeated reallocation during the pass.
  3. 3Adding a key function turns an exact-equality dedup into one keyed on any derived comparable value.

Related explainers

go 
package main
 
import (
	"errors"

Parsing and validating CLI flags in Go

cli-parsing validation error-handling
Intermediate 8 steps
go 
package cache
 
import (
	"container/list"

Building a generic LRU cache in Go

lru-cache generics linked-list
Intermediate 8 steps
go 
package model
 
import (
	"encoding/json"

Custom JSON marshaling in Go

json serialization interfaces
Intermediate 5 steps
rust 
use std::collections::HashMap;
 
pub struct Memoizer<K, V, F> {
    cache: HashMap<K, V>,

A generic memoizer in Rust

memoization generics caching
Intermediate 6 steps
go 
func (h *TransactionHandler) ExportCSV(c *gin.Context) {
	ctx := c.Request.Context()
	filters := parseTransactionFilters(c)
 

Streaming a CSV export in Gin

streaming csv-export database-cursor
Intermediate 8 steps
typescript 
function throttle<T extends (...args: any[]) => void>(
  fn: T,
  limit: number
): (...args: Parameters<T>) => void {

Building a trailing-edge throttle in TypeScript

throttling closures generics
Intermediate 7 steps

Share this explainer

Here's the card — post it anywhere.

Generic deduplication in Go — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code