go 32 lines · 5 steps

Generic deduplication in Go

Two generic functions strip duplicates from a slice while preserving order, using a set tracked by an empty-struct map.

Explained by highlit
Intermediate generics deduplication maps slices order-preserving
1package collection
2 
3func Deduplicate[T comparable](items []T) []T {
4 seen := make(map[T]struct{}, len(items))
5 result := make([]T, 0, len(items))
6 
7 for _, item := range items {
8 if _, ok := seen[item]; ok {
9 continue
10 }
11 seen[item] = struct{}{}
12 result = append(result, item)
13 }
14 
15 return result
16}
17 
18func DeduplicateBy[T any, K comparable](items []T, key func(T) K) []T {
19 seen := make(map[K]struct{}, len(items))
20 result := make([]T, 0, len(items))
21 
22 for _, item := range items {
23 k := key(item)
24 if _, ok := seen[k]; ok {
25 continue
26 }
27 seen[k] = struct{}{}
28 result = append(result, item)
29 }
30 
31 return result
32}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1A map with struct{} values is Go's idiomatic zero-memory set for membership checks.
  2. 2Pre-sizing maps and slices with the input length avoids repeated reallocation during the pass.
  3. 3Adding a key function turns an exact-equality dedup into one keyed on any derived comparable value.

Related explainers

go 
package handlers
 
import (
	"net/http"

Serving embedded static meta files in Gin

embedding static assets http caching
Intermediate 6 steps
go 
package config
 
import (
	"fmt"

A thread-safe config singleton in Go

singleton concurrency environment-variables
Intermediate 7 steps
typescript 
type CsvColumn<T> = {
  header: string;
  value: (row: T) => string | number | boolean | null | undefined;
};

Building a type-safe CSV writer in TypeScript

generics serialization escaping
Intermediate 7 steps
go 
package scheduler
 
import (
	"container/heap"

A priority job queue with Go's container/heap

priority-queue heap interfaces
Intermediate 9 steps
go 
func UploadChunk(c *gin.Context) {
	uploadID := c.Param("uploadID")
	if !validUploadID.MatchString(uploadID) {
		c.JSON(http.StatusBadRequest, gin.H{"error": "invalid upload id"})

Resumable chunked uploads in Gin

file-upload content-range streaming
Advanced 9 steps
go 
func (h *ArticleHandler) Create(c *gin.Context) {
	var req struct {
		Title   string `json:"title" binding:"required"`
		Body    string `json:"body" binding:"required"`

Handling a POST request in Gin

request binding validation http status codes
Intermediate 7 steps

Share this explainer

Here's the card — post it anywhere.

Generic deduplication in Go — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code