java 41 lines · 6 steps

Three ways to deduplicate a list by key in Java

A small utility class showing distinct-by-key, latest-per-key, and a custom collector that all dedupe streams without overriding equals.

Explained by highlit
Intermediate streams deduplication collectors generics functional-interfaces
1import java.util.ArrayList;
2import java.util.Comparator;
3import java.util.HashSet;
4import java.util.List;
5import java.util.Set;
6import java.util.function.Function;
7import java.util.stream.Collector;
8import java.util.stream.Collectors;
9 
10public final class Deduplication {
11 
12 public static <T, K> List<T> distinctByKey(List<T> items, Function<? super T, ? extends K> keyExtractor) {
13 Set<K> seen = new HashSet<>();
14 return items.stream()
15 .filter(item -> seen.add(keyExtractor.apply(item)))
16 .collect(Collectors.toList());
17 }
18 
19 public static <T, K> List<T> latestPerKey(
20 List<T> items,
21 Function<? super T, ? extends K> keyExtractor,
22 Comparator<? super T> recency) {
23 return new ArrayList<>(items.stream()
24 .collect(Collectors.toMap(
25 keyExtractor,
26 Function.identity(),
27 (existing, candidate) -> recency.compare(candidate, existing) >= 0 ? candidate : existing))
28 .values());
29 }
30 
31 public static <T> Collector<T, ?, List<T>> distinctByKeyCollector(Function<? super T, ?> keyExtractor) {
32 return Collector.of(
33 () -> new java.util.LinkedHashMap<Object, T>(),
34 (map, item) -> map.putIfAbsent(keyExtractor.apply(item), item),
35 (left, right) -> {
36 right.forEach(left::putIfAbsent);
37 return left;
38 },
39 map -> new ArrayList<>(map.values()));
40 }
41}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1A HashSet's add return value doubles as a cheap first-seen test inside a stream filter.
  2. 2Collectors.toMap with a merge function lets you resolve key collisions by any rule you choose, like recency.
  3. 3A custom Collector can preserve insertion order while deduplicating, something toSet and toMap can't both guarantee.

Related explainers

java 
@Component
@Converter
public class EncryptedStringConverter implements AttributeConverter<String, String> {
 

Transparent column encryption in Spring & JPA

encryption aes-gcm jpa-converter
Advanced 10 steps
rust 
use axum::{
    extract::{Path, State},
    response::sse::{Event, KeepAlive, Sse},
};

Streaming import progress with SSE in Axum

server-sent-events streams watch-channel
Advanced 7 steps
java 
package com.acme.billing.config;
 
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.ConfigurationProperties;

Feature-flagged beans with Spring @ConditionalOnProperty

feature-flags conditional-beans strategy-pattern
Intermediate 5 steps
java 
public static Map<String, String> parseCookieHeader(String header) {
    Map<String, String> cookies = new LinkedHashMap<>();
    if (header == null || header.isBlank()) {
        return cookies;

Parsing an HTTP Cookie header in Java

string-parsing http url-decoding
Intermediate 6 steps
go 
package logging
 
import (
	"context"

Deduplicating log attributes in Go's slog

decorator-pattern structured-logging immutability
Intermediate 8 steps
java 
public class TimedSocketReader {
 
    private static final int READ_TIMEOUT_MS = 5_000;
    private static final int CONNECT_TIMEOUT_MS = 3_000;

Reading a socket with connect and read timeouts

sockets timeouts io
Intermediate 8 steps

Share this explainer

Here's the card — post it anywhere.

Three ways to deduplicate a list by key in Java — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code