ruby 51 lines · 8 steps

Deduplicating a mailing list by canonical email

A service object collapses duplicate subscribers by normalizing their emails and keeping only the most recent entry per address.

Explained by highlit
Intermediate deduplication normalization service object hash grouping
1class MailingListDeduplicator
2 GMAIL_DOMAINS = %w[gmail.com googlemail.com].freeze
3 
4 def initialize(subscribers)
5 @subscribers = subscribers
6 end
7 
8 def call
9 seen = {}
10 
11 @subscribers.each do |subscriber|
12 key = normalize(subscriber.fetch(:email))
13 next if key.nil?
14 
15 existing = seen[key]
16 if existing.nil? || newer?(subscriber, existing)
17 seen[key] = subscriber
18 end
19 end
20 
21 seen.values
22 end
23 
24 private
25 
26 def normalize(raw)
27 return nil if raw.nil?
28 
29 address = raw.strip.downcase
30 local, domain = address.split("@", 2)
31 return nil if local.nil? || domain.nil? || local.empty? || domain.empty?
32 
33 local = local.split("+", 2).first
34 
35 if GMAIL_DOMAINS.include?(domain)
36 local = local.delete(".")
37 domain = "gmail.com"
38 end
39 
40 "#{local}@#{domain}"
41 end
42 
43 def newer?(candidate, existing)
44 candidate_at = candidate[:subscribed_at]
45 existing_at = existing[:subscribed_at]
46 return false if candidate_at.nil?
47 return true if existing_at.nil?
48 
49 candidate_at > existing_at
50 end
51end
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Normalizing keys before comparison lets you treat semantically-equal values as identical.
  2. 2Keeping a hash of seen keys turns deduplication into a single linear pass.
  3. 3Encoding domain-specific rules like Gmail's dot and plus aliases makes matching match real-world identity.

Related explainers

php 
<?php
 
class NameParser
{

Parsing a full name into components in PHP

string-parsing arrays normalization
Intermediate 8 steps
ruby 
require "shellwords"
require "open3"
 
module Backup

Building safe shell commands in Ruby

shell-out subprocess command-injection
Intermediate 7 steps
ruby 
class UserAgentParser
  BROWSERS = [
    [/Edg\/([\d.]+)/, "Edge"],
    [/OPR\/([\d.]+)/, "Opera"],

Parsing user-agent strings in Ruby

regex pattern-matching lookup-tables
Intermediate 8 steps
ruby 
class LogAggregator
  BUCKET_FORMAT = "%Y-%m-%dT%H:%M"
 
  def initialize(entries)

Bucketing log entries by the minute in Ruby

aggregation hashing enumerable
Intermediate 5 steps
ruby 
class WeeklySignupsReport
  DEFAULT_WEEKS = 12
 
  def initialize(weeks: DEFAULT_WEEKS, source: User.all)

Building a weekly signups report in Rails

service object aggregation group by
Intermediate 7 steps
go 
package logging
 
import (
	"context"

Deduplicating log attributes in Go's slog

decorator-pattern structured-logging immutability
Intermediate 8 steps

Share this explainer

Here's the card — post it anywhere.

Deduplicating a mailing list by canonical email — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code