ruby 51 lines · 8 steps

Deduplicating a mailing list by canonical email

A service object collapses duplicate subscribers by normalizing their emails and keeping only the most recent entry per address.

Explained by highlit
Intermediate deduplication normalization service object hash grouping
1class MailingListDeduplicator
2 GMAIL_DOMAINS = %w[gmail.com googlemail.com].freeze
3 
4 def initialize(subscribers)
5 @subscribers = subscribers
6 end
7 
8 def call
9 seen = {}
10 
11 @subscribers.each do |subscriber|
12 key = normalize(subscriber.fetch(:email))
13 next if key.nil?
14 
15 existing = seen[key]
16 if existing.nil? || newer?(subscriber, existing)
17 seen[key] = subscriber
18 end
19 end
20 
21 seen.values
22 end
23 
24 private
25 
26 def normalize(raw)
27 return nil if raw.nil?
28 
29 address = raw.strip.downcase
30 local, domain = address.split("@", 2)
31 return nil if local.nil? || domain.nil? || local.empty? || domain.empty?
32 
33 local = local.split("+", 2).first
34 
35 if GMAIL_DOMAINS.include?(domain)
36 local = local.delete(".")
37 domain = "gmail.com"
38 end
39 
40 "#{local}@#{domain}"
41 end
42 
43 def newer?(candidate, existing)
44 candidate_at = candidate[:subscribed_at]
45 existing_at = existing[:subscribed_at]
46 return false if candidate_at.nil?
47 return true if existing_at.nil?
48 
49 candidate_at > existing_at
50 end
51end
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Normalizing keys before comparison lets you treat semantically-equal values as identical.
  2. 2Keeping a hash of seen keys turns deduplication into a single linear pass.
  3. 3Encoding domain-specific rules like Gmail's dot and plus aliases makes matching match real-world identity.

Related explainers

ruby 
class Admin::AccountsController < Admin::BaseController
  before_action :set_account, only: :destroy
 
  def destroy

A guarded destroy action in Rails

controllers callbacks transactions
Intermediate 8 steps
ruby 
class ProgressBar
  BAR_WIDTH = 40
 
  def initialize(total, io: $stdout)

Building a terminal progress bar in Ruby

state tracking terminal output time estimation
Intermediate 8 steps
ruby 
class Product < ApplicationRecord
  scope :search, ->(query) {
    return all if query.blank?
 

Building a multi-term search scope in Rails

arel scopes full-text search
Advanced 7 steps
ruby 
module Rack
  class MaintenanceMode
    RETRY_AFTER = 3600
 

A Rack maintenance-mode middleware in Rails

middleware rack http-status
Intermediate 8 steps
ruby 
class ImageNormalizer
  ORIENTATION_TRANSFORMS = {
    1 => ->(img) {},
    2 => ->(img) { img.flop },

Correcting EXIF orientation in Ruby

lookup-table lambdas image-processing
Intermediate 7 steps
javascript 
function parseHexColor(hex) {
  const cleaned = hex.trim().replace(/^#/, '');
 
  const expand = (short) =>

Parsing hex colors into RGBA channels

parsing bitwise regex
Intermediate 7 steps

Share this explainer

Here's the card — post it anywhere.

Deduplicating a mailing list by canonical email — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code