ruby 32 lines · 5 steps

Batching bulk inserts in Rails

A BulkImporter chunks records into fixed-size batches and inserts each with a single deduplicated SQL statement.

Explained by highlit

Rails Intermediate batching bulk-insert deduplication data-normalization

1class BulkImporter
BATCH_SIZE = 500
3 
def initialize(records)
  @records = records
end
7 
def import!
  inserted_ids = []
10 
  @records.each_slice(BATCH_SIZE) do |batch|
    rows = batch.map do |record|
      {
        email: record.fetch(:email).downcase.strip,
        name: record[:name],
        created_at: Time.current,
        updated_at: Time.current
      }
    end
20 
    result = User.insert_all(
      rows,
      unique_by: :email,
      returning: %w[id]
    )
26 
    inserted_ids.concat(result.rows.flatten)
  end
29 
  inserted_ids
end
32end

01 / 01

STEP 01

Walkthrough

Space play ←→ step click any line

Three takeaways

1Slicing records into fixed batches keeps memory and SQL statement size bounded on large imports.
2insert_all bypasses model callbacks, so timestamps and normalization must be set manually.
3unique_by lets the database skip duplicates atomically instead of checking in Ruby.

Related explainers

ruby

require "shellwords"
require "open3"
 
module Backup

Building safe shell commands in Ruby

shell-out subprocess command-injection

Intermediate 7 steps

ruby

class UserAgentParser
  BROWSERS = [
    [/Edg\/([\d.]+)/, "Edge"],
    [/OPR\/([\d.]+)/, "Opera"],

Parsing user-agent strings in Ruby

regex pattern-matching lookup-tables

Intermediate 8 steps

ruby

class LogAggregator
  BUCKET_FORMAT = "%Y-%m-%dT%H:%M"
 
  def initialize(entries)

Bucketing log entries by the minute in Ruby

aggregation hashing enumerable

Intermediate 5 steps

ruby

class WeeklySignupsReport
  DEFAULT_WEEKS = 12
 
  def initialize(weeks: DEFAULT_WEEKS, source: User.all)

Building a weekly signups report in Rails

service object aggregation group by

Intermediate 7 steps

package logging
 
import (
	"context"

Deduplicating log attributes in Go's slog

decorator-pattern structured-logging immutability

Intermediate 8 steps

ruby

class ApplicationController < ActionController::Base
  EXPERIMENTS = {
    checkout_button_color: %w[control blue green],
    onboarding_flow: %w[control streamlined]