ruby 34 lines · 6 steps

Streaming log lines with a Ruby enumerator

An access-log parser yields severity entries lazily, so callers can iterate or aggregate without loading the whole file.

Explained by highlit
Intermediate enumerators lazy-iteration regex-matching structs file-streaming
1module LogParser
2 class AccessLogStreamer
3 SEVERITY_PATTERN = /\b(ERROR|WARN|FATAL)\b/
4 
5 def initialize(path)
6 @path = path
7 end
8 
9 def each_error
10 return enum_for(:each_error) unless block_given?
11 
12 File.open(@path, "r") do |file|
13 file.each_line.with_index(1) do |line, number|
14 line.chomp!
15 next unless (match = line.match(SEVERITY_PATTERN))
16 
17 yield Entry.new(number, match[1], line)
18 end
19 end
20 end
21 
22 def count_by_severity
23 each_error.each_with_object(Hash.new(0)) do |entry, tally|
24 tally[entry.severity] += 1
25 end
26 end
27 
28 Entry = Struct.new(:line_number, :severity, :message) do
29 def to_s
30 "#{line_number}: [#{severity}] #{message}"
31 end
32 end
33 end
34end
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Returning enum_for when no block is given lets one method serve both block and enumerator callers.
  2. 2Streaming a file line by line keeps memory flat regardless of log size.
  3. 3Building tallies on top of an enumerator reuses iteration logic instead of duplicating the file walk.

Related explainers

ruby 
require "shellwords"
require "open3"
 
module Backup

Building safe shell commands in Ruby

shell-out subprocess command-injection
Intermediate 7 steps
ruby 
class UserAgentParser
  BROWSERS = [
    [/Edg\/([\d.]+)/, "Edge"],
    [/OPR\/([\d.]+)/, "Opera"],

Parsing user-agent strings in Ruby

regex pattern-matching lookup-tables
Intermediate 8 steps
ruby 
class LogAggregator
  BUCKET_FORMAT = "%Y-%m-%dT%H:%M"
 
  def initialize(entries)

Bucketing log entries by the minute in Ruby

aggregation hashing enumerable
Intermediate 5 steps
ruby 
class WeeklySignupsReport
  DEFAULT_WEEKS = 12
 
  def initialize(weeks: DEFAULT_WEEKS, source: User.all)

Building a weekly signups report in Rails

service object aggregation group by
Intermediate 7 steps
ruby 
class ApplicationController < ActionController::Base
  EXPERIMENTS = {
    checkout_button_color: %w[control blue green],
    onboarding_flow: %w[control streamlined]

How A/B test cohorts are assigned in Rails

a-b-testing cookies hashing
Intermediate 8 steps
ruby 
class SnippetHighlighter
  CONTEXT_RADIUS = 60
  MAX_TERMS = 8
 

Building search-result snippets in Ruby

regular-expressions text-processing search
Intermediate 9 steps

Share this explainer

Here's the card — post it anywhere.

Streaming log lines with a Ruby enumerator — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code