ruby 34 lines · 6 steps

Streaming log lines with a Ruby enumerator

An access-log parser yields severity entries lazily, so callers can iterate or aggregate without loading the whole file.

Explained by highlit
Intermediate enumerators lazy-iteration regex-matching structs file-streaming
1module LogParser
2 class AccessLogStreamer
3 SEVERITY_PATTERN = /\b(ERROR|WARN|FATAL)\b/
4 
5 def initialize(path)
6 @path = path
7 end
8 
9 def each_error
10 return enum_for(:each_error) unless block_given?
11 
12 File.open(@path, "r") do |file|
13 file.each_line.with_index(1) do |line, number|
14 line.chomp!
15 next unless (match = line.match(SEVERITY_PATTERN))
16 
17 yield Entry.new(number, match[1], line)
18 end
19 end
20 end
21 
22 def count_by_severity
23 each_error.each_with_object(Hash.new(0)) do |entry, tally|
24 tally[entry.severity] += 1
25 end
26 end
27 
28 Entry = Struct.new(:line_number, :severity, :message) do
29 def to_s
30 "#{line_number}: [#{severity}] #{message}"
31 end
32 end
33 end
34end
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Returning enum_for when no block is given lets one method serve both block and enumerator callers.
  2. 2Streaming a file line by line keeps memory flat regardless of log size.
  3. 3Building tallies on top of an enumerator reuses iteration logic instead of duplicating the file walk.

Related explainers

ruby 
require "csv"
 
class SalesReport
  def initialize(path)

Aggregating CSV sales data in Ruby

data-aggregation memoization group_by
Intermediate 6 steps
ruby 
module DurationFormatter
  UNITS = [
    ['week', 604_800],
    ['day', 86_400],

Turning seconds into human-readable durations in Ruby

greedy-decomposition modular-arithmetic formatting
Intermediate 7 steps
ruby 
class Comment < ApplicationRecord
  belongs_to :post
  belongs_to :author, class_name: "User"
 

Live-updating comments with Turbo in Rails

turbo-streams callbacks associations
Intermediate 8 steps
ruby 
require 'json'
require 'set'
 
class SensitiveScrubber

Recursively scrubbing secrets from JSON

recursion data-masking pattern-matching
Intermediate 7 steps
ruby 
class ReportBatcher
  BATCH_SIZE = 500
 
  def initialize(account)

Batching monthly email summaries in Rails

batching service-object background-jobs
Intermediate 7 steps
ruby 
class Comment < ApplicationRecord
  belongs_to :commentable, polymorphic: true, counter_cache: true
  belongs_to :author, class_name: "User"
 

How polymorphic comments work in Rails

polymorphic-association concerns counter-cache
Intermediate 8 steps

Share this explainer

Here's the card — post it anywhere.

Streaming log lines with a Ruby enumerator — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code