ruby 34 lines · 6 steps

Streaming log lines with a Ruby enumerator

An access-log parser yields severity entries lazily, so callers can iterate or aggregate without loading the whole file.

Explained by highlit
Intermediate enumerators lazy-iteration regex-matching structs file-streaming
1module LogParser
2 class AccessLogStreamer
3 SEVERITY_PATTERN = /\b(ERROR|WARN|FATAL)\b/
4 
5 def initialize(path)
6 @path = path
7 end
8 
9 def each_error
10 return enum_for(:each_error) unless block_given?
11 
12 File.open(@path, "r") do |file|
13 file.each_line.with_index(1) do |line, number|
14 line.chomp!
15 next unless (match = line.match(SEVERITY_PATTERN))
16 
17 yield Entry.new(number, match[1], line)
18 end
19 end
20 end
21 
22 def count_by_severity
23 each_error.each_with_object(Hash.new(0)) do |entry, tally|
24 tally[entry.severity] += 1
25 end
26 end
27 
28 Entry = Struct.new(:line_number, :severity, :message) do
29 def to_s
30 "#{line_number}: [#{severity}] #{message}"
31 end
32 end
33 end
34end
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Returning enum_for when no block is given lets one method serve both block and enumerator callers.
  2. 2Streaming a file line by line keeps memory flat regardless of log size.
  3. 3Building tallies on top of an enumerator reuses iteration logic instead of duplicating the file walk.

Related explainers

ruby 
class ToastBroadcaster
  include ActionView::RecordIdentifier
 
  def self.broadcast_to(user, message:, type: :notice)

How Turbo Stream toasts broadcast in Rails

turbo-streams service-object real-time
Intermediate 6 steps
ruby 
class TemplateInterpolator
  PLACEHOLDER = /\{\{\s*([\w.]+)\s*\}\}/
 
  def initialize(strict: false)

Interpolating templates with dotted keys in Ruby

regex string-interpolation hash-traversal
Intermediate 6 steps
ruby 
class KeyTransformer
  def self.camelize(data)
    new.camelize(data)
  end

Recursively camelizing nested Ruby data

recursion data-transformation pattern-matching
Intermediate 7 steps
ruby 
module Paginatable
  extend ActiveSupport::Concern
 
  private

A reusable pagination concern in Rails

pagination concerns http-headers
Intermediate 8 steps
ruby 
class Api::MessagesController < ApiController
  before_action :authenticate_api_key!
 
  rate_limit to: 100,

Layered API rate limiting in Rails

rate-limiting api-authentication throttling
Intermediate 9 steps
ruby 
class Order < ApplicationRecord
  include AASM
 
  belongs_to :warehouse

Modeling an order lifecycle with AASM in Rails

state machine guards callbacks
Intermediate 10 steps

Share this explainer

Here's the card — post it anywhere.

Streaming log lines with a Ruby enumerator — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code