php 59 lines · 8 steps

Streaming user activity with PHP generators

A generator paginates the users table and lazily yields each user enriched with their activity, keeping memory flat regardless of table size.

Explained by highlit
Intermediate generators pagination lazy-evaluation batching pdo
1final class UserActivityStream
2{
3 public function __construct(
4 private readonly \PDO $db,
5 private readonly int $batchSize = 500
6 ) {
7 }
8 
9 public function forActiveUsers(): \Generator
10 {
11 $offset = 0;
12 
13 do {
14 $users = $this->fetchUserBatch($offset, $this->batchSize);
15 $ids = array_column($users, 'id');
16 
17 if ($ids === []) {
18 return;
19 }
20 
21 $activityByUser = $this->fetchActivityFor($ids);
22 
23 foreach ($users as $user) {
24 $user['activity'] = $activityByUser[$user['id']] ?? [];
25 yield $user;
26 }
27 
28 $offset += $this->batchSize;
29 } while (count($users) === $this->batchSize);
30 }
31 
32 private function fetchUserBatch(int $offset, int $limit): array
33 {
34 $stmt = $this->db->prepare(
35 'SELECT id, email FROM users WHERE active = 1 ORDER BY id LIMIT :limit OFFSET :offset'
36 );
37 $stmt->bindValue(':limit', $limit, \PDO::PARAM_INT);
38 $stmt->bindValue(':offset', $offset, \PDO::PARAM_INT);
39 $stmt->execute();
40 
41 return $stmt->fetchAll(\PDO::FETCH_ASSOC);
42 }
43 
44 private function fetchActivityFor(array $userIds): array
45 {
46 $placeholders = implode(',', array_fill(0, count($userIds), '?'));
47 $stmt = $this->db->prepare(
48 "SELECT user_id, action, created_at FROM activity_log WHERE user_id IN ($placeholders)"
49 );
50 $stmt->execute($userIds);
51 
52 $grouped = [];
53 foreach ($stmt->fetchAll(\PDO::FETCH_ASSOC) as $row) {
54 $grouped[$row['user_id']][] = $row;
55 }
56 
57 return $grouped;
58 }
59}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Generators let you process arbitrarily large result sets while holding only one batch in memory at a time.
  2. 2Fetching related data once per batch and grouping it in a hash map avoids the N+1 query trap.
  3. 3Keyset or offset pagination combined with yielding turns a huge query into a streamable pipeline.

Related explainers

php 
<?php
 
namespace App\Validation;
 

Building a reusable address form validator in PHP

validation error-accumulation regex
Intermediate 9 steps
php 
<?php
 
namespace App\Http\Controllers;
 

A cached autocomplete endpoint in Laravel

caching validation query-ranking
Intermediate 8 steps
javascript 
const express = require('express');
const router = express.Router();
const db = require('../db');
 

Building a paginated orders page in Express

pagination routing sql-queries
Intermediate 7 steps
php 
<?php
 
declare(strict_types=1);
 

Normalizing human names in PHP

unicode text-normalization transliteration
Intermediate 8 steps
ruby 
class BackfillUsersFullName < ActiveRecord::Migration[7.1]
  disable_ddl_transaction!
 
  BATCH_SIZE = 5_000

How to backfill a column safely in Rails

migrations batching backfill
Intermediate 7 steps
php 
<?php
 
namespace App\Http\Controllers;
 

Handling Stripe webhooks in Laravel

webhooks signature-verification dependency-injection
Intermediate 7 steps

Share this explainer

Here's the card — post it anywhere.

Streaming user activity with PHP generators — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code