php 55 lines · 9 steps

Streaming CSV imports with Laravel batches

An action streams a large CSV lazily, splits it into chunked jobs, and dispatches them as a monitored batch.

Explained by highlit
Laravel Advanced lazy-collections job-batching streaming queues memory-efficiency
1<?php
2 
3namespace App\Actions\Imports;
4 
5use App\Jobs\ImportProductChunk;
6use App\Models\Import;
7use App\Notifications\ImportCompleted;
8use Illuminate\Bus\Batch;
9use Illuminate\Support\Facades\Bus;
10use Illuminate\Support\Facades\Storage;
11use Illuminate\Support\LazyCollection;
12use Throwable;
13 
14class StartProductImport
15{
16 public function handle(Import $import): Batch
17 {
18 $jobs = LazyCollection::make(function () use ($import) {
19 $handle = Storage::readStream($import->path);
20 $header = fgetcsv($handle);
21 
22 while (($row = fgetcsv($handle)) !== false) {
23 yield array_combine($header, $row);
24 }
25 
26 fclose($handle);
27 })
28 ->chunk(500)
29 ->map(fn ($rows) => new ImportProductChunk($import->id, $rows->all()));
30 
31 return Bus::batch($jobs)
32 ->name("Product import #{$import->id}")
33 ->allowFailures()
34 ->progress(function (Batch $batch) use ($import) {
35 $import->update(['progress' => $batch->progress()]);
36 })
37 ->then(function (Batch $batch) use ($import) {
38 $import->update([
39 'status' => 'completed',
40 'progress' => 100,
41 'completed_at' => now(),
42 ]);
43 
44 $import->user->notify(new ImportCompleted($import));
45 })
46 ->catch(function (Batch $batch, Throwable $e) use ($import) {
47 $import->update([
48 'status' => 'failed',
49 'error' => $e->getMessage(),
50 ]);
51 })
52 ->onQueue('imports')
53 ->dispatch();
54 }
55}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1LazyCollection with a generator keeps a huge file out of memory by yielding one row at a time.
  2. 2Chunking rows into jobs turns a single unbounded import into parallelizable, retryable units of work.
  3. 3Bus batches give you lifecycle hooks — progress, then, and catch — to track and finalize long-running work.

Related explainers

php 
<?php
 
namespace App\Http\Controllers;
 

Server-Sent Events in Laravel

server-sent-events streaming long-polling
Advanced 9 steps
php 
namespace App\Providers;
 
use Illuminate\Database\Eloquent\Model;
use Illuminate\Support\Facades\Log;

Catching N+1 queries with Eloquent strict mode in Laravel

n+1 queries strict mode eager loading
Intermediate 7 steps
php 
<?php
 
namespace App\Support;
 

Recursively finding files with SPL iterators in PHP

recursion iterators filesystem
Intermediate 7 steps
python 
from contextlib import contextmanager
from typing import Iterator
 
import psycopg2

Streaming Postgres rows with a server-side cursor

generators context-managers database-streaming
Intermediate 7 steps
php 
<?php
 
namespace App\Http\Requests;
 

A validated date-range value object in PHP

value-object validation immutability
Intermediate 7 steps
php 
<?php
 
namespace App\Http\Controllers\Api;
 

Building a cursor-paginated feed in Laravel

cursor-pagination eager-loading validation
Intermediate 8 steps

Share this explainer

Here's the card — post it anywhere.

Streaming CSV imports with Laravel batches — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code