java 35 lines · 6 steps

Streaming large result sets in Spring Data JPA

Stream rows one at a time and detach each entity so you can archive millions of orders without exhausting memory.

Explained by highlit
Spring Advanced streaming jpa memory-management batch-processing transactions
1@Repository
2public interface OrderRepository extends JpaRepository<Order, Long> {
3 
4 @QueryHints(@QueryHint(name = HINT_FETCH_SIZE, value = "1000"))
5 @Query("select o from Order o where o.status = :status")
6 Stream<Order> streamByStatus(@Param("status") OrderStatus status);
7}
8 
9@Service
10public class OrderArchivalService {
11 
12 private final OrderRepository orderRepository;
13 private final ArchiveWriter archiveWriter;
14 
15 public OrderArchivalService(OrderRepository orderRepository, ArchiveWriter archiveWriter) {
16 this.orderRepository = orderRepository;
17 this.archiveWriter = archiveWriter;
18 }
19 
20 @Transactional(readOnly = true)
21 public long archiveCompletedOrders() {
22 var counter = new AtomicLong();
23 try (Stream<Order> orders = orderRepository.streamByStatus(OrderStatus.COMPLETED)) {
24 orders.forEach(order -> {
25 archiveWriter.write(order.toArchiveRecord());
26 counter.incrementAndGet();
27 entityManager.detach(order);
28 });
29 }
30 return counter.get();
31 }
32 
33 @PersistenceContext
34 private EntityManager entityManager;
35}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Returning a Stream lets JPA pull rows lazily from a cursor instead of loading the whole result set into memory.
  2. 2Detaching each entity after use stops the persistence context from accumulating managed objects and leaking memory.
  3. 3A streaming query must run inside an open transaction and its Stream must be closed to release the underlying database cursor.

Related explainers

java 
public class ThumbnailProcessor {
 
    private static final int MAX_CONCURRENCY = 4;
 

Bounded parallel thumbnail rendering in Java

concurrency thread-pool futures
Intermediate 7 steps
java 
public class SortedListMerger {
 
    public static int[] merge(int[] a, int[] b) {
        int[] result = new int[a.length + b.length];

Merging two sorted arrays in Java

two-pointers merging arrays
Beginner 6 steps
python 
import csv
import io
from datetime import datetime
 

Streaming a CSV export in Flask

streaming generators csv
Intermediate 9 steps
java 
import java.util.ArrayDeque;
import java.util.Deque;
 
public final class RollingAverage {

A rolling average over a sliding window

sliding-window running-sum deque
Intermediate 7 steps
java 
@Target({ElementType.FIELD, ElementType.PARAMETER})
@Retention(RetentionPolicy.RUNTIME)
@Constraint(validatedBy = StrongPasswordValidator.class)
@Documented

Building a custom @StrongPassword validator in Spring

bean-validation annotations regex
Intermediate 7 steps
python 
import csv
import io
from datetime import date
 

Streaming a CSV export in FastAPI

streaming async-generators csv
Advanced 8 steps

Share this explainer

Here's the card — post it anywhere.

Streaming large result sets in Spring Data JPA — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code