java 35 lines · 6 steps

Streaming large result sets in Spring Data JPA

Stream rows one at a time and detach each entity so you can archive millions of orders without exhausting memory.

Explained by highlit
Spring Advanced streaming jpa memory-management batch-processing transactions
1@Repository
2public interface OrderRepository extends JpaRepository<Order, Long> {
3 
4 @QueryHints(@QueryHint(name = HINT_FETCH_SIZE, value = "1000"))
5 @Query("select o from Order o where o.status = :status")
6 Stream<Order> streamByStatus(@Param("status") OrderStatus status);
7}
8 
9@Service
10public class OrderArchivalService {
11 
12 private final OrderRepository orderRepository;
13 private final ArchiveWriter archiveWriter;
14 
15 public OrderArchivalService(OrderRepository orderRepository, ArchiveWriter archiveWriter) {
16 this.orderRepository = orderRepository;
17 this.archiveWriter = archiveWriter;
18 }
19 
20 @Transactional(readOnly = true)
21 public long archiveCompletedOrders() {
22 var counter = new AtomicLong();
23 try (Stream<Order> orders = orderRepository.streamByStatus(OrderStatus.COMPLETED)) {
24 orders.forEach(order -> {
25 archiveWriter.write(order.toArchiveRecord());
26 counter.incrementAndGet();
27 entityManager.detach(order);
28 });
29 }
30 return counter.get();
31 }
32 
33 @PersistenceContext
34 private EntityManager entityManager;
35}
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Returning a Stream lets JPA pull rows lazily from a cursor instead of loading the whole result set into memory.
  2. 2Detaching each entity after use stops the persistence context from accumulating managed objects and leaking memory.
  3. 3A streaming query must run inside an open transaction and its Stream must be closed to release the underlying database cursor.

Related explainers

java 
public interface GitHubClient {
 
    @GetExchange("/users/{username}")
    GitHubUser getUser(@PathVariable String username);

Declarative HTTP clients in Spring

http-client declarative-api proxy
Intermediate 8 steps
php 
<?php
 
namespace App\Console\Commands;
 

Releasing stale document locks in Laravel

artisan-command transactions row-locking
Intermediate 6 steps
java 
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

Building a trie for autocomplete in Java

trie prefix-tree recursion
Intermediate 8 steps
typescript 
import { Injectable, NestInterceptor, ExecutionContext, CallHandler } from '@nestjs/common';
import { Observable, catchError, concatMap, finalize } from 'rxjs';
import { DataSource, QueryRunner } from 'typeorm';
 

Wrapping requests in a transaction with NestJS

interceptors transactions rxjs
Advanced 7 steps
java 
@RestController
@RequestMapping("/api/products")
public class ProductSearchController {
 

Binding collection query params in Spring

rest-api query-parameters dependency-injection
Intermediate 6 steps
java 
import java.util.ArrayDeque;
import java.util.Deque;
import java.util.Map;
 

Evaluating math expressions with two stacks

stacks parsing operator-precedence
Intermediate 9 steps

Share this explainer

Here's the card — post it anywhere.

Streaming large result sets in Spring Data JPA — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code