java 60 lines · 10 steps

Retrying failed jobs on a Spring schedule

A scheduled Spring component pulls failed jobs in batches and retries them with exponential backoff until they succeed or die.

Explained by highlit
1@Component
2@RequiredArgsConstructor
3@Slf4j
4public class FailedJobReprocessor {
5 
6 private static final int MAX_ATTEMPTS = 5;
7 private static final int BATCH_SIZE = 50;
8 
9 private final ScheduledJobRepository jobRepository;
10 private final JobDispatcher jobDispatcher;
11 
12 @Scheduled(fixedDelayString = "${jobs.reprocess.interval-ms:30000}")
13 @Transactional
14 public void reprocessFailedJobs() {
15 Instant now = Instant.now();
16 List<ScheduledJob> due = jobRepository.findRetryable(
17 JobStatus.FAILED, now, MAX_ATTEMPTS,
18 PageRequest.of(0, BATCH_SIZE));
19 
20 if (due.isEmpty()) {
21 return;
22 }
23 
24 log.info("Reprocessing {} failed jobs", due.size());
25 
26 for (ScheduledJob job : due) {
27 job.setStatus(JobStatus.RUNNING);
28 job.setAttempts(job.getAttempts() + 1);
29 job.setLastAttemptAt(now);
30 
31 try {
32 jobDispatcher.dispatch(job.getType(), job.getPayload());
33 job.setStatus(JobStatus.COMPLETED);
34 job.setLastError(null);
35 } catch (Exception ex) {
36 log.warn("Job {} failed on attempt {}", job.getId(), job.getAttempts(), ex);
37 job.setLastError(truncate(ex.getMessage()));
38 if (job.getAttempts() >= MAX_ATTEMPTS) {
39 job.setStatus(JobStatus.DEAD);
40 } else {
41 job.setStatus(JobStatus.FAILED);
42 job.setNextRetryAt(now.plus(backoff(job.getAttempts())));
43 }
44 }
45 
46 jobRepository.save(job);
47 }
48 }
49 
50 private Duration backoff(int attempts) {
51 return Duration.ofSeconds((long) Math.pow(2, attempts) * 10);
52 }
53 
54 private String truncate(String message) {
55 if (message == null) {
56 return "unknown error";
57 }
58 return message.length() > 500 ? message.substring(0, 500) : message;
59 }
60}
01 / 01
STEP 01

Walkthrough

Space play step click any line
Three takeaways
  1. 1Capping attempts and moving exhausted jobs to a DEAD state prevents infinite retry loops.
  2. 2Exponential backoff spreads retries out so a flaky dependency isn't hammered on every cycle.
  3. 3Processing in bounded batches keeps each scheduled run predictable under a large backlog.

Related explainers

Share this explainer

Here's the card — post it anywhere.

Retrying failed jobs on a Spring schedule — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code