java 67 lines · 10 steps

How a Spring Batch CSV import job is wired

A chunk-oriented Spring Batch job reads a CSV, transforms each row into a User, and bulk-inserts them with fault tolerance.

Explained by highlit

Spring Intermediate batch processing chunk oriented fault tolerance etl dependency injection

1@Configuration
2public class UserImportJobConfig {
3 
  @Bean
  public Job userImportJob(JobRepository jobRepository, Step importUsersStep) {
      return new JobBuilder("userImportJob", jobRepository)
              .incrementer(new RunIdIncrementer())
              .start(importUsersStep)
              .build();
  }
11 
  @Bean
  public Step importUsersStep(JobRepository jobRepository,
                              PlatformTransactionManager transactionManager,
                              ItemReader<UserCsvRecord> reader,
                              ItemProcessor<UserCsvRecord, User> processor,
                              ItemWriter<User> writer) {
      return new StepBuilder("importUsersStep", jobRepository)
              .<UserCsvRecord, User>chunk(100, transactionManager)
              .reader(reader)
              .processor(processor)
              .writer(writer)
              .faultTolerant()
              .skip(FlatFileParseException.class)
              .skipLimit(25)
              .build();
  }
28 
  @Bean
  @StepScope
  public FlatFileItemReader<UserCsvRecord> reader(@Value("#{jobParameters['inputFile']}") Resource inputFile) {
      return new FlatFileItemReaderBuilder<UserCsvRecord>()
              .name("userCsvReader")
              .resource(inputFile)
              .linesToSkip(1)
              .delimited()
              .names("email", "firstName", "lastName", "country")
              .targetType(UserCsvRecord.class)
              .build();
  }
41 
  @Bean
  public ItemProcessor<UserCsvRecord, User> processor() {
      return record -> {
          String email = record.email().trim().toLowerCase();
          if (!email.contains("@")) {
              return null;
          }
          User user = new User();
          user.setEmail(email);
          user.setFullName("%s %s".formatted(record.firstName(), record.lastName()).trim());
          user.setCountry(record.country().toUpperCase());
          user.setStatus(UserStatus.PENDING);
          return user;
      };
  }
57 
  @Bean
  public JdbcBatchItemWriter<User> writer(DataSource dataSource) {
      return new JdbcBatchItemWriterBuilder<User>()
              .dataSource(dataSource)
              .sql("INSERT INTO users (email, full_name, country, status) "
                      + "VALUES (:email, :fullName, :country, :status)")
              .beanMapped()
              .build();
  }
67}

01 / 01

STEP 01

Walkthrough

Space play ←→ step click any line

Three takeaways

1Spring Batch splits ETL into reader, processor, and writer beans that a step orchestrates in fixed-size chunks.
2Chunk processing commits per batch, so tuning chunk size trades transaction overhead against memory and rollback cost.
3Fault tolerance with skip rules lets a job survive a bounded number of bad rows instead of failing outright.

Related explainers

java

public interface GitHubClient {
 
    @GetExchange("/users/{username}")
    GitHubUser getUser(@PathVariable String username);

Declarative HTTP clients in Spring

http-client declarative-api proxy

Intermediate 8 steps

java

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

Building a trie for autocomplete in Java

trie prefix-tree recursion

Intermediate 8 steps

java

@RestController
@RequestMapping("/api/products")
public class ProductSearchController {

Binding collection query params in Spring

rest-api query-parameters dependency-injection

Intermediate 6 steps

java

import java.util.ArrayDeque;
import java.util.Deque;
import java.util.Map;

Evaluating math expressions with two stacks

stacks parsing operator-precedence

Intermediate 9 steps

java

@Entity
@Table(name = "orders")
@SQLDelete(sql = "UPDATE orders SET deleted = true, deleted_at = now() WHERE id = ?")
@Where(clause = "deleted = false")

Soft deletes with Hibernate in Spring

soft-delete jpa hibernate

Intermediate 9 steps

java

@GetMapping("/files/{id}")
public ResponseEntity<StreamingResponseBody> download(
        @PathVariable String id,
        @RequestHeader(value = HttpHeaders.RANGE, required = false) String rangeHeader) throws IOException {