Why Batch Processing

Some work doesn't fit a web request. A user clicks a button and waits a second or two - that's the online world. But a nightly overdue-loan report for BookVault has to scan every active loan, find the ones past their due date, group them by member, and email a summary. That might touch millions of rows and run for minutes. No one is sitting there watching a spinner. This is batch processing: large volumes, no interactive user, run on a schedule.

What makes work "batch"

Three signatures tell you a job belongs in a batch, not an HTTP handler:

• Volume - it processes a whole dataset (every loan, every invoice, every user), not one item.
• No user waiting - it runs on a timer or is kicked off by an operator, and completes minutes or hours later.
• Must be reliable - if it dies halfway through 2 million records, you cannot simply "click again"; it has to resume, skip bad rows, and never double-charge or double-email.

Reports, data migrations, monthly billing, ETL (extract-transform-load), reindexing a search engine - all batch.

The naive loop, and why it breaks

Your first instinct is a plain loop:

List<Loan> loans = loanRepository.findAllActive();   // load 2 million rows into memory 💥
for (Loan loan : loans) {
    if (loan.isOverdue()) {
        reportRow.add(summarize(loan));
    }
}
emailService.send(buildReport(reportRow));

This looks fine on 200 rows and collapses on 2 million. It has four fatal problems:

1. Memory - findAllActive() pulls every row into a List, exhausting the heap.
2. No transactions - one giant commit at the end (or none), so a crash loses everything or corrupts state.
3. No restart - if it dies at row 1,900,000, the next run starts again from zero.
4. No fault tolerance - one malformed loan throws, and the whole run aborts.

What a batch framework gives you

Spring Batch solves exactly these four problems so you don't reinvent them:

• Chunk-oriented processing streams records in small transactional batches (say 100 at a time) instead of loading everything - bounded memory.
• A JobRepository persists progress in the database, so a failed job restarts where it stopped.
• Skip and retry policies let a bad record be logged and skipped, or a transient failure retried, without killing the run.
• Scaling hooks - multi-threaded steps, partitioning - for when one thread is too slow.

You describe what to read, process, and write; the framework handles the transactions, the bookmarking, and the failure recovery.