Messaging with Kafka
Durable, high-throughput event streaming between services with Spring for Apache Kafka.
On this page
In-process events decouple components inside one app. To decouple separate services across a network - and to do it durably, at high throughput - you need a message broker. Apache Kafka is the dominant choice for event streaming, and Spring for Apache Kafka makes producing and consuming events straightforward.
Kafka in one picture
Kafka is a durable, append-only log of events, organized into topics:
- Producers append events to a topic.
- Consumers read events from a topic, tracking their own position (offset).
- Events are retained (for a configured time), so consumers can replay history, and new consumers can catch up from the beginning.
That durability and replayability is what sets Kafka apart from a simple queue: the event log is the source of truth, not a transient pipe.
Add it and go
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>Producing is a one-liner with KafkaTemplate; consuming is an annotated method:
@Service
class BorrowEventPublisher {
private final KafkaTemplate<String, BookBorrowedEvent> kafka;
void publish(BookBorrowedEvent event) {
kafka.send("book-borrowed", event.isbn(), event); // append to the topic
}
}
@Component
class NotificationConsumer {
@KafkaListener(topics = "book-borrowed", groupId = "notifications")
void onBorrow(BookBorrowedEvent event) {
// a *separate service* reacts - send an email, push a notification
}
}The borrow service and the notification service now share nothing but a topic name. Either can be deployed, scaled, or restarted independently.
A courier hand-off is one-to-one and gone once delivered - if the recipient misses it, it's lost. Kafka is a shared, permanent ledger: producers write entries, and any number of readers consult it at their own pace, including latecomers who read from the start. Because the ledger persists, no reader can miss an entry, and you can always audit or replay history.
From in-process event to topic
Notice the shape is identical to the application event from the last lesson - a
BookBorrowedEvent published and consumed. That's the point: you design with events
in-process, then, when you split into services, the same event becomes a Kafka message. The
mental model carries straight over.
When BookVault splits into a lending service and a notification service, the lending service could just make an HTTP call to the notification service on each borrow. Give two advantages of publishing a Kafka event instead of calling directly.
What distinguishes Apache Kafka from a simple message queue?
Key takeaways
- A message broker decouples separate services across the network; Kafka is the dominant event-streaming choice.
- Kafka is a durable, append-only log of events per topic: producers append, consumers read at their own offset, events are retained and replayable.
- Spring for Apache Kafka: send with KafkaTemplate, consume with @KafkaListener - the two sides share only a topic name.
- Publishing an event beats a direct call: the consumer can be down/slow without failing the producer, and new consumers subscribe freely.
- The event shape matches in-process application events, so the design carries over when you distribute.