Apache Kafka: KRaft, versions and upgrading from ZooKeeper

Updated November 2026

Apache Kafka is a distributed event-streaming platform: applications write streams of records into it, Kafka stores them durably for as long as you choose, and other applications read them in order. As of November 2026 the latest release is 4.4.0 (October 22, 2026), and since 4.0 Kafka runs only in KRaft mode, without Apache ZooKeeper.

What Kafka is

Think of Kafka as a set of append-only logs shared over the network. Each log is a topic, such as orders or page-views. Producers append records (a key, a value and a timestamp) to topics. Consumers read them, each keeping its own position, called an offset. Reading does not delete anything: records stay until the retention limit is reached (the sample broker configuration keeps them for 168 hours, or 7 days), so a new consumer can replay history and a failed one can resume where it stopped.

Two components ship with Kafka. Kafka Connect moves data between Kafka and other systems (databases, object storage, search engines) using connector plugins. Kafka Streams is a Java library for building applications that read, transform and write topics.

Typical uses

Architecture in plain words

A Kafka cluster is made of servers that take one or both of two roles, set with process.roles:

Brokers (process.roles=broker)
Store the data and serve producers and consumers. Each topic is split into partitions so it can be spread over many brokers, and each partition is copied to several brokers (the replication factor). One copy is the leader; the others follow it and can take over if its broker fails. Records are ordered within a partition, and records with the same key always go to the same partition.
Controllers (process.roles=controller)
Manage cluster metadata: which topics and partitions exist, which broker leads each partition, configuration and access rules. In KRaft mode the controllers keep this metadata in their own replicated log using the Raft consensus protocol (hence the name: Kafka Raft). The documentation says administrators typically choose 3 or 5 controllers. A group of 3 keeps working if 1 fails; a group of 5 survives 2 failures.
Combined (process.roles=broker,controller)
One process does both. Good for development and the quickstart; the documentation says combined mode is not recommended in critical deployment environments.

Consumers that share a group id split a topic's partitions between them, so you scale reading by adding consumers, up to one per partition.

Producers Consumers Brokers (data) Broker 1 Broker 2 Broker 3 KRaft controllers (3 or 5) metadata log, Raft consensus
Brokers hold the data; a small quorum of controllers holds the metadata. Before 4.0 that metadata could live in ZooKeeper instead.

Supported versions

The downloads page lists three supported releases, as of November 2026. Bug-fix releases are made only for supported releases, and the project aims for three feature releases a year (one every four months), so each line is supported for roughly a year.

Supported Apache Kafka releases, as of November 2026
ReleaseReleasedLine started
4.4.0October 22, 2026October 2026
4.3.2October 23, 2026May 22, 2026 (4.3.0)
4.2.2September 29, 2026February 17, 2026 (4.2.0)

Binaries for 4.x are built for Scala 2.13 only, so the file is named kafka_2.13-4.4.0.tgz; the "2.13" is the Scala version, not the Kafka version. The quickstart requires Java 17 or later. When 4.0 came out, brokers, Connect and the tools moved to Java 17, while clients and Kafka Streams still ran on Java 11.

Try it: the quickstart

With Java 17+ installed, download kafka_2.13-4.4.0.tgz from the downloads page, check it against the .sha512 and .asc files published beside it, then:

tar -xzf kafka_2.13-4.4.0.tgz
cd kafka_2.13-4.4.0

# One-time: create a cluster ID and format the storage directory
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c config/server.properties

# Start a combined broker+controller (leave this terminal open)
bin/kafka-server-start.sh config/server.properties

In a second terminal, create a topic, write a few lines, and read them back:

bin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server localhost:9092
bin/kafka-console-producer.sh --topic quickstart-events --bootstrap-server localhost:9092
# type a few lines, then Ctrl+C
bin/kafka-console-consumer.sh --topic quickstart-events --from-beginning --bootstrap-server localhost:9092

To check the KRaft quorum (cluster ID, leader, voters):

bin/kafka-metadata-quorum.sh --bootstrap-server localhost:9092 describe --status

Or run the project's container image instead: docker run -p 9092:9092 apache/kafka:4.4.0 (a GraalVM-based apache/kafka-native image also exists).

Not for production as is. The sample config/server.properties uses PLAINTEXT listeners with no authentication or encryption, a replication factor of 1, and stores data under /tmp/kraft-combined-logs, which many systems wipe at reboot. For real clusters, run separate controllers, use replication factor 3, put data on dedicated disks, and turn on TLS and SASL authentication with ACLs.

ZooKeeper removal and upgrading

For most of its history Kafka needed a separate Apache ZooKeeper ensemble to store metadata. KRaft replaced it gradually, and the timeline matters if you still run an old cluster:

So the upgrade path from a ZooKeeper cluster is:

  1. Upgrade the ZooKeeper-mode cluster to 3.9.x.
  2. Migrate the metadata to KRaft controllers while still on 3.9, following the "ZooKeeper to KRaft Migration" section of the 3.9 documentation. A cluster in ZooKeeper mode cannot be upgraded to 4.x directly.
  3. Upgrade the KRaft cluster to 4.x. Brokers need software and metadata version 3.3 or later; KRaft clusters older than 3.3 should go to 3.9 first.
  4. Once every node runs the new version and you are sure you will not roll back, finalize the metadata version, for example: 
    bin/kafka-features.sh --bootstrap-server localhost:9092 upgrade --release-version 4.0
    The upgrade notes warn that metadata downgrade is not supported for this version, so this step is one-way.

Also update scripts and monitoring: command-line tools connect with --bootstrap-server, and ZooKeeper-only metrics no longer exist. Major releases remove APIs that were deprecated for at least 12 months, so test client applications before upgrading them.

Security notes

Sources