Abstract

Apache Kafka is a leading platform for building scalable, distributed event streaming systems. Traditionally, Kafka has relied on Apache ZooKeeper for managing cluster metadata and coordinating controller elections. However, the recent introduction of KRaft (Kafka Raft Metadata mode) eliminates this dependency by embedding a Raft-based consensus mechanism directly within Kafka [1] [6]. This architectural evolution raises key questions about the comparative performance, reliability, and operational efficiency of ZooKeeper-based versus KRaft-based deployments. [7] This study presents a comprehensive performance evaluation of Kafka's ZooKeeper and KRaft modes across multiple dimensions, including topic scalability, producer throughput, controller failover response, and memory efficiency. Through reproducible benchmarks involving 1,000-topic workloads, multi-threaded producers, and real-world failure simulations, the report analyzes the behavioral differences between the two architectures. The findings offer valuable insights for platform engineers, DevOps practitioners, and architects seeking to optimize Kafka deployments for high-throughput, cloud-native environments. [9] [15]

References

• Apache Kafka Documentation, "KRaft Mode (KIP-500) – Removing the dependency on ZooKeeper," Apache, 2024. [Online]. Available: https://kafka.apache.org/documentation/
• Apache ZooKeeper Documentation, "Apache ZooKeeper Overview and Configuration," Apache, 2024. [Online]. Available: https://zookeeper.apache.org/doc/
• Confluent, "Apache Kafka Performance"[Online]. Available: https://developer.confluent.io/learn/kafka-performance/
• Confluent Developer, "Understanding KRaft Mode," 2024. [Online]. Available: https://developer.confluent.io/learn/kraft/
• Spoud.io, "Embracing the Future of Kafka: Why It’s Time to Migrate from ZooKeeper to KRaft," Medium, 2024. [Online]. Available: https://spoud-io.medium.com/embracing-the-future-of-kafka-why-its-time-to-migrate-from-zookeeper-to-kraft-f1a5225ac48a
• Apache Kafka Improvement Proposal (KIP-500), "Replace ZooKeeper with a Self-Managed Metadata Quorum," Apache, 2024. [Online]. Available: https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum
• Apache Kafka Improvement Proposal (KIP-631), "The Quorum-based Kafka Controller," Apache, 2024. [Online]. Available: https://cwiki.apache.org/confluence/display/KAFKA/KIP-631%3A+The+Quorum-based+Kafka+Controller
• Confluent, "The Architecture of KRaft Mode: How Kafka Is Evolving," 2024. [Online]. Available: https://docs.confluent.io/platform/current/kafka-metadata/kraft.html
• Confluent, "Why replace ZooKeeper with Kafka Raft (KRaft)," 2024. [Online]. Available: https://www.confluent.io/blog/why-replace-zookeeper-with-kafka-raft-the-log-of-all-logs/
• Confluent Documentation, "KRaft Overview," 2024. [Online]. Available: https://docs.confluent.io/platform/current/kafka-metadata/kraft.html
• Confluent Documentation, "Migrate from ZooKeeper to KRaft on Confluent Platform," 2024. [Online]. Available: https://docs.confluent.io/platform/current/installation/migrate-zk-kraft.html
• Confluent, "Kafka 4.0 Release: Default KRaft, Queues, Faster Rebalances," 2024. [Online]. Available: https://www.confluent.io/blog/latest-apache-kafka-release/
• SoftwareMill, "Apache Kafka 4.0: Simplified Architecture with Default KRaft," 2024. [Online]. Available: https://softwaremill.com/apache-kafka-4-0-0-released-kraft-queues-better-rebalance-performance/
• The New Stack, "Kafka Drops ZooKeeper for ‘Real-Time’ KRaft," 2024. [Online]. Available: https://thenewstack.io/kafka-drops-zookeeper-for-real-time-kraft/
• Arvind Kumar, "Deep Dive: How KRaft Improves Over ZooKeeper in Kafka," Medium, 2025. [Online]. Available: https://codefarm0.medium.com/deep-dive-how-kraft-improves-over-zookeeper-in-kafka-f50b971e0c0e
• Sion Smith, "Apache Kafka’s KRaft Protocol: How to Eliminate ZooKeeper and Boost Performance by 8x," OSO, 2025. [Online]. Available: https://oso.sh/blog/apache-kafkas-kraft-protocol-how-to-eliminate-zookeeper-and-boost-performance-by-8x/