Controller Broker
Controller Broker (KafkaController) is a Kafka service that runs on every broker in a Kafka cluster, but only one can be active (elected) at any point in time.
The process of promoting a broker to be the active controller is called Kafka Controller Election.
Quoting Kafka Controller Internals:
In a Kafka cluster, one of the brokers serves as the controller, which is responsible for managing the states of partitions and replicas and for performing administrative tasks like reassigning partitions.
Kafka Controller registers handlers to be notified about changes in Zookeeper and propagate them across brokers in a Kafka cluster.
Lifecycle
When started, KafkaController
emits Startup controller event. That starts controller election (on the controller-event-thread).
During controller election, one KafkaController
becomes active (elected) and onControllerFailover. The ControllerContext is built on what is available in Zookeeper.
While in initializeControllerContext, KafkaController
updateLeaderAndIsrCache (and reads partition state from /brokers/topics/[topic]/partitions/[partition]/state
paths in Zookeeper that is then stored as partitionLeadershipInfo of the ControllerContext).