Broker Nodes — Kafka Servers

A Kafka server, a Kafka broker and a Kafka node all refer to the same concept and are synonyms (see the scaladoc of KafkaServer).

A Kafka broker is modelled as KafkaServer that hosts topics.

Given topics are always partitioned across brokers in a cluster a single broker hosts topic partitions of one or more topics actually (even when a topic is only partitioned to just a single partition).

Quoting Broker article (from Wikipedia, the free encyclopedia):

A broker is an individual person who arranges transactions between a buyer and a seller for a commission when the deal is executed.

A broker’s prime responsibility is to bring sellers and buyers together and thus a broker is the third-person facilitator between a buyer and a seller.

A Kafka broker receives messages from producers and stores them on disk keyed by unique offset.

A Kafka broker allows consumers to fetch messages by topic, partition and offset.

Kafka brokers can create a Kafka cluster by sharing information between each other directly or indirectly using Zookeeper.

A Kafka cluster has exactly one broker that acts as the Controller.

You can start a single Kafka broker using script.

Starting Kafka Broker

Start Zookeeper.

./bin/ config/

Only when Zookeeper is up and running you can start a Kafka server (that will connect to Zookeeper).

./bin/ config/ script starts a Kafka broker.

$ ./bin/
USAGE: ./bin/ [-daemon] [--override property=value]*
Before you run make sure that Zookeeper is up and running. Use zookeeper-server-start shell script. uses config/ for logging configuration that you can override using KAFKA_LOG4J_OPTS environment variable.

KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:config/" accepts KAFKA_HEAP_OPTS and EXTRA_ARGS environment variables.

Command-line options:

  1. -name — defaults to kafkaServer when in daemon mode.

  2. -loggc — enabled when in daemon mode.

  3. -daemon — enables daemon mode.

  4. --override property=value — value that should override the value set for property in file.

$ ./bin/ config/ --override
INFO [KafkaServer id=100] started (kafka.server.KafkaServer)

results matching ""

    No results matching ""