Broker Nodes — Kafka Servers

Note
 A Kafka server, a Kafka broker and a Kafka node all refer to the same concept and are synonyms (see the scaladoc of KafkaServer).

A Kafka broker is modelled as KafkaServer that hosts topics.

Note
 Given topics are always partitioned across brokers in a cluster a single broker hosts topic partitions of one or more topics actually (even when a topic is only partitioned to just a single partition).

Quoting Broker article (from Wikipedia, the free encyclopedia):

A broker is an individual person who arranges transactions between a buyer and a seller for a commission when the deal is executed.

A broker’s prime responsibility is to bring sellers and buyers together and thus a broker is the third-person facilitator between a buyer and a seller.

A Kafka broker receives messages from producers and stores them on disk keyed by unique offset.

A Kafka broker allows consumers to fetch messages by topic, partition and offset.

Kafka brokers can create a Kafka cluster by sharing information between each other directly or indirectly using Zookeeper.

A Kafka cluster has exactly one broker that acts as the Controller.

You can start a single Kafka broker using kafka-server-start.sh script.

Starting Kafka Broker

Start Zookeeper.

./bin/zookeeper-server-start.sh config/zookeeper.properties

Only when Zookeeper is up and running you can start a Kafka server (that will connect to Zookeeper).

./bin/kafka-server-start.sh config/server.properties
Tip
 Read kafka-server-start.sh script.

kafka-server-start.sh script

kafka-server-start.sh starts a Kafka broker.

$ ./bin/kafka-server-start.sh
USAGE: ./bin/kafka-server-start.sh [-daemon] server.properties [--override property=value]*
Note
 Before you run kafka-server-start.sh make sure that Zookeeper is up and running. Use zookeeper-server-start shell script.

kafka-server-start.sh uses config/log4j.properties for logging configuration that you can override using KAFKA_LOG4J_OPTS environment variable.

KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:config/log4j.properties"

kafka-server-start.sh accepts KAFKA_HEAP_OPTS and EXTRA_ARGS environment variables.

Command-line options:

  1. -name — defaults to kafkaServer when in daemon mode.

  2. -loggc — enabled when in daemon mode.

  3. -daemon — enables daemon mode.

  4. --override property=value — value that should override the value set for property in server.properties file.

$ ./bin/kafka-server-start.sh config/server.properties --override broker.id=100
...
INFO [KafkaServer id=100] started (kafka.server.KafkaServer)

results matching ""

    No results matching ""