Kafka Properties

Table 1. Properties
Name Description


Comma-separated list of URIs and listener names for other Kafka brokers and Kafka clients to use

Default: listeners

In IaaS environments (e.g. Docker or Kubernetes), advertised.listeners may likely need to be different from the interface to which a Kafka broker binds. That ensures that the Kafka broker advertises an address that is accessible from both local and external hosts.

Unlike listeners it is invalid to advertise the non-routable meta-address.


How often (in milliseconds) consumer offsets should be auto-committed when enable.auto.commit is enabled


Enables auto creation of a topic

Default: true


Enables auto leader balancing

Default: true

A background thread checks and triggers leader balance if required.


Reset policy — what to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted):

  • earliest — automatically reset the offset to the earliest offset

  • latest — automatically reset the offset to the latest offset

  • none — throw an exception to the consumer if no previous offset is found for the consumer’s group

  • anything else: throw an exception to a consumer

Default: latest



The broker id of a Kafka broker for identification purposes

If unset, a unique broker id will be generated. To avoid conflicts between zookeeper generated broker id’s and user configured broker id’s, generated broker IDs start from reserved.broker.max.id + 1.

Use KafkaConfig.brokerId to access the current value.


Enables automatic broker id generation of a Kafka broker

Default: true

When enabled (true) the value configured for reserved.broker.max.id should be checked.


The number of threads to use for various background processing tasks

As least 1 thread is required

Default: 10


A comma-separated list of host:port pairs that are the addresses of one or more brokers in a Kafka cluster, e.g. localhost:9092 or localhost:9092,another.host:9092.

Default: (empty)



Automatically check the CRC32 of the records consumed. This ensures no on-the-wire or on-disk corruption to the messages occurred. This check adds some overhead, so it may be disabled in cases seeking extreme performance.

Use ConsumerConfig.CHECK_CRCS_CONFIG


Default: (randomly-generated)


Connection close delay on failed authentication: this is the time (in milliseconds) by which connection close will be delayed on authentication failure. This must be configured to be less than connections.max.idle.ms to prevent connection timeout.

Default: 100

Has to be at least 0


Idle connections timeout: the server socket processor threads close the connections that idle more than this

Default: 10 * 60 * 1000L


The default replication factor that is used for auto-created topics

Default: 1

Increase the default value to at least 2



Enables topic deletion

Deleting topic through the admin tool has no effect with the property disabled.

Default: true


When enabled (i.e. true) consumer offsets are committed automatically in the background (aka consumer auto commit) every auto.commit.interval.ms

Default: true

When disabled, offsets have to be committed manually (synchronously using KafkaConsumer.commitSync or asynchronously KafkaConsumer.commitAsync). On restart restore the position of a consumer using KafkaConsumer.seek.

Used when KafkaConsumer is created and creates a ConsumerCoordinator.


The minimum amount of data the server should return for a fetch request. If insufficient data is available the request will wait for that much data to accumulate before answering the request. The default setting of 1 byte means that fetch requests are answered as soon as a single byte of data is available or the fetch request times out waiting for data to arrive. Setting this to something greater than 1 will cause the server to wait for larger amounts of data to accumulate which can improve server throughput a bit at the cost of some additional latency.



The maximum amount of data the server should return for a fetch request. Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum. The maximum record batch size accepted by the broker is defined via message.max.bytes (broker config) or max.message.bytes (topic config). Note that the consumer performs multiple fetches in parallel.



The maximum amount of time the server will block before answering the fetch request if there isn’t sufficient data to immediately satisfy the requirement given by fetch.min.bytes.



The name of the consumer group the consumer is part of.


The expected time between heartbeats to the group coordinator when using Kafka’s group management facilities.


The hostname a Kafka broker listens on

Default: (empty)


Name of the listener that is used for inter-broker communication

It is not allowed to set inter.broker.listener.name and security.inter.broker.protocol properties at the same time.



Comma-separated list of ConsumerInterceptor class names.

Default: (empty)


How often the active KafkaController schedules the partition rebalance check (aka AutoLeaderRebalance or AutoPreferredReplicaLeaderElection or auto leader balancing)

Default: 300


Comma-separated list of URIs and listener names that a Kafka broker will listen on

Default: PLAINTEXT://host.name:port

Use to bind to all the network interfaces on a machine or leave it empty to bind to the default interface.


The directory in which the log data is kept

Default: /tmp/kafka-logs


The directories in which the log data is kept

Default: log.dir

Use KafkaConfig.logDirs to access the current value.


How to deserialize message keys.



The maximum amount of data per-partition the server will return. Records are fetched in batches by the consumer. If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress. The maximum record batch size accepted by the broker is defined via message.max.bytes (broker config) or max.message.bytes (topic config).


Use fetch.max.bytes for limiting the consumer request size.


(KafkaConsumer) The maximum number of records returned from a Kafka Consumer when polling topics for records.

The default setting (-1) sets no upper bound on the number of records, i.e. Consumer.poll() will return as soon as either any data is available or the passed timeout expires.

max.poll.records was added to Kafka in by KIP-41: KafkaConsumer Max Records.

From kafka-clients mailing list:

max.poll.records only controls the number of records returned from poll, but does not affect fetching. The consumer will try to prefetch records from all partitions it is assigned. It will then buffer those records and return them in batches of max.poll.records each (either all from the same topic partition if there are enough left to satisfy the number of records, or from multiple topic partitions if the data from the last fetch for one of the topic partitions does not cover the max.poll.records).


Internally, max.poll.records is used exclusively when KafkaConsumer is created (to create a Fetcher).



The list of fully-qualified classes names of the metrics reporters.

Default: JmxReporter


Number of samples to compute metrics.


Time window (in milliseconds) a metrics sample is computed over.


The minimum number of replicas in ISR that is needed to commit a produce request with required.acks=-1 (or all)

Default: 1

When a Kafka producer sets acks to all (or -1), this configuration specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.

If this minimum cannot be met, then the producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend).

When used together, min.insync.replicas and acks allow you to enforce greater durability guarantees.

A typical scenario would be to create a topic with a replication factor of 3, set min.insync.replicas to 2, and produce with acks of "all". This will ensure that the producer raises an exception if a majority of replicas do not receive a write.


The number of threads that KafkaServer uses for processing requests, which may include disk I/O

Default: 8


The number of threads that SocketServer uses for the number of processors per endpoint.

Default: 3


The number of log partitions for auto-created topics

Default: 1

Increase the default value (1) since it is better to over-partition a topic that leads to a better data balancing and aids consumer parallelism.


The port a Kafka broker listens on

Default: 9092


The number of queued requests allowed before blocking the network threads

Default: 500

Must be at least 1


The maximum allowed time for each worker to join the group once a rebalance has begun.


The hint about the size of the TCP network receive buffer (SO_RCVBUF) to use (for a socket) when reading data. If the value is -1, the OS default will be used.




Maximum number that can be used for broker.id. Has to be at least 0.

Default: 1000

* Use KafkaConfig.MaxReservedBrokerIdProp to reference the property

* Use KafkaConfig.maxReservedBrokerId to access the current value


Time to wait before attempting to retry a failed request to a given topic partition. This avoids repeatedly sending requests in a tight loop under some failure scenarios.



The configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted.




The hint about the size of the TCP network send buffer (SO_SNDBUF) to use (for a socket) when sending data. If the value is -1, the OS default will be used.


The timeout used to detect worker failures.

Default: 10000


The maximum number of bytes in a socket request

Default: 100 * 1024 * 1024

Must be at least 1


The maximum allowed timeout for transactions (in millis).

If a client’s requested transaction time exceed this, then the broker will return an error in InitProducerIdRequest. This prevents a client from a too large timeout that can stall consumers reading from topics included in the transaction.

Default: 15 minutes

Must be at least 1


Controls whether to enable replicas not in the ISR set to be elected as leader as the last resort, even though doing so may result in data loss.

Default: 1


How to deserialize message values


Comma-separated list of Zookeeper hosts (as host:port pairs) that brokers register to, e.g. localhost:2181,,,

Default: (empty)

Zookeeper URIs can have an optional chroot path suffix at the end, e.g.,,

If the optional chroot path suffix is used, all paths are relative to this path.

It is recommended to include all the hosts in a Zookeeper ensemble (cluster)


The max time that the client waits to establish a connection to zookeeper


The maximum number of unacknowledged requests the client will send to Zookeeper before blocking. Has to be at least 1

Default: 10


Zookeeper session timeout

Default: 6000


Enables secure ACLs

Default: false

results matching ""

    No results matching ""