Sender — Kafka Producer I/O Thread

Sender is a Kafka producer I/O thread (of execution) that sends produce requests to a Kafka cluster.

Sender.png
Figure 1. Sender and KafkaProducer

Sender is created exclusively when KafkaProducer is created.

Sender is started under the name kafka-producer-network-thread | [clientId] when KafkaProducer is created.

Table 1. Sender’s Internal Properties (e.g. Registries and Counters)
Name Description

running

Flag that controls whether run should finish or not

Sending ProduceRequests — sendProduceRequests Internal Method

void sendProduceRequests(Map<Integer, List<ProducerBatch>> collated, long now)

sendProduceRequests…​FIXME

Note
sendProduceRequests is used exclusively when Sender is requested to sendProducerData.

Sending ProduceRequest — sendProduceRequest Internal Method

void sendProduceRequest(
  long now,
  int destination,
  short acks,
  int timeout,
  List<ProducerBatch> batches)

sendProduceRequest…​FIXME

Note
sendProduceRequest is used exclusively when Sender is requested to sendProduceRequests.

sendProducerData Internal Method

long sendProducerData(long now)

sendProducerData…​FIXME

Note
sendProducerData is used exclusively when Sender is requested to run a single iteration of sending.

Running Single Iteration of Sending — run Method

void run(long now)

run does…​FIXME…​when the transactionManager is available.

run sendProducerData and then requests the KafkaClient to poll.

Note
run is used exclusively when Sender is requested to start (as a thread of execution).

Starting Thread of Execution — run Method (of Java’s Runnable)

void run()
Note
run is a part of java.lang.Runnable that is executed when the thread is started.

run first prints out the following DEBUG message to the logs:

Starting Kafka producer I/O thread.

run keeps running until the running flag is off.

run…​FIXME

Creating Sender Instance

Sender takes the following when created:

  • LogContext

  • KafkaClient

  • Metadata

  • RecordAccumulator

  • guaranteeMessageOrder flag

  • maxRequestSize

  • acks

  • The number of retries

  • SenderMetricsRegistry

  • Time

  • requestTimeout

  • retryBackoffMs

  • TransactionManager

  • ApiVersions

Sender initializes the internal registries and counters.

maybeSendTransactionalRequest Internal Method

boolean maybeSendTransactionalRequest(long now)

maybeSendTransactionalRequest…​FIXME

Note
maybeSendTransactionalRequest is used exclusively when Sender is running.

maybeWaitForProducerId Internal Method

void maybeWaitForProducerId()

maybeWaitForProducerId…​FIXME

Note
maybeWaitForProducerId is used exclusively when Sender is running.

completeBatch Internal Method

void completeBatch(
  ProducerBatch batch,
  ProduceResponse.PartitionResponse response,
  long correlationId,
  long now,
  long throttleUntilTimeMs)

completeBatch…​FIXME

Note
completeBatch is used exclusively when Sender is requested to handle a ProduceResponse.

Handling ProduceResponse — handleProduceResponse Internal Method

void handleProduceResponse(
  ClientResponse response,
  Map<TopicPartition, ProducerBatch> batches,
  long now)

handleProduceResponse…​FIXME

Note
handleProduceResponse is used exclusively when Sender is requested to sendProduceRequest.

awaitLeastLoadedNodeReady Internal Method

Node awaitLeastLoadedNodeReady(long remainingTimeMs)

awaitLeastLoadedNodeReady…​FIXME

Note
awaitLeastLoadedNodeReady is used when Sender is requsted to maybeSendTransactionalRequest and maybeWaitForProducerId.

results matching ""

    No results matching ""