RecordCollectorImpl

RecordCollectorImpl is a concrete RecordCollector that is created and immediately initialized exclusively when StreamTask is created.

RecordCollectorImpl uses task [streamTaskId] for the logPrefix (that uses the specified streamTaskId).

Creating RecordCollectorImpl Instance

RecordCollectorImpl takes the following to be created:

  • StreamTask ID

  • LogContext

  • ProductionExceptionHandler

  • skippedRecords sensor

RecordCollectorImpl initializes the internal properties.

send Method

<K, V> void send(
  final String topic,
  final K key,
  final V value,
  final Long timestamp,
  final Serializer<K> keySerializer,
  final Serializer<V> valueSerializer,
  final StreamPartitioner<? super K, ? super V> partitioner)
Note
send is part of RecordCollector Contract to…​FIXME.

send…​FIXME

Flushing Kafka Producer — flush Method

void flush()
Note
flush is part of the RecordCollector Contract to flush the internal Kafka producer.

flush prints out the following DEBUG message to the logs:

Flushing producer

flush requests the Kafka Producer to flush the buffered (accumulated) records (via Producer.flush).

In the end, flush checkForException.

Closing RecordCollector — close Method

void close()
Note
close is part of the RecordCollector Contract to close a RecordCollector.

close prints out the following DEBUG message to the logs:

Closing producer

close requests the internal Producer to close.

checkForException Internal Method

void checkForException()

checkForException checks the sendException internal registry and throws the exception if specified.

Note
checkForException is used when…​FIXME

Internal Properties

Name Description

sendException

KafkaException

offsets

Map<TopicPartition, Long> offsets

Initialized (as empty) when RecordCollectorImpl is created

A new pair of a Kafka TopicPartition with the offset added (or updated) after RecordCollectorImpl has finished sending a record

producer

Kafka Producer (Producer<byte[], byte[]>)

Used when…​FIXME

results matching ""

    No results matching ""