ConsumerStrategy Contract — Kafka Consumer Providers

ConsumerStrategy is the contract for Kafka Consumer providers that can create a Kafka Consumer given Kafka parameters.

package org.apache.spark.sql.kafka010

sealed trait ConsumerStrategy {
  def createConsumer(kafkaParams: ju.Map[String, Object]): Consumer[Array[Byte], Array[Byte]]
}
Table 1. ConsumerStrategy Contract
Property Description

createConsumer

Creates a Kafka Consumer (of keys and values of type Array[Byte])

Used exclusively when KafkaOffsetReader is requested to creating a Kafka Consumer
Table 2. ConsumerStrategies
ConsumerStrategy createConsumer

AssignStrategy

Uses KafkaConsumer.assign(Collection<TopicPartition> partitions)

SubscribeStrategy

Uses KafkaConsumer.subscribe(Collection<String> topics)

SubscribePatternStrategy

Uses KafkaConsumer.subscribe(Pattern pattern, ConsumerRebalanceListener listener) with NoOpConsumerRebalanceListener.

Tip
 Refer to java.util.regex.Pattern for the format of supported topic subscription regex patterns.
Note
 ConsumerStrategy is a Scala sealed trait which means that all the implementations are in the same compilation unit (a single file).

results matching ""

    No results matching ""