ConsumerStrategy Contract — Kafka Consumer Providers

ConsumerStrategy is the contract for Kafka Consumer providers that can create a Kafka Consumer given Kafka parameters.

package org.apache.spark.sql.kafka010

sealed trait ConsumerStrategy {
  def createConsumer(kafkaParams: ju.Map[String, Object]): Consumer[Array[Byte], Array[Byte]]
}
Table 1. ConsumerStrategy Contract
Property Description

createConsumer

Creates a Kafka Consumer (of keys and values of type Array[Byte])

Used exclusively when KafkaOffsetReader is requested to creating a Kafka Consumer

Table 2. ConsumerStrategies
ConsumerStrategy createConsumer

AssignStrategy

Uses KafkaConsumer.assign(Collection<TopicPartition> partitions)

SubscribeStrategy

Uses KafkaConsumer.subscribe(Collection<String> topics)

SubscribePatternStrategy

Tip
Refer to java.util.regex.Pattern for the format of supported topic subscription regex patterns.
Note
ConsumerStrategy is a Scala sealed trait which means that all the implementations are in the same compilation unit (a single file).

results matching ""

    No results matching ""