KafkaSourceRDDOffsetRange
KafkaSourceRDDOffsetRange is an offset range that one KafkaSourceRDDPartition partition of a KafkaSourceRDD has to read.
KafkaSourceRDDOffsetRange is created when:
-
KafkaRelationis requested to build a distributed data scan with column pruning (as a TableScan) (and creates a KafkaSourceRDD) -
KafkaSourceRDDis requested to resolveRange -
(Spark Structured Streaming)
KafkaSourceis requested togetBatch
KafkaSourceRDDOffsetRange takes the following when created:
-
Kafka TopicPartition
|
Note
|
TopicPartition is a topic name and partition number. |