KafkaSourceOffset is a OffsetV2 that…​FIXME

KafkaSourceOffset takes a collection of Kafka TopicPartitions and their offsets when created.

Creating KafkaSourceOffset Instance


Getting Partition Offsets — getPartitionOffsets Method

getPartitionOffsets(offset: Offset): Map[TopicPartition, Long]

getPartitionOffsets takes KafkaSourceOffset.partitionToOffsets from offset.

If offset is KafkaSourceOffset, getPartitionOffsets takes the partitions and offsets straight from it.

If however offset is SerializedOffset, getPartitionOffsets deserializes the offsets from JSON.

getPartitionOffsets reports an IllegalArgumentException when offset is neither KafkaSourceOffset or SerializedOffset.

Invalid conversion from offset of [class] to KafkaSourceOffset
getPartitionOffsets is used exclusively when KafkaSource generates a DataFrame with records from Kafka for a batch.

results matching ""

    No results matching ""