MicroBatchReadSupport Contract — Data Sources with MicroBatchReaders

MicroBatchReadSupport is the abstraction of data sources with a MicroBatchReader in Micro-Batch Stream Processing.

ContinuousReadSupport defines a single createMicroBatchReader method to create a MicroBatchReader.

MicroBatchReader createMicroBatchReader(
  Optional<StructType> schema,
  String checkpointLocation,
  DataSourceOptions options)

createMicroBatchReader is used when:

Table 1. MicroBatchReadSupports
MicroBatchReadSupport Description

KafkaSourceProvider

Data source provider for kafka format

RateStreamProvider

TextSocketSourceProvider

results matching ""

    No results matching ""