MicroBatchReadSupport Contract — Data Sources with MicroBatchReaders

MicroBatchReadSupport is the extension of the DataSourceV2 for data sources with a MicroBatchReader for Micro-Batch Stream Processing.

MicroBatchReadSupport defines a single createMicroBatchReader method to create a MicroBatchReader.

MicroBatchReader createMicroBatchReader(
  Optional<StructType> schema,
  String checkpointLocation,
  DataSourceOptions options)

createMicroBatchReader is used when:

Table 1. MicroBatchReadSupports
MicroBatchReadSupport Description

KafkaSourceProvider

Data source provider for kafka format

RateStreamProvider

Data source provider for rate format

TextSocketSourceProvider

Data source provider for socket format

results matching ""

    No results matching ""