BaseStreamingSource Contract — Base of Streaming Readers and Sources

BaseStreamingSource is the abstraction of streaming readers and sources that can be stopped.

The main purpose of BaseStreamingSource is to share a common abstraction between the former Data Source API V1 (Source API) and the modern Data Source API V2 (until Spark Structured Streaming migrates to the Data Source API V2 fully).

Table 1. BaseStreamingSource Contract
Method Description

stop

void stop()

Stops the streaming source or reader (and frees up any resources it may have allocated)

Used when:

Table 2. BaseStreamingSources (Extensions Only)
BaseStreamingSource Description

ContinuousReader

Data source readers in Continuous Stream Processing (based on Data Source API V2)

MemoryStreamBase

Base implementation of ContinuousMemoryStream (for Continuous Stream Processing) and MemoryStream (for Micro-Batch Stream Processing)

MicroBatchReader

Data source readers in Micro-Batch Stream Processing (based on Data Source API V2)

Source

Streaming sources for Micro-Batch Stream Processing (based on Data Source API V1)

results matching ""

    No results matching ""