void commit(Offset end)
ContinuousReader Contract — Data Source Readers in Continuous Stream Processing
ContinuousReader
is the extension of Spark SQL’s DataSourceReader
(and BaseStreamingSource) contracts for data source readers in Continuous Stream Processing.
ContinuousReader
is part of the novel Data Source API V2 in Spark SQL.
Tip
|
Read up on Data Source API V2 in The Internals of Spark SQL book. |
Method | Description | ||
---|---|---|---|
|
Commits the specified offset Used exclusively when |
||
|
Deserializes an offset from JSON representation Used when |
||
|
|
||
|
Used exclusively when |
||
|
Indicates that the reader needs reconfiguration (e.g. to generate new input partitions) Used exclusively when |
||
|
Used exclusively when |
ContinuousReader | Description |
---|---|
TextSocketContinuousReader |