StreamSinkProvider Contract

StreamSinkProvider is the abstraction of providers that can create a streaming sink for a file format (e.g. parquet) or system (e.g. kafka).

Important
 StreamWriteSupport is a newer version of StreamSinkProvider (aka DataSource API V2) and new data sources should use the contract instead.
Table 1. StreamSinkProvider Contract
Method Description

createSink

 
createSink(
  sqlContext: SQLContext,
  parameters: Map[String, String],
  partitionColumns: Seq[String],
  outputMode: OutputMode): Sink

Creates a streaming sink

Used exclusively when DataSource is requested for a streaming sink (when DataStreamWriter is requested to start a streaming query)
Note
 KafkaSourceProvider is the only known StreamSinkProvider in Spark Structured Streaming.

results matching ""

    No results matching ""