ConsoleSinkProvider is a DataSourceV2 with StreamWriteSupport for console data source format.

ConsoleSinkProvider is a DataSourceRegister and registers itself as the console data source format.

import org.apache.spark.sql.streaming.Trigger
val q = spark
  .format("console") // <-- requests ConsoleSinkProvider for a sink
scala> println(q.lastProgress.sink)
  "description" : "[email protected]2cfb1"

When requested for a StreamWriter, ConsoleSinkProvider simply creates a ConsoleWriter (with the given schema and options).

ConsoleSinkProvider is a CreatableRelationProvider.

createRelation Method

  sqlContext: SQLContext,
  mode: SaveMode,
  parameters: Map[String, String],
  data: DataFrame): BaseRelation
createRelation is part of the CreatableRelationProvider Contract to support writing a structured query (a DataFrame) per save mode.


results matching ""

    No results matching ""