SchemaRelationProvider Contract — Relation Providers With Mandatory User-Defined Schema

The requirement of specifying a user-defined schema is enforced when DataSource is requested for a BaseRelation for a given data source format. If not specified, DataSource throws a AnalysisException:

A schema needs to be specified when using [className].
package org.apache.spark.sql.sources

trait SchemaRelationProvider {
  def createRelation(
    sqlContext: SQLContext,
    parameters: Map[String, String],
    schema: StructType): BaseRelation
}
Table 1. SchemaRelationProvider Contract
Method Description

createRelation

Creates a BaseRelation for the user-defined schema

Used exclusively when DataSource is requested for a BaseRelation for a given data source format

Note
There are no known direct implementation of PrunedFilteredScan Contract in Spark SQL.
Tip
Use RelationProvider for data source providers with schema inference.
Tip
Use both SchemaRelationProvider and RelationProvider if a data source should support both schema inference and user-defined schemas.

results matching ""

    No results matching ""