SessionConfigSupport Contract — Data Sources with Session-Scoped Configuration Options

SessionConfigSupport is the contract of DataSourceV2 data sources in Data Source API V2 that use custom key prefix for configuration options (i.e. options with spark.datasource prefix for the keys in SQLConf).

With SessionConfigSupport, a data source can be configured by additional (session-scoped) configuration options that are specified in SparkSession that extend user-defined options.

String keyPrefix()

keyPrefix is used exclusively when DataSourceV2Utils object is requested to extract session configuration options (i.e. options with spark.datasource prefix for the keys) for DataSourceV2 data sources with SessionConfigSupport.

keyPrefix must not be null or an IllegalArgumentException is thrown:

The data source config key prefix can't be null.

results matching ""

    No results matching ""