DataSourceV2Utils Helper Object

DataSourceV2Utils is a helper object that is used exclusively to extract session configuration options (i.e. options with spark.datasource prefix for the keys in SQLConf) for DataSourceV2 data sources with SessionConfigSupport in Data Source API V2.

  ds: DataSourceV2,
  conf: SQLConf): Map[String, String]
extractSessionConfigs supports data sources with SessionConfigSupport only.

extractSessionConfigs requests the SessionConfigSupport data source for the custom key prefix for configuration options that is used to find all configuration options with the keys in the format of spark.datasource.[keyPrefix] in the given SQLConf.

extractSessionConfigs returns the matching keys with the spark.datasource.[keyPrefix] prefix removed (i.e. spark.datasource.keyPrefix.k1 becomes k1).


extractSessionConfigs is used when:

results matching ""

    No results matching ""