HiveMetastoreCatalog — Legacy SessionCatalog for Converting Hive Metastore Relations to Data Source Relations

HiveMetastoreCatalog is a legacy session-scoped catalog of relational entities that HiveSessionCatalog uses exclusively for converting Hive metastore relations to data source relations (when RelationConversions logical evaluation rule is executed).

HiveMetastoreCatalog is created exclusively when HiveSessionStateBuilder is requested for SessionCatalog (and creates a HiveSessionCatalog).

spark sql HiveMetastoreCatalog.png
Figure 1. HiveMetastoreCatalog, HiveSessionCatalog and HiveSessionStateBuilder

HiveMetastoreCatalog takes a SparkSession when created.

Converting HiveTableRelation to LogicalRelation — convertToLogicalRelation Method

convertToLogicalRelation(
  relation: HiveTableRelation,
  options: Map[String, String],
  fileFormatClass: Class[_ <: FileFormat],
  fileType: String): LogicalRelation

convertToLogicalRelation…​FIXME

Note
convertToLogicalRelation is used exclusively when RelationConversions logical evaluation rule is requested to convert a HiveTableRelation to a LogicalRelation for parquet, native and hive ORC storage formats.

inferIfNeeded Internal Method

inferIfNeeded(
  relation: HiveTableRelation,
  options: Map[String, String],
  fileFormat: FileFormat,
  fileIndexOpt: Option[FileIndex] = None): CatalogTable

inferIfNeeded…​FIXME

Note
inferIfNeeded is used exclusively when HiveMetastoreCatalog is requested to convertToLogicalRelation.

results matching ""

    No results matching ""