HiveSessionStateBuilder — Builder of Hive-Specific SessionState

HiveSessionStateBuilder is a concrete builder to produce a Hive-aware SessionState for…​FIXME

HiveSessionStateBuilder comes with Hive-specific Analyzer, SparkPlanner, HiveSessionCatalog, HiveExternalCatalog and HiveSessionResourceLoader.

spark sql HiveSessionStateBuilder.png
Figure 1. HiveSessionStateBuilder’s Hive-Specific Properties

HiveSessionStateBuilder is created (using newBuilder) when…​FIXME

spark sql HiveSessionStateBuilder SessionState.png
Figure 2. HiveSessionStateBuilder and SessionState (in SparkSession)
Table 1. HiveSessionStateBuilder’s Properties
Name Description

analyzer

catalog

HiveSessionCatalog with the following:

Note
 If parentState is defined, the state is copied to catalog

Used to create Hive-specific Analyzer and a RelationConversions logical evaluation rule (as part of Hive-Specific Analyzer’s PostHoc Resolution Rules)

externalCatalog

HiveExternalCatalog

planner

SparkPlanner with Hive-specific strategies.

resourceLoader

HiveSessionResourceLoader

SparkPlanner with Hive-Specific Strategies — planner Property

planner: SparkPlanner
Note
 planner is part of BaseSessionStateBuilder Contract to create a query planner.

planner is a SparkPlanner with…​FIXME

planner uses the Hive-specific strategies.

Table 2. Hive-Specific SparkPlanner’s Hive-Specific Strategies
Strategy Description

HiveTableScans

Replaces HiveTableRelation logical operators with HiveTableScanExec physical operators

Scripts

Logical Query Plan Analyzer with Hive-Specific Rules — analyzer Property

analyzer: Analyzer
Note
 analyzer is part of BaseSessionStateBuilder Contract to create a logical query plan analyzer.

analyzer is a Analyzer with Hive-specific SessionCatalog (and SQLConf).

analyzer uses the Hive-specific extended resolution, postHoc resolution and extended check rules.

Table 3. Hive-Specific Analyzer’s Extended Resolution Rules (in the order of execution)
Logical Rule Description

ResolveHiveSerdeTable

FindDataSourceTable

ResolveSQLOnFile

Table 4. Hive-Specific Analyzer’s PostHoc Resolution Rules
Logical Rule Description

DetermineTableStats

RelationConversions

PreprocessTableCreation

PreprocessTableInsertion

DataSourceAnalysis

HiveAnalysis
Table 5. Hive-Specific Analyzer’s Extended Check Rules
Logical Rule Description

PreWriteCheck

PreReadCheck

Creating HiveSessionStateBuilder Instance

HiveSessionStateBuilder takes the following when created:

Builder Function to Create HiveSessionStateBuilder — newBuilder Method

newBuilder: (SparkSession, Option[SessionState]) => BaseSessionStateBuilder
Note
 newBuilder is part of BaseSessionStateBuilder contract to…​FIXME.

newBuilder…​FIXME

results matching ""

    No results matching ""