DDLUtils Utility

DDLUtils is a helper object that…​FIXME

DDLUtils uses hive value to denote Hive data source (Hive provider).

verifyPartitionProviderIsHive Utility

verifyPartitionProviderIsHive(
  spark: SparkSession,
  table: CatalogTable,
  action: String): Unit

verifyPartitionProviderIsHive requests the given CatalogTable for the TableIdentifier that is in turn requested for the table name.

verifyPartitionProviderIsHive throws an AnalysisException when spark.sql.hive.manageFilesourcePartitions configuration property is disabled (false) and the input CatalogTable is a data source table:

[action] is not allowed on [tableName] since filesource partition management is disabled (spark.sql.hive.manageFilesourcePartitions = false).

verifyPartitionProviderIsHive throws an AnalysisException when the tracksPartitionsInCatalog of the given CatalogTable is disabled (false) and the input CatalogTable is a data source table:

[action] is not allowed on [tableName] since its partition metadata is not stored in the Hive metastore. To import this information into the metastore, run `msck repair table [tableName]`
Note
 verifyPartitionProviderIsHive is used when AlterTableAddPartitionCommand, AlterTableRenamePartitionCommand, AlterTableDropPartitionCommand, AlterTableSetLocationCommand, TruncateTableCommand, DescribeTableCommand, and ShowPartitionsCommand commands are executed.

isDatasourceTable Utility

isDatasourceTable(
  table: CatalogTable): Boolean

isDatasourceTable is positive (true) when the provider of the input CatalogTable is not hive when defined. Otherwise, isDatasourceTable is negative (false).

Note

isDatasourceTable is used when:

isHiveTable Utility

isHiveTable(
  provider: Option[String]): Boolean

isHiveTable is positive (true) when the provider is hive when defined. Otherwise, isHiveTable is negative (false).

Note

isHiveTable is used when:

  • HiveAnalysis logical resolution rule is executed

  • DDLUtils utility is used to isHiveTable

  • HiveOnlyCheck extended check rule is executed

verifyNotReadPath Utility

verifyNotReadPath(
  query: LogicalPlan,
  outputPath: Path) : Unit

verifyNotReadPath…​FIXME

Note

verifyNotReadPath is used when…​FIXME

results matching ""

    No results matching ""