import org.apache.spark.sql.SaveMode
spark.range(10e4.toLong)
.write
.bucketBy(4, "id")
.sortBy("id")
.mode(SaveMode.Overwrite)
.saveAsTable("bucketed_4_10e4")
scala> sql("SHOW CREATE TABLE bucketed_4_10e4").show(truncate = false)
+----------------------------------------------------------------------------------------------------------------------------------------------------+
|createtab_stmt |
+----------------------------------------------------------------------------------------------------------------------------------------------------+
|CREATE TABLE `bucketed_4_10e4` (`id` BIGINT)
USING parquet
OPTIONS (
`serialization.format` '1'
)
CLUSTERED BY (id)
SORTED BY (id)
INTO 4 BUCKETS
|
+----------------------------------------------------------------------------------------------------------------------------------------------------+
scala> sql("SHOW CREATE TABLE bucketed_4_10e4").as[String].collect.foreach(println)
CREATE TABLE `bucketed_4_10e4` (`id` BIGINT)
USING parquet
OPTIONS (
`serialization.format` '1'
)
CLUSTERED BY (id)
SORTED BY (id)
INTO 4 BUCKETS
ShowCreateTableCommand Logical Command
ShowCreateTableCommand
is a logical command that executes a SHOW CREATE TABLE
SQL statement (with a data source / non-Hive or a Hive table).
ShowCreateTableCommand
is created when SparkSqlAstBuilder
is requested to parse SHOW CREATE TABLE SQL statement.
ShowCreateTableCommand
uses a single createtab_stmt
column (of type StringType) for the output schema.
ShowCreateTableCommand
takes a single TableIdentifier
when created.
Executing Logical Command — run
Method
run(sparkSession: SparkSession): Seq[Row]
Note
|
run is part of RunnableCommand Contract to execute (run) a logical command.
|
run
requests the SparkSession
for the SessionState that is used to access the SessionCatalog.
run
then requests the SessionCatalog
to retrieve the table metadata from the external catalog (metastore).
run
then showCreateDataSourceTable for a data source / non-Hive table or showCreateHiveTable for a Hive table (per the table metadata).
In the end, run
returns the CREATE TABLE
statement in a single Row
.
showHiveTableNonDataColumns
Internal Method
showHiveTableNonDataColumns(metadata: CatalogTable, builder: StringBuilder): Unit
showHiveTableNonDataColumns
…FIXME
Note
|
showHiveTableNonDataColumns is used exclusively when ShowCreateTableCommand logical command is requested to showCreateHiveTable.
|
showCreateHiveTable
Internal Method
showCreateHiveTable(metadata: CatalogTable): String
showCreateHiveTable
…FIXME
Note
|
showCreateHiveTable is used exclusively when ShowCreateTableCommand logical command is executed (with a Hive table).
|
showHiveTableHeader
Internal Method
showHiveTableHeader(metadata: CatalogTable, builder: StringBuilder): Unit
showHiveTableHeader
…FIXME
Note
|
showHiveTableHeader is used exclusively when ShowCreateTableCommand logical command is requested to showCreateHiveTable.
|