ParserInterface Contract — SQL Parsers

ParserInterface is the abstraction of SQL parsers that can convert (parse) textual representation of SQL statements into Expressions, LogicalPlans, TableIdentifiers, FunctionIdentifier, StructType, and DataType.

Table 1. ParserInterface Contract
Method	Description
`parseDataType`	`parseDataType( sqlText: String): DataType` Parses a SQL text to an DataType Used when: `DataType` utility is requested to convert a DDL into a DataType (DataType.fromDDL) `StructType` is requested to add a field Column.cast `HiveClientImpl` utility is requested to getSparkSQLDataType `OrcFileOperator` is requested to `readSchema` `PythonSQLUtils` is requested to `parseDataType` `SQLUtils` is requested to `createStructField` `OrcUtils` is requested to `readSchema`
`parseExpression`	`parseExpression( sqlText: String): Expression` Parses a SQL text to an Expression Used in the following: Dataset operators: Dataset.selectExpr, Dataset.filter and Dataset.where expr standard function
`parseFunctionIdentifier`	`parseFunctionIdentifier( sqlText: String): FunctionIdentifier` Parses a SQL text to an `FunctionIdentifier` Used when: `SessionCatalog` is requested to listFunctions `CatalogImpl` is requested to getFunction and functionExists
`parsePlan`	`parsePlan( sqlText: String): LogicalPlan` Parses a SQL text to a LogicalPlan Used when: `SessionCatalog` is requested to look up a relation (table or view) in catalogs `SparkSession` is requested to execute a SQL query (aka SQL Mode)
`parseTableIdentifier`	`parseTableIdentifier( sqlText: String): TableIdentifier` Parses a SQL text to a `TableIdentifier` Used when: `DataFrameWriter` is requested to insertInto and saveAsTable `Dataset` is requested to createTempViewCommand `SparkSession` is requested to table `CatalogImpl` is requested to listColumns, getTable, tableExists, createTable, recoverPartitions, uncacheTable, and refreshTable `SessionState` is requested to refreshTable
`parseTableSchema`	`parseTableSchema( sqlText: String): StructType` Parses a SQL text to a schema (StructType) Used when: `DataType` utility is requested to convert a DDL into a DataType (DataType.fromDDL) `StructType` utility is requested to create a StructType for a given DDL-formatted string (StructType.fromDDL) `JdbcUtils` utility is requested to parseUserSpecifiedCreateTableColumnTypes and getCustomSchema

Note	AbstractSqlParser is the base extension of the `ParserInterface` contract in Spark SQL.

ParserInterface is available as sqlParser property of SessionState.

scala> :type spark
org.apache.spark.sql.SparkSession

scala> :type spark.sessionState.sqlParser
org.apache.spark.sql.catalyst.parser.ParserInterface

ParserInterface

ParserInterface Contract — SQL Parsers

results matching ""

No results matching ""