SparkSqlParser — Default SQL Parser

SparkSqlParser is the default parser of the SQL statements supported in Spark SQL with the astBuilder as SparkSqlAstBuilder and support for variable substitution.

Spark SQL supports SQL statements as described in SqlBase.g4 ANTLR grammar.

SparkSqlParser is available as sqlParser of a SessionState.

val spark: SparkSession = ...

SparkSqlParser is used to translate an expression to its corresponding Column in the following:

scala> expr("token = 'hello'")
16/07/07 18:32:53 INFO SparkSqlParser: Parsing command: token = 'hello'
res0: org.apache.spark.sql.Column = (token = hello)

SparkSqlParser is used to parse table strings into their corresponding table identifiers in the following:

SparkSqlParser is used to translate a SQL text to its corresponding LogicalPlan in sql method in SparkSession.


Enable INFO logging level for org.apache.spark.sql.execution.SparkSqlParser logger to see what happens inside.

Add the following line to conf/

Refer to Logging.

Variable Substitution

FIXME See SparkSqlParser and substitutor.

results matching ""

    No results matching ""