// Note "order by 1" clause
val sqlText = "select id from VALUES 1, 2, 3 t1(id) order by 1"
val logicalPlan = spark.sql(sqlText).queryExecution.logical
scala> println(logicalPlan.numberedTreeString)
00 'Sort [1 ASC NULLS FIRST], true
01 +- 'Project ['id]
02 +- 'SubqueryAlias t1
03 +- 'UnresolvedInlineTable [id], [List(1), List(2), List(3)]
import org.apache.spark.sql.catalyst.analysis.SubstituteUnresolvedOrdinals
val rule = new SubstituteUnresolvedOrdinals(spark.sessionState.conf)
val logicalPlanWithUnresolvedOrdinals = rule.apply(logicalPlan)
scala> println(logicalPlanWithUnresolvedOrdinals.numberedTreeString)
00 'Sort [unresolvedordinal(1) ASC NULLS FIRST], true
01 +- 'Project ['id]
02 +- 'SubqueryAlias t1
03 +- 'UnresolvedInlineTable [id], [List(1), List(2), List(3)]
import org.apache.spark.sql.catalyst.plans.logical.Sort
val sortOp = logicalPlanWithUnresolvedOrdinals.collect { case s: Sort => s }.head
val sortOrder = sortOp.order.head
import org.apache.spark.sql.catalyst.analysis.UnresolvedOrdinal
val unresolvedOrdinalExpr = sortOrder.child.asInstanceOf[UnresolvedOrdinal]
scala> println(unresolvedOrdinalExpr)
unresolvedordinal(1)UnresolvedOrdinal Unevaluable Leaf Expression
UnresolvedOrdinal is a leaf expression that represents a single integer literal in Sort logical operators (in SortOrder ordering expressions) and in Aggregate logical operators (in grouping expressions) in a logical plan.
UnresolvedOrdinal is created when SubstituteUnresolvedOrdinals logical resolution rule is executed.
UnresolvedOrdinal takes a single ordinal integer when created.
UnresolvedOrdinal is an unevaluable expression and cannot be evaluated (i.e. produce a value given an internal row).
|
Note
|
An unevaluable expression cannot be evaluated to produce a value (neither in interpreted nor code-generated expression evaluations) and has to be resolved (replaced) to some other expressions or logical operators at analysis or optimization phases or they fail analysis. |
UnresolvedOrdinal can never be resolved (and is replaced at analysis phase).
|
Note
|
UnresolvedOrdinal is resolved when ResolveOrdinalInOrderByAndGroupBy logical resolution rule is executed.
|
UnresolvedOrdinal has no representation in SQL.
|
Note
|
UnresolvedOrdinal in GROUP BY ordinal position is not allowed for a select list with a star (*).
|