SortOrder Unevaluable Unary Expression

SortOrder is a unary expression that represents the following operators in a logical plan:

SortOrder is used to specify the output data ordering requirements of a physical operator.

SortOrder is an unevaluable expression and cannot be evaluated (i.e. produce a value given an internal row).

Note
 An unevaluable expression cannot be evaluated to produce a value (neither in interpreted nor code-generated expression evaluations) and has to be resolved (replaced) to some other expressions or logical operators at analysis or optimization phases or they fail analysis.

SortOrder is never foldable (as an unevaluable expression with no evaluation).

Tip
 Use asc, asc_nullsLast, desc or desc_nullsFirst operators from the Catalyst DSL to create a SortOrder expression, e.g. for testing or Spark SQL internals exploration.
Note
 Dataset.repartitionByRange, Dataset.sortWithinPartitions, Dataset.sort and WindowSpec.orderBy default to Ascending sort direction.

Creating SortOrder Instance — apply Factory Method

apply(
  child: Expression,
  direction: SortDirection,
  sameOrderExpressions: Set[Expression] = Set.empty): SortOrder

apply is a convenience method to create a SortOrder with the defaultNullOrdering of the SortDirection.

Note
 apply is used exclusively in window function.

Catalyst DSL — asc, asc_nullsLast, desc and desc_nullsFirst Operators

asc: SortOrder
asc_nullsLast: SortOrder
desc: SortOrder
desc_nullsFirst: SortOrder

asc, asc_nullsLast, desc and desc_nullsFirst create a SortOrder expression with the Ascending or Descending sort direction, respectively.

import org.apache.spark.sql.catalyst.dsl.expressions._
val sortNullsLast = 'id.asc_nullsLast
scala> println(sortNullsLast.sql)
`id` ASC NULLS LAST

Creating SortOrder Instance

SortOrder takes the following when created:

SortDirection Contract

SortDirection is the base of sort directions.

Table 1. SortDirection Contract
Method Description

defaultNullOrdering

 
defaultNullOrdering: NullOrdering

Used when…​FIXME

sql

 
sql: String

Used when…​FIXME

Ascending and Descending Sort Directions

There are two sorting directions available, i.e. Ascending and Descending.

results matching ""

    No results matching ""