Powered by GitBook

Window Utility Object — Defining Window Specification

Window utility object is a set of static methods to define a window specification.

Table 1. Window API
Method	Description
`currentRow`	`currentRow: Long` Value representing the current row that is used to define frame boundaries.
`orderBy`	`orderBy(cols: Column): WindowSpec orderBy(colName: String, colNames: String): WindowSpec` Creates a WindowSpec with the ordering defined.
`partitionBy`	`partitionBy(cols: Column): WindowSpec partitionBy(colName: String, colNames: String): WindowSpec` Creates a WindowSpec with the partitioning defined.
`rangeBetween`	`rangeBetween(start: Column, end: Column): WindowSpec rangeBetween(start: Long, end: Long): WindowSpec` Creates a WindowSpec with the frame boundaries defined, from `start` (inclusive) to `end` (inclusive). Both `start` and `end` are relative to the current row based on the actual value of the `ORDER BY` expression(s).
`rowsBetween`	`rowsBetween(start: Long, end: Long): WindowSpec` Creates a WindowSpec with the frame boundaries defined, from `start` (inclusive) to `end` (inclusive). Both `start` and `end` are positions relative to the current row based on the position of the row within the partition.
`unboundedFollowing`	`unboundedFollowing: Long` Value representing the last row in a partition (equivalent to "UNBOUNDED FOLLOWING" in SQL) that is used to define frame boundaries.
`unboundedPreceding`	`unboundedPreceding: Long` Value representing the first row in a partition (equivalent to "UNBOUNDED PRECEDING" in SQL) that is used to define frame boundaries.

import org.apache.spark.sql.expressions.Window
import org.apache.spark.sql.functions.{currentRow, lit}
val windowSpec = Window
  .partitionBy($"orderId")
  .orderBy($"time")
  .rangeBetween(currentRow, lit(1))
scala> :type windowSpec
org.apache.spark.sql.expressions.WindowSpec

Creating "Empty" WindowSpec — `spec` Internal Method

spec: WindowSpec

spec creates an "empty" WindowSpec, i.e. with empty partition and ordering specifications, and a UnspecifiedFrame.

Note	`spec` is used when: Column.over operator is used (with no `WindowSpec`) `Window` utility object is requested to partitionBy, orderBy, rowsBetween and rangeBetween

results matching ""

No results matching ""