Window Utility Object — Defining Window Specification

Window utility object is a set of static methods to define a window specification.

Table 1. Window API
Method Description

currentRow

currentRow: Long

Value representing the current row that is used to define frame boundaries.

orderBy

orderBy(cols: Column*): WindowSpec
orderBy(colName: String, colNames: String*): WindowSpec

Creates a WindowSpec with the ordering defined.

partitionBy

partitionBy(cols: Column*): WindowSpec
partitionBy(colName: String, colNames: String*): WindowSpec

Creates a WindowSpec with the partitioning defined.

rangeBetween

rangeBetween(start: Column, end: Column): WindowSpec
rangeBetween(start: Long, end: Long): WindowSpec

Creates a WindowSpec with the frame boundaries defined, from start (inclusive) to end (inclusive). Both start and end are relative to the current row based on the actual value of the ORDER BY expression(s).

rowsBetween

rowsBetween(start: Long, end: Long): WindowSpec

Creates a WindowSpec with the frame boundaries defined, from start (inclusive) to end (inclusive). Both start and end are positions relative to the current row based on the position of the row within the partition.

unboundedFollowing

unboundedFollowing: Long

Value representing the last row in a partition (equivalent to "UNBOUNDED FOLLOWING" in SQL) that is used to define frame boundaries.

unboundedPreceding

unboundedPreceding: Long

Value representing the first row in a partition (equivalent to "UNBOUNDED PRECEDING" in SQL) that is used to define frame boundaries.

import org.apache.spark.sql.expressions.Window
import org.apache.spark.sql.functions.{currentRow, lit}
val windowSpec = Window
  .partitionBy($"orderId")
  .orderBy($"time")
  .rangeBetween(currentRow, lit(1))
scala> :type windowSpec
org.apache.spark.sql.expressions.WindowSpec

Creating "Empty" WindowSpec — spec Internal Method

spec: WindowSpec

spec creates an "empty" WindowSpec, i.e. with empty partition and ordering specifications, and a UnspecifiedFrame.

Note

spec is used when:

results matching ""

    No results matching ""